20 September 2011

The Value of the Seal

The implementation of government open data initiatives presents a number of interesting challenges to governments on the leading edge of Gov 2.0 policy making. Established business practices invented in another age sometimes seem impervious to ideas like open data. One such practice is charging money for public data under the banner "cost recovery".

When looking at the bureaucracies and systems required to manage the authorized distribution of and the collection of payments for data (i.e. products whose marginal cost of distribution is effectively zero) it's easy to wonder if the operating costs are mainly due to the billing apparatus itself.

But let's take the case where there is in fact a cost to distributing this data. Perhaps there is a collection cost or some sort of initial investment cost. In some cases, governments have chosen to offset that tangible short term cost with a fee for access, to either the data itself or some byproduct of that data.

People or organizations are willing to pay money for this service of data retrieval. From this, one could deduce at least two things: 1) provision of data is a valuable service; and 2) the government is the considered the best or perhaps the only place where they can get this data.

Let's look at the BC Corporate Registry for example. To search for a registered company in BC I have to first become a member of BC Online. To become a member of BC Online I have to first deposit $100 into a BC Online account, and then to perform a search they will deduct $8.68 from my account. People do use this service, so there is some value. People are willing to pay for this data.

Or, are they?

What if BC Corporate Registry data was released as open data. The deal is, I would have access to that data to do what I want with it, including creating an app, and making money, the only condition being that I could not claim it was official.

So, I get the data, I offer the data to lawyers and corporate deal makers and entrepreneurs and whoever else I think can use this data and I offer to sell it to them for 1/2 of what the government currently charges for exactly the same thing except that my data comes with a clause that says something like "this data is 'not official'". My guess is that no one would take me up on that offer. In fact, my guess is that I wouldn't be able to give that data away for free. Why? Because it's not official. It turns out being official is pretty important.

I think that this whole model of people paying for public data is a collapse of two concepts and what's missing is a distinction. People do not pay for public data. People pay for data with an "official seal". I would go one step further. Often the seal is all they are paying for.

This is not a new idea. Just as I can get a certified engineer to come over to my house and talk with me for an hour about the structural changes I want to make to my house, and she'll give me all sorts of ideas and draw me a sketch and perhaps help me with my design ideas, so long as it's all "unofficial". Some will even provide me with an non-certified printout of my plans. To build my house though, I need the official certified plan, and for that there is a fee. I am not paying the engineer for the plan - I am paying her for the plan stamped by a certified engineer.

I suspect this won't even be a surprise to the folks involved in crafting the open data licenses that we use. The clue is in the licenses themselves. For example, if you look at the Government of B.C.'s new Open Government License you will see in clause 5. b) "ensure that you do not use the Information in a way that suggests any official status or that the Information Provider endorses you or your use of the Information".

That clause is there because someone recognized that that's where the value is. It's the official status, or said another way, the authenticity that the government brings to the data that people need and are willing to pay for and often for legal reasons must obtain.

My personal preference would be to have all publicly funded data free and available for any lawful purpose by default. The question of whether or not people should have to pay for the official status of anything is a separate and worthy question. In any case, fees can stay in place for "official" data as there really isn't any threat to existing business models. Releasing the data itself can happen using existing policy and the existing licenses. And, my guess is that in time governments will discover what open source and many other folks already know, that by opening the data, the demand for paid services will actually increase.

And for some young entrepreneur, who wants to build the next iPhone app to accelerate the process of creating a new business in BC, that in turn could create jobs and attract talent to our province, that's good news. Because his app doesn't actually require that the data have the official seal.

He just needs data he can use.

08 August 2011

Internet Ready Music

I love music. I love listening to it, and I love creating it.

I used to buy Vinyl records, then switched to CDs and now I buy mostly online. I have created mp3 files of most of my CDs so I can listen to them in one big huge playlist without ever putting in a CD.

A couple of months ago, Google invited me to use their new Google Music Beta. It's awesome. With it I can easily place all of my music on their servers and then access that music from any of my computers or Android devices.

But I haven't done that. Why? Because the record companies have been attacking people use the internet to enjoy music. And, while I don't think it breaks any laws to use Google Music Beta, I am not 100% sure. Even if it's 100% legal today, with the music industry lobbying governments and influencing law makers and suing students, and now getting the laws changed so that ISPs are required to spy on us, it's not inconceivable that they could convince law makers to make my uploading of music to Google Plus illegal, retroactively. And since once I upload music, there is really no way to make sure it's removed from wherever I upload it, uploading is an action that I can't really undo.

It seems more and more that the vast majority of digital music that I have purchased thus far is becoming legally incompatible with internet technologies and it's actually becoming increasingly risky to own it at all. My iTunes purchases have declined as I have become more aware of the associated risks of owning music that's incompatible with the internet.

Just like driving a horse and buggy on a modern freeway today would be considered dangerous, using music locked into outdated copyright schemes that prohibit uploading on something designed for uploading and downloading is dangerous.

Some of the music I own, however, is compatible with the internet. Bands such as The Charlatans, Radiohead, Nine Inch Nails, and others have been making their music available for free on the internet for some time.

I have also recently (re-)discovered Jamendo which is a music site that contains 300,000+ songs from all over the world and in all genres, all of which are available for downloading and can be uploaded and shared freely with others using the Creative Commons license.

With Jamendo I have discovered some new music that I am totally hooked on, such as the fantastic Canadian artist Brad Sucks, who you can currently find me listening to most days.

Since the internet is all about sharing, I am starting to call this new music, "internet ready". I can upload it to Google Music Beta without worrying about any scary legal incompatibilities down the road, and I can share it with Friends.

Incidentally, I still pay for much of this "free" music. Usually $20 an album. I like that the whole $20 goes to the artists because I want them to be able to continue to create great music. But mostly, it's worth more to me than the risky legacy "recording industry" music because it's internet compatible and as such I am free do so much more with it.

20 July 2011

About the License

Yesterday while we were all celebrating the awesome work of the BC Provincial government, a very well respected Harvard researcher, software developer and open data advocate was arrested by the US Federal government on charges related to computer hacking, based on allegations that he downloaded too many scholarly journals that he was entitled to get for free.  He now faces a possible 30 years in jail.

I don't know Aaron, but I feel as though I do.  He writes brilliant software code and releases it to the world as completely free public domain software.  He advocates for open data and transparency and democracy in the US and he is a founder of demandprogress.org an organization dedicated to progressive policy changes for ordinary people.

Please consider visiting demandprogress.org and reading about what's happening to Aaron.

How does this relate to open data and the new DataBC portal?

It's about the license.

License : official or legal permission to do or own a specified thing.

It's strange that we need permission to use our own data and information.  It's strange that we sometimes have to pay to use data that we have already paid to have created.

Being an open data advocate and application developer comes with a certain level of risk and anxiety.  We are actively trying to do things that have not been done before.  And unlike other areas where innovation takes place, we are innovating in an area involving strange legalities, usually as individuals with no corporate backing or protection, and the consequences of making a mistake can be severe.  There is a lot of uncertainty.  Although many of us are open source developers and thus are pretty familiar with copyright law, licensing and the jargon that goes along with them, very few of us are lawyers.

This is why we want governments to use standard licenses.

By choosing to invent a new license rather than use an existing one the BC Government has added to the uncertainty.  Yes, they based it on the UK license, but it's clearly not the same as the UK license, otherwise they could have just used that.

Because they chose to invent a license I spent several hours last night pouring over the license and comparing it to both the PDDL and the UK license to see where those differences are and to see what additional risk I might have to take on as a result.  Every developer I know will have to do the same thing now before they start using the data.

Many won't bother.

And that's the lost opportunity. People who get turned off by the custom license won't use the data, or won't bother coming to the hackathons, or won't bother creating that new app. It's just too risky. Sadly, we all lose. because as I understand it, the BC Provincial government is in this for the right reasons.  It's clear to me that they absolutely get it.  Innovative new ideas and applications will be generated as a result of this.  This increased transparency and engagement and collaboration with the citizens will build trust and goodwill and is good for the government and good for the people of BC.

From what I can tell, not being a lawyer, as a standalone license, the BC Open Government License is actually mostly good (check out unrest.ca for some of the details on issues with the license).  And, there are a handful of us, that will push through this licensing thing, grumble a bit, say "it's pretty good" because it is, and weigh our risks and move forward with our apps and visualizations and innovations.

BC is seen as a leader in citizen engagement and open data by local governments, other provinces and internationally.  Looking at what's going on in the rest of the world, particularly in the US, we are really very fortunate to live where we do and to have the public service and leaders that we have.

I will encourage others to take the time to read the BC license rather than blowing it off because it's not standard.  And I will continue to urge local governments and other provinces to use a standard license rather than invent their own.

This is a first great first attempt, and as Christy Clark said in her excellent and encouraging video, this is very much a work in progress.  The license does have a version number, which to me implies that they are open to input, discussion and changing it if necessary, which is awesome.

19 July 2011

Remembering "Open"

Today the British Columbia Provincial Government launched a new Open Data Portal, making thousands of our publicly owned datasets available to us including everything from employee salaries to historical school locations to Local Government Incorporation Dates to the data catalogue itself. As a citizen and taxpayer in BC and an open data advocate I am very excited to see my own Provincial Government take these steps toward innovation and transparency. I congratulate those public servants within the BC Government that understood the opportunity, recognized value and championed the cause.

These days, governments all over the world are starting realize the value of "open" but it wasn't so long ago that we were at the opposite end of the spectrum. As a public servant in the 1980's, employed as a junior data analyst, I personally produced the monthly report for the minister and deputy minister showing the basic metrics of the ministry I was working for. As part of that job I was required to produce 5 copies of that report and place them in brown paper bags and tape them closed. I would then personally attempt delivery to the recipients office. If the recipient or their assistant weren't there to receive the report, I was required to take the report with me and try again later.

Those same metrics are still being used today and were released today as part of the Provincial Open Data portal. It's striking how far we have come in such a short amount of time.

Although we have gone through a very opaque phase with our governments, the idea of governments being open and transparent is not actually new. Our own BC Government has been publishing the public accounts and other financial information for decades. They also produce a monthly publication called the British Columbia Gazette and have done so since the 1920's that is teeming with useful information about our province, from disposition of Crown Lands, to election results, to tree farm licenses to road name changes.

Technology has evolved since these publications were originally developed. Where at one time publishing this type of data on paper was about as usable as one could have imagined, these days its available electronically, and hopefully soon it will be included as part of the open data portal.

So, although openness and transparency aren't new, they were definitely forgotten, and as I like to say, we are now remembering the value of "open".

Congratulations and a big "Thank you!" to our public service employees and political leaders who are helping to make this happen.

Now, I need to go to the portal and look for some data. :)

30 June 2011

Government as Platform - An Example

Tim O'Reilly uses the words "government as platform" to describe an interpretation of what Government 2.0 is. I am often asked what "government as platform" means. I think the question arises because much of government already operates as a platform. People don't distinguish it as something new because it's routine.

To explain what government as platform is, you can look at an example where it's already the norm such as our public roads. Our three levels of government are involved in the construction, maintenance and regulation of roads. Together they deliver infrastructure upon which we ride our bicycles and drive our cars.

We are used to the idea that we can hop on our bicycle or get in our car and and transport our selves on public roads. As a platform we are left to decide where we are going, when and how to get there. The outcomes of the platform, the actual transportation conducted, is not determined beforehand. The roads are built to certain standards and the drivers are left to figure out how to get from point A to point B.

And because the system is open, people and companies can and do invent innovative new ways to use the platform. Because they are free to choose their route, people optimize the routes they use to get from point A to Point B.  Inventions like automobile GPS are created to help with navigation.  Every year the entire vehicle industry (bicycles, cars, buses) releases new versions of their transportation products and introduces innovative new ones. Several huge industries can rely on the fact that our roads systems is a platform that they can build on.

And for travelers, the system works so well we can travel almost anywhere and we're usually not even aware of which level of government is responsible for which roads. We can even travel to different countries and travel, because all of the public roads work pretty much the same way. Done well, the platform becomes transparent. We don't even notice it.

I am convinced that this what we need with our public data. Our data – all of it – made freely available on the internet, with standard licensing, in formats we can use - would provide a platform for innovation like we've seen with transportation. Entire industries could grow on such a platform, providing jobs and value we can barely imagine right now.

And though only a few people will use the data in the beginning, like the few people building cars and bicycles, those people will create huge value for their fellow citizens all built on a platform managed by our governments.

17 March 2011

Selling Data - Privacy Is Not The Issue

When data can't be released it's usually for one of two reasons, privacy or cost. Leaving cost for a future post, let's focus on the privacy issue.

We don't want our personal medical records released to the public for free or for cost, for example. Most people I talk to can agree on that. In fact, in BC, we have laws that state that personal information collected for one purpose cannot be subsequently used for a different purpose. There are exceptions, but that's the general idea.

Sometimes governments sell data to the public. Sometimes that data contains people's contact information. Sometimes it's about organizations or places. But, before it can be sold, this data typically undergoes rigorous processes and checks to ensure that no personal data is compromised.

For example, for $32 anyone can request a business name search from BC Registry Services to find out if a company exists . For "as low as $5,250" anyone can purchase a business site license for postal code address data from Canada Post. And, anyone can download a wide variety of key socioeconomic data from Statistics Canada - for a price.

How does this relate to open data? Well, sometimes, privacy is held up as a reason for why some data that is currently sold by governments, cannot be freely released as open data. As if, somehow, paying money for that data alleviates any privacy concerns. It doesn't, because there aren't any. If there were, not only could it not be made available as open data, it also could not be sold.

While there may be very good reasons why data that is currently for sale by governments cannot be made available for free, privacy isn't one of them.

10 February 2011

Requirements of Open Data

As more and more governments start to realize the benefits of opening up their data, sometimes I hear chatter about enterprise solutions and though I understand the logic of hitching the wagon the the latest thing, I think there is a lot to gain by taking a strategic and pragmatic view of things. Open Data does not have to be an expensive exercise. In fact it can be very inexpensive from the data publisher's point of view.

There are three main requirements that have to be satisfied before data is considered "open data". They are:

1. Legal Framework - Anyone can use it for any legal purpose (PDDL or CC0 license)
2. Accessible - I can download it on the internet free of any mechanisms of control
3. Readable - it's published in a non-proprietary format

Of these three, the first requirement is most important and costs the least. And, in fact, without the first one there is really no point in doing the other two. If it's not legal for me to use the data then it doesn't matter what format it's in or whether I can get my hands on it... I won't use it.

The great thing is though that governments in some cases have already done the other two steps so the Legal Framework is all that's left to do. And, if they do that, then instantly and without any expensive technology, a raft of published data becomes "Open Data", ready to use.

For example, the City of Courtenay publishes a great RSS feed of "Surplus Equipment for Sale" but nowhere on their site does it say that I can use it. It's both accessible and readable so it's satisfying two of the three requirements but the first requirement isn't met so developers would likely shy away from using it. That's too bad because an app that went around and gathered up this type of information from all local governments and made it available as an mobile app would be pretty cool and would help the local governments sell their surplus equipment.

The BC Government on the other hand has this page which looks pretty much what it looked like over 10 years ago, and pages like this that basically say you can't use this data to make anything.

Here again, there are many examples of data that is both accessible and readable, but because of these pages, we can't use it. And that's unfortunate for both the government organizations that could benefit from the huge talent pool outside of government and for the citizens who pay for the data.

The good news is that governments already have a large amount of data online, and by getting the legal framework sorted out, citizens will instantly be able to use it to create innovative solutions and tools to help themselves and others.

04 January 2011

Book: Rework

I just finished reading Rework by the 37 Signals team (Jason Fried and David Heinemeier Hansson), a Christmas gift from my youngest son.  If you are not familiar with 37signals and their work I recommend checking them out.

The book provides an outline of the 37signals philosophy including tips and opinion.  It is aimed at small businesses, however, I think everyone from small business to large business to government organizations should read this book and think about how it might be applied to their work.

They don't say this explicitly but what the book is really about is rethinking some of the dogma of business-as-usual.  I have often thought that any organization that has an "innovation" branch is already in trouble.  If you read this book you'll find out why.  An innovative culture isn't something you can install, or force directly.  Innovative cultures happen by consistently rewarding innovation.  Sharing cultures happen by consistently rewarding sharing.  Organizations that consistently treat employees as untrustworthy, end up with a culture of fear and lack of trust.

For examples of how to do it right in a government context, I think the Province of BC is on the right track with their new B.C. Government Social Media Guidelines and their public statement of Open Data as Defining Principle No. 1 of their Citizens @ the Centre: B.C. Government 2.0 strategy. The very fact that these documents exist sends a signal to employees of a relatively large organization that they are trusted and empowered to engage citizens and empower citizens to create value from open government data.  This kind of positive reinforcement will go a long way to creating a culture of learning and trust as people come up to speed with the new tools of social media and open data.  Kudos to the folks that made this happen and the executives that supported them.  I look forward to seeing what they do next.

Creating an environment where innovation happens, where sharing is rewarded, where great work is recognized and where trust is leveraged is the hallmark of an organization that gets it.  37signals definitely gets it.  Rework is an easy and worthwhile read.  If you're interested in innovation in the workplace, I recommend you read it.

01 January 2011

Shipped in 2010

One of my favorite authours of all time is Seth Godin.  I purchase and read every book he writes and often give them as gifts.  One of the things Seth talks about is "shipping".  We use this term to describe the act of completing a project, getting it out the door.  It doesn't matter if it was a hit or not, it just matters that it's done.

In a recent blog post Seth encourages people to publish their list of things they shipped last year, because it's not something we often do.  I encourage everyone to 1) make your own list; and 2) read Seth's blog.

Here is a list of things that I shipped last year:
  • Launched OpenDataBC.ca
  • Identified 149 BC datasets from all levels of government
  • Created OpenDataBC Google Group which now has 70+ members
  • Established Open Government conference for BC
  • Held two hackathons for provinicial Apps for Climate Action Contest
  • Created Waterly.ca which won two awards including Best in BC (Yay!)
  • Participated in Google IO and OSCON conferences
  • Spoke on Open Data at the Ideawave conference
  • Sat on an Open Data panel at the Global Knowledge eGov conference
  • Accepted a CTO position with an exciting new Canadian startup
  • Held the first annual Victoria International Open Data Hackathon
  • Developed DataZoomer version 4
  • 25 blog posts
  • 500+ tweets
This isn't the entire list of things I worked on.  I worked on many other things that either failed or that I didn't ship (yet).  I also didn't do this alone.  I was fortunate to be able to work with a bunch of dedicated and talented people this year.

2010 was a great year of learning and I look forward to an exciting 2011.