The UK’s public data tsunami gathers speed
Thursday, June 17, 2010
When the British Government said at the beginning of May that they would be releasing a ‘tsunami of public data’ you had to wonder whether reality would match up to the rhetoric. Oh ye of little faith... A fortnight ago, the Government released hundreds of new datasets - including a full list of Government expenditure - and this week, Transport for London announced that they too would be releasing lots of their transport data for free to the public.
The policy itself is not new. It was kicked off in the UK back in 2009 by Sir Tim Berners Lee and has resulted in the establishment of the data.gov site as the primary place for public data to be accessed. But the new Government has sent some strong signals from the top that it is an even greater priority for them to get government departments and agencies to shift from being information hoarders to information sharers.
This new mood makes it a really exciting time for the data geeks amongst us in the UK. It’s clear that the new British Government is getting as excited about data as the developer community itself, and that it is determined to be a European - and perhaps even global - leader in transparency. Countries around the world are waking up to the huge potential for re-use of data financed by the taxpayer. There are good social and democratic arguments behind this policy - but it also makes economic sense. If Governments want to stimulate the national growth of their local Internet economy, making data public for developers to reuse is a great way to build skills and position the country for data based business models of the future.
But, as we all know, the usability of the data is just as important as the data itself. To the ordinary citizen, a gargantuan list of numbers means nothing. Data only becomes useful when it is rendered accessible to the citizen: the task traditionally of statisticians and, increasingly, creative web developers who ‘mash’ different data sets, drop them into data crunching tools and turn them into citizen friendly applications. From Tube schedules to postcode databases, information works best when it can be overlaid with other datasets and correlations can be made, as services such as those created by the Government’s new Transparency Advisor, Tom Steinberg.
Some departments and agencies have been better than others at releasing data in truly useful ways. A machine readable format rather than in a PDF is preferable - so that developers spend as little time as possible manually reformatting and more time innovating. And there should be as few ‘strings’ attached as possible so developers can easily reuse it, mash it up, and create new tools and services. These details matter - and the next step for the UK officials behind the public data agenda is to ensure that all data released has consistently high standards of usability.
We hope this is the start of something big and that, rather than a one-off tidal wave, this becomes an ever flowing river of information that is released.
Posted by Sarah Hunter, UK Policy Manager.
The policy itself is not new. It was kicked off in the UK back in 2009 by Sir Tim Berners Lee and has resulted in the establishment of the data.gov site as the primary place for public data to be accessed. But the new Government has sent some strong signals from the top that it is an even greater priority for them to get government departments and agencies to shift from being information hoarders to information sharers.
This new mood makes it a really exciting time for the data geeks amongst us in the UK. It’s clear that the new British Government is getting as excited about data as the developer community itself, and that it is determined to be a European - and perhaps even global - leader in transparency. Countries around the world are waking up to the huge potential for re-use of data financed by the taxpayer. There are good social and democratic arguments behind this policy - but it also makes economic sense. If Governments want to stimulate the national growth of their local Internet economy, making data public for developers to reuse is a great way to build skills and position the country for data based business models of the future.
But, as we all know, the usability of the data is just as important as the data itself. To the ordinary citizen, a gargantuan list of numbers means nothing. Data only becomes useful when it is rendered accessible to the citizen: the task traditionally of statisticians and, increasingly, creative web developers who ‘mash’ different data sets, drop them into data crunching tools and turn them into citizen friendly applications. From Tube schedules to postcode databases, information works best when it can be overlaid with other datasets and correlations can be made, as services such as those created by the Government’s new Transparency Advisor, Tom Steinberg.
Some departments and agencies have been better than others at releasing data in truly useful ways. A machine readable format rather than in a PDF is preferable - so that developers spend as little time as possible manually reformatting and more time innovating. And there should be as few ‘strings’ attached as possible so developers can easily reuse it, mash it up, and create new tools and services. These details matter - and the next step for the UK officials behind the public data agenda is to ensure that all data released has consistently high standards of usability.
We hope this is the start of something big and that, rather than a one-off tidal wave, this becomes an ever flowing river of information that is released.
Posted by Sarah Hunter, UK Policy Manager.
I'm interested to see what data the Government can provide under this new banner: something more than static small datasets, prettily presented. What more than ONS data can it provide that would move towards openness and Google style data mining?
ReplyDeleteHi sarah - we've been toiling away on open data in the UK since the Power of Information work in 2007,
ReplyDeletehttp://www.opsi.gov.uk/advice/poi/power-of-information-review.pdf
and it's great that the new coaliton government is giving the work a big boost.
There are some basic things you can do in a community with raw open data and a calculator to drive better public service outcomes. See this from 2008
http://www.kingscrossenvironment.com/2008/02/not-blinded-by.html
The London Data store is also a goldmine of useful information that you can handle with basic excel skills that are present in most small business for instance. See this rather grim league table that took tne minutes with a spreadsheet
http://www.kingscrossenvironment.com/2010/01/arson-incidents-in-islington-1.html
Whilst in one dimension you can only go so far before the 'middle ware' level of processing is needed, there is a huge amount that can be done at a more simple level by people with basic math skills who just need to know that the data it there.
The real potential for the UK and any other country is to build use of open data into social science studies in further and higher education. Simple exercises could easily be set for older school children studying economics, politics or geography and there is no limit to the potential for undergraduates.
Can google perhaps help with this?
cheers
william
Rosslyn Analytics, a London-based technology company that specialises in spend analysis, is passionate about making the government’s public sector expenditure data available to the public (and public sector employees) with easy-to-view and use web-based reports. People simply need to visit RA.Pid Gateway at https://hmrc.rapidintel.com.
ReplyDelete