What’s the benefit of gov data warehouses?

In truth, no one really knows as yet but early evidence emerging from the US indicates that initial hopes may have been overly optimistic.

The US government launched a massive data warehouse initiative under data.gov brand in 2009. The objective was to put most, if not all, information gathered by various government departments and agencies into the hands of private sector and nonprofit Web and mobile app developers. To jumpstart rapid uptake of that data and provide extra incentive to developers the government sponsored a myriad of code-a-thons and app development competitions through Challenge.gov, paying out $38 million to prize-winning developers in the first year. But, as a recent article from Nextgov suggests, “turning government data into private sector products has proved more complicated in practice”.

As this author suggests, the key problems are that many agencies show little interest in devoting dwindling resources to making data more accessible. Agency data publication schedules also are often too slow for the go-go world of mobile apps. In some cases, agencies publish data in difficult-to-manipulate forms such as PDFs, significantly increasing the upfront work for developers who then have to create and organize spreadsheets. And agencies often release new information in a different structure, making it inconsistent with historic information.

Following the example of US government, Australian and British governments also embarked on similar initiatives, and State governments in Australia followed with their own localised versions. The end result is a proliferation of data catalogues with varying degree of useability. However, the biggest hurdle with government data isn't finding it, but getting it quickly and in a form that can be put to immediate use.

All in all, despite a massive effort and significant resources spent on enabling online access to data, many users still prefer to deal with individual agencies to get fast access to that data in a useable format, even if it is delivered via snail-mail on a CD-ROM. Because of this, getting a clear picture of the benefits of establishing data warehouses with free public data will be rather difficult to determine, with a fair dose of accuracy.

Related posts:
Free data a GFC casualty
No copyright on databases and maps?
Governments intensify fee data efforts
UK unlocks vaults of spatial data