Saturday, July 7, 2012

Launching Census 2011 Online Maps

The first set of 2011 Census of Population and Housing data was officially released on 21 June 2012. This rich source of geographically referenced demographic statistics is a treasure trove for any spatial data and mapping enthusiast. Best of all, the data is totally free (but a small fee applies for convenience of having it all on a DVD, and right away).

I was hoping that there will be a mad rush of releases of various mapping applications with demographic statistics post June 21st but a quick search on the internet for “census maps” is not turning up much as yet. So far only general media put some effort into making census statistics more accessible with interactive presentations and simple maps - I listed the most interesting examples in my previous post. How disappointing. The official ABS version of online maps – TableBuilder, is still a few months away (free Basic version will be released on August 21, 2012; if you are prepared to pay $750 for an individual licence or $9,950 for a corporate licence, TableBuilder Pro version will be available to you on August 7, 2012). As someone twitted recently, census data release day felt like a total anticlimax…

So, to fill the void, I decided to rush the release of my own version of Census 2011 Online Maps. There are over 70 census data items available in the first release of the app (just under 150 if derived information is counted). By popular demand, postal area boundaries are the default geography (and the only option a the moment but more will be added later).

Unlike some traditional online GIS applications where every single map is preconfigured upfront, my Census 2011 maps are generated “on the fly” so, users have a lot more flexibility in customising map parameters and visual presentation (such as, the number of data classes and values of data ranges, colours of thematic drapes, locations of interest – again, more options will be added in the future).

This flexibility is very important in order to enable creation of meaningful thematic maps since there is such a large volume of data available for mapping (Basic Community Profile dataset has close to 8,000 data items!) and secondly, because each dataset, representing either counts, $ values or proportions, has very different characteristics. For example, a handful of datasets can be characterised as having normal distribution, there are some approximating uniform distribution but most have irregular distribution. And quite a few datasets contain extreme values/outliers which are “off the scale” (for example, how about a $132,500 median weekly household income for postcode 2139 or median weekly personal income of $8,312 for postcode 6710!). Single, automated approach to map such a diverse set of data will simply not work. Therefore, I included data histogram to help in selecting the most appropriate classification method (ie. either Standard Deviation, Equal Range, Quantiles or Natural Break – Jenks algorithm). More on how to make the most of available selectors in Census 2011 Online Maps User Guide which is published as a separate post.

Census 2011 Online Maps application is built with free Google Map, Fusion Tables and Visualisation APIs so, no infrastructure for me to look after! Most of the code has been recycled from various tools of mine shortening the development time. Census 2011 Online Maps actually comprises of two separate parts: map configuration module and an updated version of my free-standing reference map (available as a Web Application Service to anyone – read more on the concept of WAS in one of my earlier posts). So, for all practical reasons, Census 2011 Online Maps application is more like a “map creation" or "thematic mapping tool” that allows selection and configuration of raw data, stored in Fusion Tables, for display on a Google Map.

If you really like a particular map you have created, just detach it from the main page and bookmark it, or share with others! Or embed it in your web site… That portability of maps is a special feature of Census 2011 Online Maps application – users are no longer confined to visit my site to take advantage of the information. 

The volume and complexity of Census data is probably the reason why not many online maps have been released so far. In all honesty, not so long ago, desktop applications with functionality similar to my Census 2011 Online Maps used to cost tens of thousands of dollars. And online versions costed many more times over that due to the cost of infrastructure. Today anyone can create online mapping applications - for free…  I hope that my version will prove a popular choice. Maybe even more popular than a related Postcode Finder app! Help me spread the word so it can reach all those users you cannot afford pricey commercial GIS solutions. Your feedback and feature requests most welcome. The code is available to a selected few and/or can be customised to meet your specific business requirements.

Related posts:
Census 2011 Online Maps User Guide
Mapping Australian social diversity
Mapping social diversity in NSW

More free data with reference map
Web Application Service concept


Eric said...

The Data Packs became available this week. A more convenient format than the plain profiles, but I'll still be waiting for TableBuilder for most of what I'll do.

For the 2006 data, I had a Perl script that trawled through hundreds of spreadsheets getting the figures I wanted. No need for that now!

What do you use to get the boundaries in a convenient format?

In Mappage, I ended op with this rather messy algo to map from a scalar to 0-1.
2% and 98% percentiles map to the two ends of the colour range.
The figure that mapped to the middle colour value was the average of the 2%, the median (twice) and the 98%.
Then I do the same kind of thing again to get the vlaue that maps to the 1/4 and 3/4 colour values. It's a messy halfway between using a linear scale and using a percentile scale.

Arek said...

Hi Eric,

Yes, 2006 was a different story :-)

I use QGIS to generalise shp and convert to KML for upload to Fusion Tables.

Fusion Tables app supports only 5 colours via API so my colour schemes are fixed for now (selected by trial and error). To some extents it makes it easier but I wish there was a way to implement “unclassified” option as well.