Landslides in Australia

Geoscience Australia has just released a simple mapping application that allows members of the public to search the internal landslide database for events that have occurred across Australia since 1842. A landslide consists of one or more related “landslips”. Since landslips are a naturally occurring phenomenon, they are difficult to predict and can often occur suddenly without any witnesses. This makes it difficult to determine some of the properties of a landslip, such as the triggering and contributing factors, and when exactly it occurred. Subsequently, landslip event details are sometimes incomplete as the information could not be determined. This is particularly true for historical records.

The Landslide Search application is intended to raise public awareness about this particular type of natural hazard in Australia. Users can customise search according to location, time or impact of the event, and view the results on a map and in an expandable table.

Visualising census data on maps

2011 Census of Population and Housing data was officially released by the Australian Bureau of Statistics in late June 2011. Data is freely accessible on ABS site however, it is rather difficult to work with numeric tables alone to determine distribution of people with specific characteristics. Knowing geographic location of groups of interest can be invaluable in decision making for all variety of users: governments, businesses as well as individuals.

TableBuilder Basic, a free mapping application from ABS, will be released in August 2012 but you can start mapping and analysing of Census data right away: aus-emaps.com has just launched a free tool that enables creation of custom thematic maps with 70+ census data items from the Basic Community Profile. Below are just a few examples of what is available and how this information can be put to use.

Case 1: Locating areas with the greatest purchasing power

Personal incomes are indicative of propensity to spend so, areas with high median weekly income would be a primary target to concentrate promotional activities on if you want to attract customers for luxury items or high value investment products.

Case 2: Locating ethnic niches

Ethnicity related data can be used by various government departments delivering social services to profile their local customers in order, for example, to determine the need for personnel with specific language skills and/or adequate training to deal properly with cultural differences.

Case 3: Estimating market potential

Census data will help in determining if there is enough potential customers in a given location to support a business. For example, if you provide goods or services for infants and toddlers you would need to know how many are there in your catchment area.  

Case 4: Benchmarking investment performance

Property investors can take advantage of Census data related to housing. For example, median weekly rent statistics can help in determining whether their rental income is keeping up with the changes in the local market or not.

Explore the full set of data on Census 2011 Online Maps app.


Related Posts:
Census 2011 Online Maps User Guide
Mapping Australian social diversity
How maps can improve sales
Maps and property investment
Reference Maps for Fusion Tables

Reference Map for Fusion Tables

Reference Map for Fusion Tables is a free Web Application Service (WAS) provided by aus-emaps.com. The application enables displaying information stored in Google’s Fusion Tables as a thematic choropleth map. There are a few tricks you will need to know to put it to a good use. This post describes in detail how to configure Version 3 of the application to display custom choropleth maps (the concept of WAS is explained in an earlier post on this blog).

Firstly, some background information. Although Google included simple map configuration functionality within Fusion Tables, it is a manual process allowing formatting only one dataset at a time. If you have multiple columns of data available for mapping in a single table, the task of creating individual thematic maps can be quite a tedious process. Therefore, creating such maps programmatically, via Fusion Tables API, is the only viable option.

My recently released Census 2011 Online Maps application is an example of how to automate choropleth map creation process. There are two separate modules in that application. The first module enables selecting and analysing data stored in Fusion Tables, then formatting it for display as a thematic layer on a Google Map (ie. selecting number of classes, defining range break values and selecting colour scheme for thematic layer). The second module is a free-standing application, built with Google Map API, that displays the information from Fusion Tables as a thematic layer, formatted according to user requirements.

Now specifically about the mapping app. When designing Census 2011 Online Maps application I opted to incorporate Reference Map for Fusion Tables rather than coding mapping functionality from the scratch, but I had to update the application in order to enable proper formatting of numeric data retrieved from Fusion Tables. It also was an opportunity to expand the functionality of the reference map to enable generating thematic choropleth maps with 2, 3, 4 as well as 5 data ranges (five is a limit imposed by Fusion Tables API). All in all, the latest version of Reference Map for Fusion Tables now accepts a number of new parameters which enable quite extensive customisation of thematic layer for presentation on the map.

Below is a full list of accepted parameters and relevant options. Parameters flagged as “minimum” are compulsory in order to display Fusion Table in its “native style” (ie. as styled manually using relevant Fusion Tables functionality). “Optional” parameters can be omitted without degrading presentation of the information on the map. Please note, each parameter must be separated in compiled URL with “&” symbol.

Reference Map for Fusion Tables v3 – Acceptable Parameters:

ftl=  table id number eg: 1234567 [minimum]

q1c=geometry [minimum] - specifies which column contains spatial data

r1= defines the number of ranges for choropleth map: eg. 2, 3, 4 or 5

st1= thematic layer colour scheme selector, eg. b0=blue, pp0=purple, g0=green 2br0= blue-red; it is also possible to specify colours in reverse order as b0r, pp0r, g0r, 2br0r accordingly

qst1= SQL like data ranges specification; must correspond to the number of ranges specified for r1 parameter and be comma separated; column names must be between apostrophes; can  be either in ascending or descending order, eg.: 'Relevant Column Name' > 2030 AND 'Relevant Column Name ' <= 3035,'Relevant Column Name ' >= 0 AND 'Relevant Column Name' <= 2030  

q1=  SQL like WHERE query specification, to limit results to only specific set of data, eg. location: postcode > 1999 AND postcode < 3000  [optional]

n1= map title [optional]

xyz= allows to set map location and zoom level as longitude, latitude and zoom values, eg: 147.971361,-32.801441,6 [optional]

ad= can be used to hide ads on the map when set to 0, default is 1 [optional]


The following 3 parameters go together and are optional; each must contain the same number of variables separated by comma:

iw= allows to specify columns to be used to generate content of information window (“balloon”) that appears on the map when coloured polygons are clicked, eg. PC, D32 - if omitted the content of the window will be as specified manually using relevant Fusion Table functionality [optional]

iwalt= allows to specify alternative names for columns listed in iw parameter, eg. Postcode, Proportions of All Sales; default is column names as named in Fusion Tables [optional]

iwf= allows to specify format of numeric values returned from Fusion Tables, eg. n0, c2, %1, $0, where the first character indicates format option and the second (numeric value) specifies the number of decimal points; eg. if Fusion Tables column contains value 1.548  it will be formatted accordingly as: 1 (ie. “no format” and 0 decimal points),  1.55 (“comma format” for 1,000’s etc. and 2 decimal places), 154.8% (“percent format” with one decimal place) and $1 (“currency format” with 0 decimal places) [optional]. For columns with plain text use t0 code.

This is how fully compiled URL should look like:

http://www.aus-emaps.com/svs/ft/mapv3.php?
ftl=4369389
&xyz=147.971361,-32.801441,6
&n1=Median total family income weekly
&q1=name > 1999 AND name < 3000
&q1c=geometry
&r1=5
&st1=2br0
&qst1='B113' > 2030 AND 'B113' <= 3035,'B113' > 1484.5 AND 'B113' <= 2030,'B113' > 1194 AND 'B113' <= 1484.5,'B113' > 986 AND 'B113' <= 1194,'B113' >= 0 AND 'B113' <= 986
&ad=0
&iw=name,B113
&iwalt=Postcode,Median total family income weekly
&iwf=n0,$0

Finally, a short paragraph about the format of data in Fusion Tables that can be used with the Reference Map. The table must have polygons as a geographic component (eg. postcodes, suburbs, sales territories or other administrative boundaries) and it has to have a secondary descriptive column expressed as a number (eg. postcode number). The attributes to be mapped have to be numbers in order to divide them into meaningful ranges and to assign specific colours to those ranges. Those values could be any numbers representing characteristics of a given area (polygon) such as counts, percentages, densities or similar, or numbers used as “category descriptors” (eg. 1 means “mortgage stress belt”, 2 is “no mortgage burden areas”, etc...).

If it all gets too complicated, I will be happy to assist in setting up your own Fusion Tables account with a custom Reference Map.

Launching Census 2011 Online Maps

The first set of 2011 Census of Population and Housing data was officially released on 21 June 2012. This rich source of geographically referenced demographic statistics is a treasure trove for any spatial data and mapping enthusiast. Best of all, the data is totally free (but a small fee applies for convenience of having it all on a DVD, and right away).

I was hoping that there will be a mad rush of releases of various mapping applications with demographic statistics post June 21st but a quick search on the internet for “census maps” is not turning up much as yet. So far only general media put some effort into making census statistics more accessible with interactive presentations and simple maps - I listed the most interesting examples in my previous post. How disappointing. The official ABS version of online maps – TableBuilder, is still a few months away (free Basic version will be released on August 21, 2012; if you are prepared to pay $750 for an individual licence or $9,950 for a corporate licence, TableBuilder Pro version will be available to you on August 7, 2012). As someone twitted recently, census data release day felt like a total anticlimax…

So, to fill the void, I decided to rush the release of my own version of Census 2011 Online Maps. There are over 70 census data items available in the first release of the app (just under 150 if derived information is counted). By popular demand, postal area boundaries are the default geography (and the only option a the moment but more will be added later).

Unlike some traditional online GIS applications where every single map is preconfigured upfront, my Census 2011 maps are generated “on the fly” so, users have a lot more flexibility in customising map parameters and visual presentation (such as, the number of data classes and values of data ranges, colours of thematic drapes, locations of interest – again, more options will be added in the future).

This flexibility is very important in order to enable creation of meaningful thematic maps since there is such a large volume of data available for mapping (Basic Community Profile dataset has close to 8,000 data items!) and secondly, because each dataset, representing either counts, $ values or proportions, has very different characteristics. For example, a handful of datasets can be characterised as having normal distribution, there are some approximating uniform distribution but most have irregular distribution. And quite a few datasets contain extreme values/outliers which are “off the scale” (for example, how about a $132,500 median weekly household income for postcode 2139 or median weekly personal income of $8,312 for postcode 6710!). Single, automated approach to map such a diverse set of data will simply not work. Therefore, I included data histogram to help in selecting the most appropriate classification method (ie. either Standard Deviation, Equal Range, Quantiles or Natural Break – Jenks algorithm). More on how to make the most of available selectors in Census 2011 Online Maps User Guide which is published as a separate post.

Census 2011 Online Maps application is built with free Google Map, Fusion Tables and Visualisation APIs so, no infrastructure for me to look after! Most of the code has been recycled from various tools of mine shortening the development time. Census 2011 Online Maps actually comprises of two separate parts: map configuration module and an updated version of my free-standing reference map (available as a Web Application Service to anyone – read more on the concept of WAS in one of my earlier posts). So, for all practical reasons, Census 2011 Online Maps application is more like a “map creation" or "thematic mapping tool” that allows selection and configuration of raw data, stored in Fusion Tables, for display on a Google Map.

If you really like a particular map you have created, just detach it from the main page and bookmark it, or share with others! Or embed it in your web site… That portability of maps is a special feature of Census 2011 Online Maps application – users are no longer confined to visit my site to take advantage of the information. 

The volume and complexity of Census data is probably the reason why not many online maps have been released so far. In all honesty, not so long ago, desktop applications with functionality similar to my Census 2011 Online Maps used to cost tens of thousands of dollars. And online versions costed many more times over that due to the cost of infrastructure. Today anyone can create online mapping applications - for free…  I hope that my version will prove a popular choice. Maybe even more popular than a related Postcode Finder app! Help me spread the word so it can reach all those users you cannot afford pricey commercial GIS solutions. Your feedback and feature requests most welcome. The code is available to a selected few and/or can be customised to meet your specific business requirements.

Related posts:
Census 2011 Online Maps User Guide
Mapping Australian social diversity
Mapping social diversity in NSW
More free data with reference map
Web Application Service concept

Census 2011 Online Maps User Guide

This is a comprehensive introduction to a free online application available on @ aus-emaps.com website and called Census 2011 Online Maps. The application contains a selection of Australian 2011 Census of Population and Housing data published by the Australian Bureau of Statistics. It is designed specifically to facilitate creation of custom thematic choropleth maps with Census 2011 data (that is, maps are not predefined upfront and each is generated “on the fly”, according to user specifications).

Let’s start with some warnings and tips which will help in creating meaningful and informative maps.

Warning 1: Mapping Census data for the entire country poses some challenges since there are very significant differences between individual regions of Australia. Take for example median weekly personal incomes that vary from $76 to $8,312. It is therefore more practical to generate maps for smaller areas, such as individual States, rather than attempting to create a meaningful summary measure for the country as a whole.

Warning 2: Thematic choropleth maps are considered not suitable for mapping absolute values (eg. counts of persons). If absolute values are used for mapping areas that vary in size, misleading maps can be produced.

On the other hand using ratios, which show the relationship between two quantities, eliminates the influence of area, so that the map becomes meaningful by portraying accurately the distribution of features. Therefore, it is recommended to use only datasets labelled as “proportions” or “median values” for thematic choropleth mapping.

Nevertheless, there are legitimate cases where mapping absolute values can be justified. For example, when geography areas are of equal size (which is most likely the case with smaller geography units and/or urban locations) or when you deliberately want to “visualise absolute numbers” to help in defining territories based on counts of people with certain characteristics (since it easy to identify on the map which polygons are next to each other).

GENERAL INFORMATION

Base Geography:

The first release of Census 2011 Online Maps offers only one geography option - Postal Areas (which are good approximation of Australia Post Postcodes).

Data:

Census 2011 Online Maps application provides convenient access to approximately 150 datasets selected from the Basic Community Profile (only a fraction of 7942 individual datasets published by the ABS but of most interest to all). This initial selection will be extended in due course. 

Where appropriate, “proportions of total” for individual datasets were calculated and added to selection. There is also an overall “population density” dataset available for mapping. Full list of available datasets can be downloaded in a csv format by clicking on the following link: Census 2011 Online Maps Data List.

CONFIGURING MAPS

Start with clicking on a blue configure map text in the top right corner of the page, then follow the steps below.

Data Selection:

To simplify selection of individual datasets they have been grouped into themes and topics.

Select Theme:
Select from a dropdown list: Persons, Family or Dwellings

Select Topic:
Select relevant topic from a dropdown list (topics are specific for individual themes).

Select Data Item:
A list of available data items for a given theme and topic will be printed on the page. Select dataset of interest by clicking on the relevant box. Your selection will be indicated by change of box colour to a light blue. 

Tip: to reload the current selection just “unselect” and “select” it again by clicking twice on the light blue box.

Region Selection:

Select from a dropdown list. Options include Australia, States and larger capital cities. Default is “Australia”.

Changing region will trigger data refresh (ie. histogram, summary statistics, and data ranges will be recalculated and reloaded).

Data Exclusions:

Sometimes data is not available for all locations, either due to confidentiality or lack of valid answers to Census questions. By default empty cells are not counted in calculations of data statistics (ie. counts, mean and median, etc).

Some data series have lots of zero values. Users have an option to remove cells with “0” from calculations of data statistics. Changing option will trigger data refresh (ie. histogram, summary statistics, and data ranges will be recalculated and reloaded).

Histogram Customisation:

Default method for calculating optimal number of histogram bins is sq root of count of records, divided by 2 and rounded to a full number. Users can adjust the number of bins for a histogram by typing in a preferred number. It will trigger data refresh (ie. histogram will be reloaded).

Selecting Best Data Classification Method:

Start with inspecting histogram to determine how data is distributed. General rule is that for normally distributed data (symmetrical, bell shaped histogram) Standard Deviation is the optimal method for defining values for range breaks.

[Example of histogram of normally distributed data]

In this application St Deviation with 3 classes defines range breaks at -1SD and +1SD. With 5 classes, range breaks are placed at -1.5SD, -0.5SD, +0.5SD and +1.5SD.


Equal Range (also known as Equal Distribution or Interval) is the best option for uniformly distributed data (histogram with bars of similar height) as for example, Median Rents for QLD.

[Example of histogram of uniformly distributed data]

In Equal Range breaks are determined by dividing the difference between max and min values by the selected number of ranges. This method can result in ranges with few or no records, if used with other than uniformly distributed data.


The majority of Census data will have irregular distribution for which Natural Break or Quantiles classification methods are the most appropriate.

[Example of histogram of irregularly distributed data]

Jenks algorithm is used to determine range breaks for Natural Break classification. The algorithm groups data into classes that are themselves as separate as possible, but where the data values within each class are fairly close together. That is, it maximises the differences between the classes and minimises the differences within the classes. This classification can be used to discover spatial patterns within the data, but it can lead to some classes being populated by low numbers of observations. ABS recommends Natural Break as a default classification for Census data but prefers an alternative methodology, based on the Dalenious Hodges algorithm.


Quantiles also work well with irregularly distributed data and this is probably the most well known data classification method. Range breaks for quantiles are calculated by sorting data in an ascending order and dividing into equal parts, based on the number of selected classes. Therefore each range will contain a similar number of units (ie. equal count or proportion of records, polygons). If the entire dataset is divided into halves, the range break is a median. Dividing data into four classes produces quartiles separated by three range breaks. Quintiles have five classes with four range breaks.


The fifth classification method available in Census 2011 Online Maps is “Custom Ranges”. Simply, type in your preferred values in relevant boxes to overwrite automatically generated numbers.


Min and Max values are used to define respectively the lower and the upper limits for the “outer ranges”, inclusive of these numbers. Selectors can be adjusted to exclude those values from ranges, if required.

Changing Classification Method and Colour Scheme:

Select Classification Method:
Select from a dropdown list: St Deviation, Equal Range, Quantiles or Natural Break.

Note to Internet Explorer 7 users: this browser version has very inefficient Javascript processing engine so, when executing complex code that handles large volume of data the browser may be struggling. This could be the case when computing Natural Break ranges. A pop-up window will appear with the following warning: “Stop running this script? A script on this page is causing Internet Explorer to run slowly. If it continues to run, your computer may become unresponsive”.  Keep clicking “No” button until the process is finished (usually more than a few times!) and the browser will resume  normal operation.

 

Select Number of Ranges:
Select from a dropdown list a value corresponding to preferred number or ranges. Options vary depending on selected data classification method. Selecting ranges will trigger data refresh (ie. data ranges will be recalculated and reloaded).

Please note, Fusion Tables API has a limit of 5 colour ranges for thematic choropleth drapes. This limitation should not pose an issue for the majority of Census 2011 Online Maps users since ABS recommends 5 data ranges for thematic mapping as an optimal number.

Selecting Colours:
There are 3 single colour schemes (blue, purple and green) and one 2-colour scheme (red-blue) available for selection in the Census 2011 Online Maps application. For convenience, reverse colour option have also been added. Functionality allowing customising colours will be enabled at a later date.

Generating Thematic Map:

To complete map configuration step simply click on a blue button “Update Map”. A new thematic drape will be generated and loaded onto the map.

Please note, you may need to zoom in/ zoom out a few times to “force” update of thematic drapes (it is a known issue with Fusion Tables dynamic layer generation functionality).

Clicks on coloured polygons will open windows with detailed information about individual postcodes.

QUESTIONS

If you have any questions regarding the functionality of Census 2011 Online Maps please submit them as comments to this post. Requests for customisation of the application to specific business requirements can be sent to info@aus-emaps.com.

Historic Atlas of the Suburbs of Sydney

If you like historic maps, you will have great fun with this simple mapping application created by dictionaryofsydney.org. It brings together digitised version of The Atlas of the Suburbs of Sydney, originally published in 1885, and contemporary online street directory (ie. Google Maps). All up, there are 51 late-nineteenth-century maps of Sydney municipalities that provide a portrait of the city during a period of rapid growth and suburbanisation. 

The Atlas of the Suburbs of Sydney was created by Higinbotham, Robinson and Harrison, map publishers and lithographers established in 1882 with premises in Macquarie Place. They had obtained permission to produce maps from government survey information, so were able to advertise them as an authoritative source '…compiled from official plans in the Surveyor-General and Registrar-General's offices'.

The Atlas of the Suburbs of Sydney could be considered the first officially published street directory of Sydney. Read more about The Atlas of the Suburbs of Sydney history on dictionaryofsydney.org website. To compare Sydney “now and then” just click on images of old maps and then select "Full record" link to load Google Map interactive application. 

SBS - Census Explorer on Google Maps

Keir Clarke from Google Maps Mania has just written a short post on the SBS - Census Explorer. So far it remains the only substantial mapping applications for exploration of the key demographic indicators from Australia's 2011 Census.



The tool allows users to interrogate the census data and visualise on the map the number of persons using one of several dozens of different languages spoken throughout Australia. Using a drop-down menu it is possible to select an individual language and a coloured thematic drape indicating concentration of people speaking that language is then displayed on the map. By clicking on individual suburbs users can get the exact number of the language's speakers at that location.The application has also a handy functionality which allows to compare two language groups for a given location.