REIV maps auction statistics

The Real Estate Institute of Victoria (REIV) has turned to maps to present its latest set of property related information. In particular, the Institute now publishes a weekly update of auction clearance rates and information on prevailing sales methods by suburb. The information is presented on a Google map as a thematic overlay.

The Clearance rates by suburb map allows users to view information about total auctions, number of sales and a percent of successful sales for all suburbs with more than 20 auctions in 2012. Clearance rates for the suburbs are then compared to Melbourne-wide clearance rate. Individual suburbs are coloured based on whether their clearance rate is higher or lower than the average rate. 

REIV published also a set of maps that compare historical statistics on auction and private sales for 2010 and 2011. They provide buyers, sellers and anyone with an interest in the local market with data about the most popular sales method in each suburb. These maps highlight the fact that sales by auction are far less prevalent in the outer suburbs than the inner city.  A comparison of the use of auctions over time highlights the link with the state of the property market. That is, auctions are more popular when the market is stronger and private sales increase in popularity when there is a lower level of competition between buyers.

The maps are a part of ongoing effort by the Institute to provide timely, in depth analysis about the state of the property market in Victoria.

Related Posts:
Map of Melbourne house prices 
Sydney house prices
Maps and property investment
WA housing affordability index
Aircraft noise maps
Mapping sun position anywhere

Australian Floods Mapped

Earlier this week Geosciece Australia released a proof of concept web service displaying trial study areas of historical flood mapping derived from satellite imagery. It contains one low resolution national layer and several study areas in high resolution. The national layer displays a 500x500m grid summarising how often water was observed in specific location by the MODIS sensor from 2000 to 2012. A more detailed information, derived from Landsat-5 and mapped into a 25x25m grid is available for:

• Condamine River system between Condamine and Chinchilla, Queensland
• Northwest Victorian rivers between Shepparton and Kerang
• North Queensland rivers, near Normanton

Over the next two years this service will be developed into a nationwide portal to ultimately provide data on flooding which has been observed by satellite across Australia since 1987.

The information has some limitations as not all foods can be detected by satellites since observation of the Earth by satellite depends on clear skies and satellites have specific, scheduled observation times. Nevertheless, it’s a first attempt to map floods consistently across the entire continent and the service will provide valuable insights into flood patterns and food extents.

Google unlocks Landsat archives

It took Google 2 ½ years to convert 40 years of Landsat imagery from tapes stored in USGS archives to the cloud. Two petabytes of data in total. Now all this data is made available to researchers and non-profit organisation for comparison and number-crunching through the Google Earth Engine tool under the Google Earth Outreach program.

There is a great expectation that free access to data and significant computational capability will lead to a surge in public benefit maps coming out of Australia. Organisations that have already partnered with Google to access a suite of GIS products and learn how to use them for free include the Australian Wildlife Conservancy (AWC) and the Great Barrier Reef Marine Park Authority (GBRMPA).

Despite numerous attempts over the years to open up the vaults of valuable satellite data to academics, research institutions and individuals it took a determined private company to make it all possible. A good example of public-private collaboration that is probably a sign of things to come. This is a side of Google that we all would like to hear about more often.


More free data: Satellite Imagery Catalogue 

Competitive advantage with free GIS tools

For years I have been advocating the advantage of matching the tools to the job, that is selecting those that are “fit for the purpose”. Nothing more and nothing less, because it is either a waste of money or you end up with inadequate solution which will not deliver expected results. Logical, but not very common approach in corporate life…

Why would you spend thousands on something that you can use for free to get the job done? Many apply this principle in their private lives but do not extend it to their business activities. There is always more than one way to skin the cat, and very often applying your “old toolbox” to new problems will simply not work (eg. big data in GIS!). In a competitive world, or where resources to accomplish the task are scarce, only the smart ones, who are able to think outside the square, will get the job done and will advance… 

In particular, very often organisations buy packaged solutions, which come bundled with extensive range of functional tools, but which are rarely used, if at all. Microsoft’s Word or Excel are classic examples (not picking on Microsoft specifically but they are a good point in case that everybody can relate to). It could be argued that 80% of users utilise less than 20% of built-in functionality. But organisations are paying the full price for all of them. As an old saying goes, “No one was ever sacked for buying Microsoft Windows”… because it is presumably the least risk option. The same can be extended to many GIS solutions…

It could also be argued that the larger the organisation, the less likely it is to look for cheaper alternatives. Why? It is “easy” for the IT department to deal with limited number of software, easy for buyers because they are buying “the brand”, easy for users because their skills are transferable to the next job… But spare a though for what it does to the bottom line of an organisation. Don’t get me wrong, there are many, many cases where buying the whole package makes perfect sense, even financial, but it can also be argued that in many more cases this is simply overkill. The issue is particularly relevant for smaller organisations.

What if you could get 80% of functionality of “branded solutions” (functionality you really need) for 20% of the price? The answer: you could actually end up with 4 times the functional capability of your competitors who are paying the full price for one solution!

Modern GIS, BI, reporting dashboards and similar solutions do not have to be expensive. If you are a small business, or organisation that is conscious about the costs, you can deploy solutions at a fraction of the cost of branded software used by the big business.

Take for example this single page, satellite imagery catalogue created with free Google Map and Google Fusion Tables. It includes attribute and time range filters as well as location search. It is capable of handling of up to 100,000 items:

Software cost: $0
Infrastructure cost: $0
Maintenance cost: $0* (until Google deprecates the service)
Development cost: “a few” hours

Simple, yet delivering in full the core search functionality of bigger systems that would cost tens of thousands to deploy and maintain. It allows users to quickly determine what imagery is available for a given location and within a given time frame and under what licence. Individual scenes can be inspected to determine precise coverage extents (ie. scene footprint) as well as cloud density (ie. preview quicklook) before following up with an email enquiry about the access to the information. 

The main game is not about “cutting the corners” or blunt “cost cutting”. It is all about being smart and applying “fitness for purpose” principle… You can gain a lot by talking to people who provide solutions and not only those who sell branded software.

Maps as dataviz tool

Maps are one of the oldest information visualisation tools and have been around for quite a while. There were times when map appearance was confined by application of the strictest rules in production, and of course the technology of the day, but once the capacity to create maps extended to general public maps as a data presentation medium blossomed in thousands of varieties. This is why working with maps is so exciting today…

Map is an abstract, graphical representation of the reality. These days that reality can be quite complex - we are after all capable of processing more and more data into information. So, as the result, there is ever increasing need to summarise and present complex spatial information in a meaningful and easy to comprehend way. This pushes the boundaries of modern cartography into seemingly unrelated disciplines of art, graphics, animations, etc.

A lot can be written on current trends in modern cartography but here is just one example of an innovative approach to presenting complex information on a map: 33 years of hurricane paths.

Opinions were divided on whether it was still “a map”, and some expressed a view that this presented “incorrect perspective on the world” but those who have open minds on modern cartography, and who have any experience in dealing with presentation of complex information, will undoubtedly be full of praises for the author of this “creation”!

Link to animated version (7.7MB): Hurricanes Flip-Book Style

ACT implements free data initiative

Following federal government and larger States, ACT government is the latest convert to the cause of liberating vast holdings of spatial and non-spatial data in hope of invigorating local business to develop innovative solution and use the data for better decision making. And in the process, to stimulate economic activity and increase the number and quality of services offered to the local community. Built on Socrata platform dataACT is an open data service initiative that will enable individuals, businesses and people in the public sector to access, explore and build upon existing government data sets.

dataACT has three core components which make it probably the most functional of all the initiatives in Australia. In particular, data is available in a machine readable and consistent range of data formats. This enables users to download static datasets in ready to use formats (eg. JSON but unfortunately, stopping short of being a dynamic web service). Secondly, the platform includes some basic data visualisation tools to make graphs or maps that can be embedded on external websites. This enables the creation of customised views of datasets for particular community needs or specific initiatives. However, the most exciting news is that data is uploaded from across the ACT Public Service through data integration systems that will update individual datasets appropriately. So, there is a commitment from day one to keep the data up to date.

Initial set of released data is only a teaser but it is a good start. Let’s hope that the ACT government will not back on its commitment to the project and take a leading role in liberating the data that would really make a difference – for the local spatial community it means access to street data, cadastre information, address files and/or geocoding service, property transactions data (current and historical), 2m contours / DEM, topographic data, tourism database and high resolution imagery over the Territory. Don’t get me wrong. There was a time when a release of information on “toilet locations” generated a furore (and many goods apps were created as the result) but if the expectation is to make a significant and lasting economic impact, more useable data needs to be put in the public domain.

Other data liberation and spatial data infrastructure initiatives seem to be fading fast after the initial flurry of activities, including myriad of hack-a-tons with rich prizes for participants. It is, after all, a hard work which requires more than just “sticking a few csv files on a web page” and hoping for the best. Let’s be brutally honest about it, it would be hard to name a single “public data mart” initiative in Australia that can really claim a success so, there is a big opportunity for dataACT to demonstrate how it should be done.

Related Posts:
What's the benefit of gov data warehouses?
Free data a GFC casualty
Governments intensify free data efforts
UK unlocks vaults of spatial data

Data viz tool from DataAppeal

Continuing the theme of data presentation, I would like to mention another application that caught my attention and which may be of interest to readers of this blog - especially as it does not require any training in GIS. It is called DataAppeal and is a web-based visualization application that allows users to transform location- specific data, stored in tabular format, into 3D animated maps.

The company behind the application set a very ambitious goal - "to bring together concepts of art, design and spatial analytics to increase interest in sharing information and knowledge between users, for quicker and better decision making".

Users can upload numeric data with location reference and render it into a 3D map using Google Earth application. The users can choose from a number of design templates or customise display options to achieve the desired "look" that will enhance the message they want to convey. Basic features are available under free subscription option but heavy users can upgrade to a more advanced version of the product, if required. The URL link of the data-map can be posted and shared through various social media platforms like Facebook, Twitter or just email.

All in all, DataAppeal is simple to use and offers no-cost introduction to visualisation of corporate data on 3D maps.

Visualising Victoria's Groundwater

A few weeks ago Victorians added a new information source to the ever growing list of online interactive GIS applications: Visualising Victoria's Groundwater. The project is an initiative aimed at federating groundwater data from disparate sources to assist in groundwater research and water management.

The application contains extensive information on approximately 300,000 bores in across Victoria. The map also includes a number of useful layers to put the information in context. Those layers include elevation, depth to watertable and surface elevation.

It is built with OpenLayers framework and utilises Web Map Service (WMS) to facilitate publication of custom data. Click on the map brings up the information about a particular dataset, for that specific location (but is not enabled on all available layers; by the way, this is a great feature of WMS which is often overlooked by developers in favour of more complex solutions).

An interesting aspect of the application is that drawing tools are utilised as a feedback generation mechanism. That is, users can mark specific location and submit comments (“undo” option to remove drawn objects from the map would be a desirable functionality extension). There is also a support for exporting of numeric data (limited to a few hundred items at a time). As all modern GIS online applications it also supports layer reordering but I could not get this function to work properly. Overall, Visualising Victoria's Groundwater is quite nice and simple to use information presentation application.

The project has been funded by the State Government of Victoria - Broadband Enabled Innovation Program


First spotted on : Google Maps Mania

Landslides in Australia

Geoscience Australia has just released a simple mapping application that allows members of the public to search the internal landslide database for events that have occurred across Australia since 1842. A landslide consists of one or more related “landslips”. Since landslips are a naturally occurring phenomenon, they are difficult to predict and can often occur suddenly without any witnesses. This makes it difficult to determine some of the properties of a landslip, such as the triggering and contributing factors, and when exactly it occurred. Subsequently, landslip event details are sometimes incomplete as the information could not be determined. This is particularly true for historical records.

The Landslide Search application is intended to raise public awareness about this particular type of natural hazard in Australia. Users can customise search according to location, time or impact of the event, and view the results on a map and in an expandable table.

Visualising census data on maps

2011 Census of Population and Housing data was officially released by the Australian Bureau of Statistics in late June 2011. Data is freely accessible on ABS site however, it is rather difficult to work with numeric tables alone to determine distribution of people with specific characteristics. Knowing geographic location of groups of interest can be invaluable in decision making for all variety of users: governments, businesses as well as individuals.

TableBuilder Basic, a free mapping application from ABS, will be released in August 2012 but you can start mapping and analysing of Census data right away: aus-emaps.com has just launched a free tool that enables creation of custom thematic maps with 70+ census data items from the Basic Community Profile. Below are just a few examples of what is available and how this information can be put to use.

Case 1: Locating areas with the greatest purchasing power

Personal incomes are indicative of propensity to spend so, areas with high median weekly income would be a primary target to concentrate promotional activities on if you want to attract customers for luxury items or high value investment products.

Case 2: Locating ethnic niches

Ethnicity related data can be used by various government departments delivering social services to profile their local customers in order, for example, to determine the need for personnel with specific language skills and/or adequate training to deal properly with cultural differences.

Case 3: Estimating market potential

Census data will help in determining if there is enough potential customers in a given location to support a business. For example, if you provide goods or services for infants and toddlers you would need to know how many are there in your catchment area.  

Case 4: Benchmarking investment performance

Property investors can take advantage of Census data related to housing. For example, median weekly rent statistics can help in determining whether their rental income is keeping up with the changes in the local market or not.

Explore the full set of data on Census 2011 Online Maps app.


Related Posts:
Census 2011 Online Maps User Guide
Mapping Australian social diversity
How maps can improve sales
Maps and property investment
Reference Maps for Fusion Tables

Reference Map for Fusion Tables

Reference Map for Fusion Tables is a free Web Application Service (WAS) provided by aus-emaps.com. The application enables displaying information stored in Google’s Fusion Tables as a thematic choropleth map. There are a few tricks you will need to know to put it to a good use. This post describes in detail how to configure Version 3 of the application to display custom choropleth maps (the concept of WAS is explained in an earlier post on this blog).

Firstly, some background information. Although Google included simple map configuration functionality within Fusion Tables, it is a manual process allowing formatting only one dataset at a time. If you have multiple columns of data available for mapping in a single table, the task of creating individual thematic maps can be quite a tedious process. Therefore, creating such maps programmatically, via Fusion Tables API, is the only viable option.

My recently released Census 2011 Online Maps application is an example of how to automate choropleth map creation process. There are two separate modules in that application. The first module enables selecting and analysing data stored in Fusion Tables, then formatting it for display as a thematic layer on a Google Map (ie. selecting number of classes, defining range break values and selecting colour scheme for thematic layer). The second module is a free-standing application, built with Google Map API, that displays the information from Fusion Tables as a thematic layer, formatted according to user requirements.

Now specifically about the mapping app. When designing Census 2011 Online Maps application I opted to incorporate Reference Map for Fusion Tables rather than coding mapping functionality from the scratch, but I had to update the application in order to enable proper formatting of numeric data retrieved from Fusion Tables. It also was an opportunity to expand the functionality of the reference map to enable generating thematic choropleth maps with 2, 3, 4 as well as 5 data ranges (five is a limit imposed by Fusion Tables API). All in all, the latest version of Reference Map for Fusion Tables now accepts a number of new parameters which enable quite extensive customisation of thematic layer for presentation on the map.

Below is a full list of accepted parameters and relevant options. Parameters flagged as “minimum” are compulsory in order to display Fusion Table in its “native style” (ie. as styled manually using relevant Fusion Tables functionality). “Optional” parameters can be omitted without degrading presentation of the information on the map. Please note, each parameter must be separated in compiled URL with “&” symbol.

Reference Map for Fusion Tables v3 – Acceptable Parameters:

ftl=  table id number eg: 1234567 [minimum]

q1c=geometry [minimum] - specifies which column contains spatial data

r1= defines the number of ranges for choropleth map: eg. 2, 3, 4 or 5

st1= thematic layer colour scheme selector, eg. b0=blue, pp0=purple, g0=green 2br0= blue-red; it is also possible to specify colours in reverse order as b0r, pp0r, g0r, 2br0r accordingly

qst1= SQL like data ranges specification; must correspond to the number of ranges specified for r1 parameter and be comma separated; column names must be between apostrophes; can  be either in ascending or descending order, eg.: 'Relevant Column Name' > 2030 AND 'Relevant Column Name ' <= 3035,'Relevant Column Name ' >= 0 AND 'Relevant Column Name' <= 2030  

q1=  SQL like WHERE query specification, to limit results to only specific set of data, eg. location: postcode > 1999 AND postcode < 3000  [optional]

n1= map title [optional]

xyz= allows to set map location and zoom level as longitude, latitude and zoom values, eg: 147.971361,-32.801441,6 [optional]

ad= can be used to hide ads on the map when set to 0, default is 1 [optional]


The following 3 parameters go together and are optional; each must contain the same number of variables separated by comma:

iw= allows to specify columns to be used to generate content of information window (“balloon”) that appears on the map when coloured polygons are clicked, eg. PC, D32 - if omitted the content of the window will be as specified manually using relevant Fusion Table functionality [optional]

iwalt= allows to specify alternative names for columns listed in iw parameter, eg. Postcode, Proportions of All Sales; default is column names as named in Fusion Tables [optional]

iwf= allows to specify format of numeric values returned from Fusion Tables, eg. n0, c2, %1, $0, where the first character indicates format option and the second (numeric value) specifies the number of decimal points; eg. if Fusion Tables column contains value 1.548  it will be formatted accordingly as: 1 (ie. “no format” and 0 decimal points),  1.55 (“comma format” for 1,000’s etc. and 2 decimal places), 154.8% (“percent format” with one decimal place) and $1 (“currency format” with 0 decimal places) [optional]. For columns with plain text use t0 code.

This is how fully compiled URL should look like:

http://www.aus-emaps.com/svs/ft/mapv3.php?
ftl=4369389
&xyz=147.971361,-32.801441,6
&n1=Median total family income weekly
&q1=name > 1999 AND name < 3000
&q1c=geometry
&r1=5
&st1=2br0
&qst1='B113' > 2030 AND 'B113' <= 3035,'B113' > 1484.5 AND 'B113' <= 2030,'B113' > 1194 AND 'B113' <= 1484.5,'B113' > 986 AND 'B113' <= 1194,'B113' >= 0 AND 'B113' <= 986
&ad=0
&iw=name,B113
&iwalt=Postcode,Median total family income weekly
&iwf=n0,$0

Finally, a short paragraph about the format of data in Fusion Tables that can be used with the Reference Map. The table must have polygons as a geographic component (eg. postcodes, suburbs, sales territories or other administrative boundaries) and it has to have a secondary descriptive column expressed as a number (eg. postcode number). The attributes to be mapped have to be numbers in order to divide them into meaningful ranges and to assign specific colours to those ranges. Those values could be any numbers representing characteristics of a given area (polygon) such as counts, percentages, densities or similar, or numbers used as “category descriptors” (eg. 1 means “mortgage stress belt”, 2 is “no mortgage burden areas”, etc...).

If it all gets too complicated, I will be happy to assist in setting up your own Fusion Tables account with a custom Reference Map.

Launching Census 2011 Online Maps

The first set of 2011 Census of Population and Housing data was officially released on 21 June 2012. This rich source of geographically referenced demographic statistics is a treasure trove for any spatial data and mapping enthusiast. Best of all, the data is totally free (but a small fee applies for convenience of having it all on a DVD, and right away).

I was hoping that there will be a mad rush of releases of various mapping applications with demographic statistics post June 21st but a quick search on the internet for “census maps” is not turning up much as yet. So far only general media put some effort into making census statistics more accessible with interactive presentations and simple maps - I listed the most interesting examples in my previous post. How disappointing. The official ABS version of online maps – TableBuilder, is still a few months away (free Basic version will be released on August 21, 2012; if you are prepared to pay $750 for an individual licence or $9,950 for a corporate licence, TableBuilder Pro version will be available to you on August 7, 2012). As someone twitted recently, census data release day felt like a total anticlimax…

So, to fill the void, I decided to rush the release of my own version of Census 2011 Online Maps. There are over 70 census data items available in the first release of the app (just under 150 if derived information is counted). By popular demand, postal area boundaries are the default geography (and the only option a the moment but more will be added later).

Unlike some traditional online GIS applications where every single map is preconfigured upfront, my Census 2011 maps are generated “on the fly” so, users have a lot more flexibility in customising map parameters and visual presentation (such as, the number of data classes and values of data ranges, colours of thematic drapes, locations of interest – again, more options will be added in the future).

This flexibility is very important in order to enable creation of meaningful thematic maps since there is such a large volume of data available for mapping (Basic Community Profile dataset has close to 8,000 data items!) and secondly, because each dataset, representing either counts, $ values or proportions, has very different characteristics. For example, a handful of datasets can be characterised as having normal distribution, there are some approximating uniform distribution but most have irregular distribution. And quite a few datasets contain extreme values/outliers which are “off the scale” (for example, how about a $132,500 median weekly household income for postcode 2139 or median weekly personal income of $8,312 for postcode 6710!). Single, automated approach to map such a diverse set of data will simply not work. Therefore, I included data histogram to help in selecting the most appropriate classification method (ie. either Standard Deviation, Equal Range, Quantiles or Natural Break – Jenks algorithm). More on how to make the most of available selectors in Census 2011 Online Maps User Guide which is published as a separate post.

Census 2011 Online Maps application is built with free Google Map, Fusion Tables and Visualisation APIs so, no infrastructure for me to look after! Most of the code has been recycled from various tools of mine shortening the development time. Census 2011 Online Maps actually comprises of two separate parts: map configuration module and an updated version of my free-standing reference map (available as a Web Application Service to anyone – read more on the concept of WAS in one of my earlier posts). So, for all practical reasons, Census 2011 Online Maps application is more like a “map creation" or "thematic mapping tool” that allows selection and configuration of raw data, stored in Fusion Tables, for display on a Google Map.

If you really like a particular map you have created, just detach it from the main page and bookmark it, or share with others! Or embed it in your web site… That portability of maps is a special feature of Census 2011 Online Maps application – users are no longer confined to visit my site to take advantage of the information. 

The volume and complexity of Census data is probably the reason why not many online maps have been released so far. In all honesty, not so long ago, desktop applications with functionality similar to my Census 2011 Online Maps used to cost tens of thousands of dollars. And online versions costed many more times over that due to the cost of infrastructure. Today anyone can create online mapping applications - for free…  I hope that my version will prove a popular choice. Maybe even more popular than a related Postcode Finder app! Help me spread the word so it can reach all those users you cannot afford pricey commercial GIS solutions. Your feedback and feature requests most welcome. The code is available to a selected few and/or can be customised to meet your specific business requirements.

Related posts:
Census 2011 Online Maps User Guide
Mapping Australian social diversity
Mapping social diversity in NSW
More free data with reference map
Web Application Service concept

Census 2011 Online Maps User Guide

This is a comprehensive introduction to a free online application available on @ aus-emaps.com website and called Census 2011 Online Maps. The application contains a selection of Australian 2011 Census of Population and Housing data published by the Australian Bureau of Statistics. It is designed specifically to facilitate creation of custom thematic choropleth maps with Census 2011 data (that is, maps are not predefined upfront and each is generated “on the fly”, according to user specifications).

Let’s start with some warnings and tips which will help in creating meaningful and informative maps.

Warning 1: Mapping Census data for the entire country poses some challenges since there are very significant differences between individual regions of Australia. Take for example median weekly personal incomes that vary from $76 to $8,312. It is therefore more practical to generate maps for smaller areas, such as individual States, rather than attempting to create a meaningful summary measure for the country as a whole.

Warning 2: Thematic choropleth maps are considered not suitable for mapping absolute values (eg. counts of persons). If absolute values are used for mapping areas that vary in size, misleading maps can be produced.

On the other hand using ratios, which show the relationship between two quantities, eliminates the influence of area, so that the map becomes meaningful by portraying accurately the distribution of features. Therefore, it is recommended to use only datasets labelled as “proportions” or “median values” for thematic choropleth mapping.

Nevertheless, there are legitimate cases where mapping absolute values can be justified. For example, when geography areas are of equal size (which is most likely the case with smaller geography units and/or urban locations) or when you deliberately want to “visualise absolute numbers” to help in defining territories based on counts of people with certain characteristics (since it easy to identify on the map which polygons are next to each other).

GENERAL INFORMATION

Base Geography:

The first release of Census 2011 Online Maps offers only one geography option - Postal Areas (which are good approximation of Australia Post Postcodes).

Data:

Census 2011 Online Maps application provides convenient access to approximately 150 datasets selected from the Basic Community Profile (only a fraction of 7942 individual datasets published by the ABS but of most interest to all). This initial selection will be extended in due course. 

Where appropriate, “proportions of total” for individual datasets were calculated and added to selection. There is also an overall “population density” dataset available for mapping. Full list of available datasets can be downloaded in a csv format by clicking on the following link: Census 2011 Online Maps Data List.

CONFIGURING MAPS

Start with clicking on a blue configure map text in the top right corner of the page, then follow the steps below.

Data Selection:

To simplify selection of individual datasets they have been grouped into themes and topics.

Select Theme:
Select from a dropdown list: Persons, Family or Dwellings

Select Topic:
Select relevant topic from a dropdown list (topics are specific for individual themes).

Select Data Item:
A list of available data items for a given theme and topic will be printed on the page. Select dataset of interest by clicking on the relevant box. Your selection will be indicated by change of box colour to a light blue. 

Tip: to reload the current selection just “unselect” and “select” it again by clicking twice on the light blue box.

Region Selection:

Select from a dropdown list. Options include Australia, States and larger capital cities. Default is “Australia”.

Changing region will trigger data refresh (ie. histogram, summary statistics, and data ranges will be recalculated and reloaded).

Data Exclusions:

Sometimes data is not available for all locations, either due to confidentiality or lack of valid answers to Census questions. By default empty cells are not counted in calculations of data statistics (ie. counts, mean and median, etc).

Some data series have lots of zero values. Users have an option to remove cells with “0” from calculations of data statistics. Changing option will trigger data refresh (ie. histogram, summary statistics, and data ranges will be recalculated and reloaded).

Histogram Customisation:

Default method for calculating optimal number of histogram bins is sq root of count of records, divided by 2 and rounded to a full number. Users can adjust the number of bins for a histogram by typing in a preferred number. It will trigger data refresh (ie. histogram will be reloaded).

Selecting Best Data Classification Method:

Start with inspecting histogram to determine how data is distributed. General rule is that for normally distributed data (symmetrical, bell shaped histogram) Standard Deviation is the optimal method for defining values for range breaks.

[Example of histogram of normally distributed data]

In this application St Deviation with 3 classes defines range breaks at -1SD and +1SD. With 5 classes, range breaks are placed at -1.5SD, -0.5SD, +0.5SD and +1.5SD.


Equal Range (also known as Equal Distribution or Interval) is the best option for uniformly distributed data (histogram with bars of similar height) as for example, Median Rents for QLD.

[Example of histogram of uniformly distributed data]

In Equal Range breaks are determined by dividing the difference between max and min values by the selected number of ranges. This method can result in ranges with few or no records, if used with other than uniformly distributed data.


The majority of Census data will have irregular distribution for which Natural Break or Quantiles classification methods are the most appropriate.

[Example of histogram of irregularly distributed data]

Jenks algorithm is used to determine range breaks for Natural Break classification. The algorithm groups data into classes that are themselves as separate as possible, but where the data values within each class are fairly close together. That is, it maximises the differences between the classes and minimises the differences within the classes. This classification can be used to discover spatial patterns within the data, but it can lead to some classes being populated by low numbers of observations. ABS recommends Natural Break as a default classification for Census data but prefers an alternative methodology, based on the Dalenious Hodges algorithm.


Quantiles also work well with irregularly distributed data and this is probably the most well known data classification method. Range breaks for quantiles are calculated by sorting data in an ascending order and dividing into equal parts, based on the number of selected classes. Therefore each range will contain a similar number of units (ie. equal count or proportion of records, polygons). If the entire dataset is divided into halves, the range break is a median. Dividing data into four classes produces quartiles separated by three range breaks. Quintiles have five classes with four range breaks.


The fifth classification method available in Census 2011 Online Maps is “Custom Ranges”. Simply, type in your preferred values in relevant boxes to overwrite automatically generated numbers.


Min and Max values are used to define respectively the lower and the upper limits for the “outer ranges”, inclusive of these numbers. Selectors can be adjusted to exclude those values from ranges, if required.

Changing Classification Method and Colour Scheme:

Select Classification Method:
Select from a dropdown list: St Deviation, Equal Range, Quantiles or Natural Break.

Note to Internet Explorer 7 users: this browser version has very inefficient Javascript processing engine so, when executing complex code that handles large volume of data the browser may be struggling. This could be the case when computing Natural Break ranges. A pop-up window will appear with the following warning: “Stop running this script? A script on this page is causing Internet Explorer to run slowly. If it continues to run, your computer may become unresponsive”.  Keep clicking “No” button until the process is finished (usually more than a few times!) and the browser will resume  normal operation.

 

Select Number of Ranges:
Select from a dropdown list a value corresponding to preferred number or ranges. Options vary depending on selected data classification method. Selecting ranges will trigger data refresh (ie. data ranges will be recalculated and reloaded).

Please note, Fusion Tables API has a limit of 5 colour ranges for thematic choropleth drapes. This limitation should not pose an issue for the majority of Census 2011 Online Maps users since ABS recommends 5 data ranges for thematic mapping as an optimal number.

Selecting Colours:
There are 3 single colour schemes (blue, purple and green) and one 2-colour scheme (red-blue) available for selection in the Census 2011 Online Maps application. For convenience, reverse colour option have also been added. Functionality allowing customising colours will be enabled at a later date.

Generating Thematic Map:

To complete map configuration step simply click on a blue button “Update Map”. A new thematic drape will be generated and loaded onto the map.

Please note, you may need to zoom in/ zoom out a few times to “force” update of thematic drapes (it is a known issue with Fusion Tables dynamic layer generation functionality).

Clicks on coloured polygons will open windows with detailed information about individual postcodes.

QUESTIONS

If you have any questions regarding the functionality of Census 2011 Online Maps please submit them as comments to this post. Requests for customisation of the application to specific business requirements can be sent to info@aus-emaps.com.

Historic Atlas of the Suburbs of Sydney

If you like historic maps, you will have great fun with this simple mapping application created by dictionaryofsydney.org. It brings together digitised version of The Atlas of the Suburbs of Sydney, originally published in 1885, and contemporary online street directory (ie. Google Maps). All up, there are 51 late-nineteenth-century maps of Sydney municipalities that provide a portrait of the city during a period of rapid growth and suburbanisation. 

The Atlas of the Suburbs of Sydney was created by Higinbotham, Robinson and Harrison, map publishers and lithographers established in 1882 with premises in Macquarie Place. They had obtained permission to produce maps from government survey information, so were able to advertise them as an authoritative source '…compiled from official plans in the Surveyor-General and Registrar-General's offices'.

The Atlas of the Suburbs of Sydney could be considered the first officially published street directory of Sydney. Read more about The Atlas of the Suburbs of Sydney history on dictionaryofsydney.org website. To compare Sydney “now and then” just click on images of old maps and then select "Full record" link to load Google Map interactive application. 

SBS - Census Explorer on Google Maps

Keir Clarke from Google Maps Mania has just written a short post on the SBS - Census Explorer. So far it remains the only substantial mapping applications for exploration of the key demographic indicators from Australia's 2011 Census.



The tool allows users to interrogate the census data and visualise on the map the number of persons using one of several dozens of different languages spoken throughout Australia. Using a drop-down menu it is possible to select an individual language and a coloured thematic drape indicating concentration of people speaking that language is then displayed on the map. By clicking on individual suburbs users can get the exact number of the language's speakers at that location.The application has also a handy functionality which allows to compare two language groups for a given location.

Census 2011 data is out!

Earlier today Australian Bureau of Statistics (ABS) released a first set of information about Australian population and housing based on data collected during a census night on August 9th 2011. Details are fascinating and you could spend hours trawling through thousands of data items describing demographic characteristics of our population - on a national level as well as down to small communities.

Here is some trivia for your information:

• Total persons counted: 21,507,717
• There are more females than males: 10,873,704 to 10,634,013  (50.6%/ 49.4%)
• Median age: 37 (unchanged)
• The most common ancestries in Australia were English 25.9%, Australian 25.4%, Irish 7.5%, Scottish 6.4% and Italian 3.3%. Proportion of people with Australian ancestry decreased from 29% in 2006…
• While the proportion of people born in Australia remains steady at 69.8% (70.9% in 2006), proportions of those born in New Zealand increased by 0.2% to 2.2%, China (excluding SARs and Taiwan) by 0.5% to 1.5%, and India 0.7% to 1.4%.
• 76.8% of people only spoke English at home. Other languages spoken at home included Mandarin 1.6%, Italian 1.4%, Arabic 1.3%, Cantonese 1.2% and Greek 1.2%.
• The median weekly personal income for people aged 15 years and over in Australia was $577. For families, median weekly income was $1,481.
• There were 9,117,033 private dwellings counted and average number of people per household now stands at 2.6 persons.
• There was a staggering number of unoccupied dwellings on the census night: 934,471 or 10.7% of all dwellings. A lot of people on holidays…
• Proportions of people owning a dwelling outright decreased to 32.1% from 34% in 2006, and there were proportionally more renters in 2011 than 5 years earlier: 29.6% and 28.1% respectively.
• Proportions of dwelling with 4 or more bedrooms increased from 28.1% to 30.3% in 2011. One bedroom dwellings are also more popular: 4.7% in comparison to 4.4% in 2006.
• The median weekly rent was $285. For 10.4% of households, their rent was more than 30.0% of their income. The median monthly mortgage repayment was $1,800.

You can explore more interesting facts with this interactive application published by The Courier:

Census 2011: In depth and interactive

But where are the maps, you may ask!? More on this later… here are just a few examples:

SBS: Census Explorer
ABC: Census 2011
Wedding bells falling silent
Old trend no leap of faith

New free weather app

A few weeks ago I was approached by a local web developer with an offer of an alternative front end to my existing weather data service. He needed a simple solution that would fit well with the website design concept for one of the clients and my existing three different versions of the weather app did not quite meet his specification. In a quid pro quo arrangement, I ended up with a new weather app to share with the rest of the world while the client gets the benefit of a tailor-made weather information widget for the newly developed site.

So, here it is - I am officially unveiling weather app v4 for your perusal.





Default setting is for Sydney but you can easily change location by adding a city code at the end the URL link. For example, for Canberra the full address is like this:

http://aus-emaps.com/ww4/index.php?code=YSCB_Canberra

Full instructions on how to set up this weather app on your site and a list of city codes are available on aus-emaps.com free widgets page.

My free weather widgets are quite popular, generating over 1 million page views per month (250,000+ visits). Not surprisingly, also getting some attention from mobile users (almost 10,000 visits in the last 30 days).

This latest arrangement gave me an idea - maybe there would be other web developers interested in designing their own versions of weather app? Hence, I am inviting all interested parties to have a go at creating alternative weather widgets with my data. I will provide a sample code and a copy of the database tables to get you started. Only two conditions – the app has to be hosted on aus-emaps.com and it has to have a link to the site. Keep me posted if you are interested!


Related posts:
Weather widget take 3
Free Australian weather widget
Weather widgets via Google Gadgets
Weather on Google Map
Weather maps for winter resorts
Hazmon weather map

Ingres Db adds spatial capability

The latest version of Ingres relational database now also includes support for geospatial data. GIS functions and spatial data types are included out-of-the-box (i.e. do not require additional plug-ins) and make it easy to enable web mapping. Ingres 10S supports spatial applications, such as Esri’s ArcGIS for Desktop and FME but also MapServer and GeoServer.

Ingres 10S is supported by other programming libraries including GDAL/OGR and GeoTools, allowing import, export and manipulation of vector data. Ingres 10S leverages the GEOS geometry and PROJ cartographic projection libraries for manipulating and transforming spatial data between dozens of geographic and planar co-ordinate systems.

Ingres 10S is an open source, enterprise grade database, not as popular as MySQL or PostgesSQL but, as the other options, can be downloaded and used for free. Ingres was first created as a research project at the University of California, Berkeley in the early 1970s. Since the mid-1980s, Ingres has spawned a number of commercial database applications, including Sybase, Microsoft SQL Server, NonStop SQL, but also open source PostgreSQL (which with PostGIS extension is the most popular open source spatial database).

Map of Melbourne house prices

Every time I come across a data table with information that can be referenced to a location I ask myself a question why the publisher did not consider adding a map? Data presented in spatial format is so much more informative than just plain tables. Yet, still not much “data journalism” is happening in Australia. Agree that more elaborate presentations are time consuming and expensive to create but these days adding a map to an article can be done almost in an instance.

For example, a few days ago I came across a post on propertyobserver.com.au which referenced information from the Real Estate Institute of Victoria on median house prices, and changes over time, for Melbourne suburbs. Since each suburb has well defined spatial extents, the information can be easily presented on a map. Numeric values relating to individual suburbs, such as median price or year-on-year change in price, can be represented with different colours on the map, giving the reader visual clues about how specific suburb compares to other suburbs.

I have created two separate maps with the data. The first shows median house prices by suburb and the second maps changes in prices over the year. There are many different ways how the data could be divided and categorised to present it on the map in a meaningful way. I opted for a very simple approach - median prices are divided into quartiles and changes in prices are split into 5% intervals. In particular, on the first map dark red indicates 25% of the most expensive suburbs (i.e. with median prices over $730,000) and the cream colour refers to 25% of the cheapest suburbs, with median prices under $420,000. The median value of median house prices is $577,000 (i.e. 50% of suburbs on the list have a lower median price than that).



The second map uses contrasting colours to present two extremes: suburbs where prices fell in the last 12 months (in various shades of blue) and suburbs where prices increased (warm colours from pale green, through yellow to dark red). The darker the colour the higher the decline/ increase. Although blue colours dominate on the map, implying more suburbs with declining median house prices than rising, those suburbs are distributed throughout the city without any obvious pattern.



Presenting information on Google Maps can add some interactivity to otherwise static articles and will help to engage the readers. Data journalism with presentation of information on interactive maps and charts has been proved very successful in the US, UK and Canada. Australia is yet to follow in any major way. I will be happy to assist!

Related posts:
Sydney house prices
Maps and property investment
WA housing affordability index
Aircraft noise maps
Mapping sun position anywhere
Gauging Australian property market

Google Public Alerts map

In addition to providing tools for developers, such as maps, databases or spreadsheets with application programming interface (API) -  which can be integrated with third party code and data to create a myriad of custom applications, Google occasionally builds complete applications utilising those tools themselves. Public Alerts map is one such application.

Google Public Alerts is a project of the Google Crisis Response team, supported by Google.org. The key objective behind the project is to make it easy for people to find critical emergency information during a crisis through the online tools they use every day. This new platform for disseminating emergency messages, such as evacuation notices for hurricanes, and everyday alerts such as storm warnings and earthquake alerts initially focuses on the US but earthquake information covers the whole world. The data comes from US National Oceanic and Atmospheric Administration (NOAA), the National Weather Service, and the US Geological Survey (USGS).

Not sure what are the prospects for the take-up of this application since over the years many developers built much more advanced and informative versions, and which are very popular with public at the time of a crisis. Google created several of those themselves, eg bushfire information maps in Australia. Time will show how useful this will become…

Related Posts:
Disasters and maps
Perth bushfires
Japan hit by massive earthquakes
Another disaster looming for Oz

Aircraft noise maps

A new proposal has been tabled by the federal government to vastly expand the areas around major airports that could face residential housing construction restrictions due to aircraft noise. Property research and advisory firm MacroPlanDimasi estimated that $33.5 billion worth of future housing projects (which comprise 134,000 new dwellings across Brisbane, Melbourne, Perth, Sydney and Canberra) could be potentially at risk. If approved, it may have a significant impact on future urban renewal of residential areas directly beneath air traffic corridors in all capital cities – both positive and negative. For example, it may help to preserve the character of older inner-city suburbs but also restrict multi-storey/multi-dwelling redevelopment opportunities.


The report contains a series of maps that depict noise zones around major airports in Australia. These maps on their own are quite a valuable information resource for prospective home buyers and property developers - they will help in assessing the likelihood of aircraft noise in various parts of the city. In particular, the maps show computed outlines of various types of Aircraft Noise Exposure Forecast (ANEF). ANEF is a measure of the aircraft noise exposure levels in decibels around aerodromes based on average daily sound pressure levels:

• ANEF 20 (zone of expected continuous noise exposure over 5-20 years period).
• N70 20 - zone with 20 or more daily events greater that 70 dB(A);
• N65 50 - zone with 50 or more daily events greater that 65 dB(A);
• N60 100 - zone with 100 or more daily events greater that 60 dB(A).

For comparison, quiet bedroom will record 35 dB(A), passenger car at 60km/h 70 dB(A); and heavy diesel lorry at 40km/h 83 dB(A).


Related Posts:
Maps and property investment
Sydney house prices
Mapping Australian social diversity
Coal seam gas exploration map

Driving directions with real-time traffic

Another small improvement has just been added to Google Maps - now it includes estimated driving time based on live as well as historic traffic data.  The information will show up every time you search for directions in the city that already has traffic data layer.

Google Maps used to show estimates of trip times based on historic traffic data alone but it didn't always reflect real-time traffic conditions so, the feature was removed last year. Now Google calculates real time travel duration based traffic data, coming from Android users who have opted to use the My Location feature on Google Maps for mobile, as well as historic records. The new directions feature on Google Maps gives you several different routes to choose from, and each presented with total distance in km and travel time base on average speed as well as based on current traffic conditions. Very handy if time to get to the destination is more important than mileage travelled!

Related Post:
Free GPS navigation in Android phones in Oz
Enhanced landmarks on Google Maps

Modelling city fringes

A critical challenge facing planners in any large and growing metropolitan area is “urban sprawl” and the increasing pressure on rural land and surrounding communities it creates. In order to address this challenge, planners require a better understanding of where future development pressures will be most pronounced and what will be the associated impacts. Atlases for Scenario Planning for Melbourne’s Peri-urban Region is an example of how GIS and spatial analysis can assist in the study of urban sprawl phenomenon and its implications. [“peri-urban - immediately adjoining an urban area; between the suburbs and the countryside”].

Spatial Vision, one of Australia’s foremost GIS consulting companies, working in partnership with the RMIT School of Global Studies, Social Science and Planning, has developed a methodology for modelling peri-urban development pressures, as a central component of the Sustainable Planning for Melbourne’s Peri-urban Region research program (commissioned by the Peri-urban group of councils and funded by Sustainability Victoria through the Victorian Local Sustainability Accord).

The research has identified how and where increases in population (and development demand) are expected to impact on Melbourne city fringes. The results are presented as a series of interactive atlases, comprising maps, tables, graphs and explanatory text, and are organised under the themes of Population Projections, Supply and Demand, Impact Analysis, Offsets/ Mitigations, and Township Analysis (click “Previous/Next” buttons positioned in the bottom of the page to navigate between the themes).

Atlases for Scenario Planning for Melbourne’s Peri-urban Region are built in Flash as an “interactive dashboard”. Google map features prominently on each page, providing spatial context to details presented in a tabular format and on graphs. The application is both, visually attractive and informative, and is a great example of how to present effectively complex concepts and large volume of information. Great resource for local communities, property developers and investors. 

Related Posts:
Maps and property investment
Mapping Australian social diversity
Mapping progress of NBN rollout
Map of landfill and recycle sites

Enhanced landmarks on Google Maps

Google is continuing with the efforts to make its maps prettier and more useable. The latest change is an enhancement of the quality of more than a thousand 3D landmarks/ buildings around the world. Keir Clarke from Google Maps Mania has created a slideshow to showcase the most attractive landmarks around the world. It uses the Styled Maps feature in the Google Maps API to take away roads and other map features to help the 3D buildings stand out. Enjoy.

Related Posts:
New style for Google Maps

Mapping sun position, anywhere

There are thousands of mapping applications on the web, many very pretty. Some are more useful than others. Occasionally I stumble on something that ends up on my list of favourite tools. SunCalc map is one of such applications. It is a simple map that shows sun movement and sunlight phases during any given day, at any given location around the world. There is nothing much to it – just click on the map in the desired location (or search for a specific locations) then select a date and you will be presented with a full set of information about sun movement for that location.

For example, you can see on the map sun positions at sunrise, specified time and sunset. The thin orange curve is the current sun trajectory, and the yellow area around is the variation of sun trajectories during the year. The closer a point is to the centre, the higher is the sun above the horizon (i.e. it is representative of a shadow). The colours on the time slider above the map show sunlight coverage hours during the day and there is a small table showing exact times for dawn, sunrise, solar noon, sunset and dusk for any given date.

SunCalc is a very handy tool for home buyers, architects and builders. Perfect, if you want to determine quickly if the sun can reach all those less then optimally positioned rooms in the house or apartment at different times throughout the year. It will help to avoid nasty surprises if it turns out that your main living area is not getting any sun for 6 coldest months of the year!

SunCalc map appears to be rather basic but believe me, a lot is happening “in the background” to present the information on the map almost instantly. SunCalc is built with Google Map API v3, jQuery javascript library and Raphael javascript vector graphic library. How about reliability of information, you may ask? I can verify that the results are very reliable - I tested the application for my home location. The match between the real direction of the sun in the sky and on the map was very accurate.

First spotted on Google Maps Mania.

Maps and property investment
Sydney house prices
More handy tools
 

Mapping crime in Canberra

Just a couple of weeks ago I reviewed a very impressive UK Crimes Map. Now it is time to put a local equivalent under the microscope. In late February Simon Corbell, ACT Minister for Police and Emergency Services, launched a new interactive online crime mapping tool: Crime Statistics. It has been developed to help provide the public with valuable and accurate information about criminal activity in their neighbourhood. The map provides the local community with crime statistics by suburb, allows drawing comparisons between suburbs and view crime trends over the past five years.

The information presented on the map is quite comprehensive. In particular, the new crime map has data on the number of homicides, traffic and Criminal Infringement Notices, other offences against the person, road fatalities and road collisions with injuries. The information can be viewed as absolute numbers of offences per suburb but also as offences per 1000 of the population, which allows to accurately compare statistics of the larger suburbs with the smaller ones.

Users can select incident type of interest (one or many) and nominate either a year, or quarter of a particular year, to see the total number of those incidents for each suburb in Canberra. The numbers are aggregated to regions on zoom out. The application gives user an option to compare statistics (absolute numbers and rate per 1,000 of population) for up to 3 suburbs. However, trends over time cannot be identified using this application. Overall, the map is simple to use, once you understand the logic how it works, and can give a quick snapshot of criminal activity in the ACT suburbia at a selected point in time.

I would also like to mentioning on this occasion another interactive application with crime statistics for the region which was developed by The Canberra Times. It consists of a series of maps presenting 2011 crime data as thematic overlays. Information is available for patrol areas for the following crime categories: assaults, property damage, theft, car theft, burglary, robbery and sexual offences. Very simple but informative although could be much more useful if data was also presented in a tabular format.

Related Posts:
UK crime statistics revisited