No copyright on databases and maps?

Telstra’s Sensis has just lost an appeal with regard to Federal Court judgement that said its White and Yellow pages are not protected by copyright law. The key to the decision was that those compilations lack “creative spark” and hence cannot be protected under copyright law that requires “independent intellectual effort” to create the works. This ruling has important implications for all kinds of factual data collections, including listings of real estate, names and addresses or… databases with map data. Sensis will be seeking leave to appeal to the High Court to revert the decision.

It begs an interesting question: does it mean that all that high resolution imagery of your neighbourhood and representation of local roads, or cadastre boundaries, or geocoded addresses, are “free for all”? Google and others who publish such information would have no legal grounds for preventing people from copying it all in droves…

Mapping Above and Beyond

Maps break all the barriers…Who would have thought that maps and music can go together! Above and Beyond - Trance Around the World site has just released an interactive Google Map application that shows real-time twits referencing @aboveandbeyond tag from all over the world. The maps also shows locations of major trance music related events and comes loaded with plenty of options for sharing via Facebook and Twitter.


I recently started to listen to quite a lot of trance music on the internet - Above and Beyond weekly sessions in particular. Session #343 in my favourite so far. Now that I think about it, “spatial references” are very prominent on this site: “around the world”, “above and beyond”… so the map was a very logical addition!

Free postcodes with Reference Map

Following on from my yesterday's announcement about free reference map service - just a quick post on how to take advantage of free postal boundaries data that come pre-packaged with it.

To view on a map any of 2,507 postal area boundaries at full resolution (as defined by the Australian Bureau of Statistics for Census 2006 population and housing data) just follow this simple URL pattern:

http://www.aus-emaps.com/svs/ref/map.php?kmzl=2000&svs=2

Multiple postcodes can be added to the map as follows (at this stage one at a time only):

?kmzl=2000,3000&svs=2,2

A set of simple instructions on how to configure other map parameters can be found on aus-emaps.com web widgets page.

Since this reference map can be embedded in a website page, a simple javascript code can add additional functionality to render the content dynamically. It means you can now start serving postal boundaries and/or any other data in kmz or geoRSS format directly on your web site! Here is a link to a simple example showing you exactly how to do it.




And this file will also be handy to script the functionality: list of postal areas (Census 2006 version in csv format). Please note, Northern Territory postal area boundary data are named with leading "0" (ie. 0800).

Related posts
More examples of use of Reference Map service
Australian Postcodes User Guide
Post code maps and population statistics
Free postcode search widget

More practical map apps from Oz

Keir from Google Map Mania blog alerted me about a couple of interesting applications from Australia he reviewed recently. The first is called 'Touring to Australia' and is essentially free online trip and itinerary planner. It allows you to find places to stay and things to do and see on your trip. Just choose your destination and then select from a number of categories of markers to view on the map. You can select from accommodation, excursions, rental services, places of interest and activities. Whilst browsing the map you can add individual hotels and points of interest to your itinerary. Great tool for planning your next trip, business or private.



The second one is called 'Neet Street' and is a tool for reporting community problems which are then passed on to the appropriate authorities. Using Neat Streets you can report road obstructions, vandalism, potholes, fallen trees, abandoned trolleys, litter, and graffiti. You can download a Neat Streets application for the iPhone, Blackberry, and Android based phones. With the apps you can take pictures and submit community problems directly from your smartphone. The latest submitted photos and problems are displayed on a Google Map on the Neat Streets website home page. Neat Streets automatically forwards the reports to the appropriate authority based on the GPS location.

Free Reference Map service

I am launching today my newest creation: free reference map service for publishing and sharing spatial information. I am rushing this release since the end of the year is very close! Currently it only supports spatial data in KML/KMZ and geoRSS format and standard Google base map options but I am planning to add a lot more functionality and content over the coming months. Simple instruction on how to take advantage of available functionality is on aus-emaps.com free web widgets page. I will highlight here only one thing you really need to know to publish and share your own maps: just reference publicly accessible KML/KMZ or geoRSS file as follows http://www.aus-emaps.com/svs/ref/map.php?kmzl= FullURLaddressOfYourFile . It is as simple as that. An example of what you can do with it is below:



Actually, the whole concept behind the service is rather complex (and exciting!) but I will not go into details at this point it time. True, there are many other initiatives, possibly far more advanced, to enable easy creation and embedding of maps but they don’t allow for as much flexibility as I have in mind for my version. My idée fixe with content portability and reusability/repurposing is taking me into uncharted territory… Just consider this option – if you find something of particular interest on a web page, wouldn’t it be nice to be able to “cut it out” and take with you to bookmark just that bit and/or reuse (give another perspective!) in your own online endeavours (whether private or public)? Today I am introducing to the world “detach” concept that will allow you to do just that!

Detach function is a very simple idea, a variation on “link to” if you like, but opens up so many possibilities! If you like the map above, just “detach” it from this page to open it in a new window. Then you can bookmark it or add to favourites for future reference, or shorten the URL with your preferred tool and share via Twitter, or Facebook, or LinkedIn or any other social network – with own commentary and spin! This process can be repeated endlessly… And if you are publishing a blog or forum, you can easily combine a number of layers in a single reference map application to provide your unique content and commentary… Again, this process can be repeated endlessly, and with no need to return to the site where the map was first published…. And if you are a developer, you can insert a script to add content dynamically… Implications of such possibilities are far reaching and will probably be best discussed in a separate post. I will only mention that “detach” is not like “clip content” functionality recently dropped by Google from its range of services when they decommissioned Google Notebook – the emphasis here are on portability of content permitted for sharing by content creator. Various back end services to discover what’s available for reuse are coming in 2011!

I first came up with “detach” concept for YouTube video player app but since it is still in alpha release, I have not widely publicised that option. I intend to enable “detach” functionality for all my widgets so they can be shared and used not only via dedicated personal pages (like iGoogle) or via embedding into web pages but also on their own, without attaching to specific “location” on the web!

As I am yet to make a switch to version 3 of Google Map API this shareable map service is build with version 2 (it has proven quality and reliability although lacks some advanced features of the latest version). Since I am reusing many components from my other applications, it was easier for me to work with that version. I will be switching all my apps to Reference Map platform and upgrade to v3 over the course of next year. Any feedback and comments most welcome!

Generalising geographic data

It often happens that the dataset you have is very detailed and hence very large. Displaying such volume of data in online mapping applications (eg. as KML on Google Map) is out of the question. The only alternative is to reduce the number of vertices to reduce the overall size of the file (ie. generalise the data). There is an excellent free online tool to do just that and surprisingly it works much better than commercial software that costs a fortune. It is called MapShaper.


It accepts spatial data in shapefile format only. It is an online service and you can upload files up to 80MB in size for processing. There are 3 generalisation algorithms to choose from. Activation of generalisation process itself is as simple as moving a slider “left to right” (from 0% reduction to 99%) – the result is visible on the screen instantaneously. Another great feature of the tool is that you can view original file underneath the generalised version to assess the quality of the process but also to pick the right generalisation level for your requirements.

Processed files can be downloaded back in either shp or EPS format (so it’s is also a great tool for straight conversion of files from shp format into EPS format, for use with vector graphics software!). There is simple edit functionality as well so you can add/delete vertices manually, if required. Generalisation process preserves topology so boundaries of adjoining polygons are generalised in an identical fashion – that is, they still “fit” perfectly. All in all, MapShaper is very straightforward to use.

File size still counts when performance of online mapping tool is the objective. For example, it will not be possible to display electoral boundaries as a single, full resolution file in Google Map application. But original 25MB shapefile can be reduced to just a few hundred KB using MapShaper tool, and then even further by converting shp to gzipped KMZ format. Here is an example of a Google Map application that uses both, full resolution data for close-ups and generalised version for whole of the country view: Australian elections map (work in progress!).

Related posts:
Free map service preview
Converting csv data into shapefile
Converting shapefile into KML
Colours for thematic mapping
Free GIS Tools – Google Map
Free Address Validation Tool
Manual geocoder for 70 countries

Converting shapefile into KML

In the inaugural post in the Free GIS Tools series I presented Google Map API as a very powerful online GIS alternative. Google Map has the capacity to display spatial data in various formats but probably the most attractive option is its native support for KML/KMZ. It takes only one line of code to display complex spatial information on a Google Map – as long as it is in that format. True, not all KLM features are supported in Google Map but you can still create great maps using a single KML file. Try aus-emaps.com free map service to test how well Google Map handles KMZ files: Victorian bushfires aftermath, current earthquakes.

KML/KMZ is the only spatial data format directly supported by free Google Map and Google Earth desktop application. However, a lot of free spatial data is distributed in another very popular GIS format: shapefile (shp). In order to display that data in Google Map or Google Earth, it has to be converted to KML/KMZ format.

There are many tools available for download, or that are bundled with commercial software to do that conversion, but I found only one that really work for me. And the best thing is that it is totally free! It took quite an effort to find so, by sharing this information I hope to save you many hours of fruitless trawling through the Internet. Here are the details.


ESRI Shapefile to KML Converter by reimers.dk

Simple but effective desktop tool (it works!) available for free download. Just point to the shp file you want to convert, select attributes to include in description field and press a button to run conversion. The only limitation is that you don't have a choice in selecting colours.




Here is an example of generated output: World Borders in KMZ (1.14MB kmz file draped on Google Map).

To reduce the size of KML file for distribution or deployment just gzip it. It is easy done using Google Earth “save as” function. That is, open your KML file in Google Earth and when you save it, it will be automatically converted to a compressed gzipped format with kmz extension.

For custom conversions from shp polygons to KMZ format, such as splitting a large file with many polygons into smaller files with individual polygons only (eg. to convert a single 60MB shp file with postcodes into 2,500 individual KMZ files), or grouping of polygons into custom areas (eg. to create sales or franchise territories by amalgamation of postcodes), or converting shp polygons into a thematic map layer (eg. by assigning colours to polygons based on specific attributes, like for example electorates by political party holding the seat), you will need more advanced tools. I can help you with those conversions for a small fee. Here is a sample of a thematic KMZ file using a selection of Sydney suburbs. I also have various polygon based data in KMZ format for immediate delivery (eg. postcode boundaries, as used in aus-emaps.com postcode finder service). Don’t hesitate to ask!

Use case example: Create outlines of polygons and show in an embedded map on a website (using collection of free tools and services available from aus-emaps.com)
Steps:

1. Define vertex coordinates of custom polygon area using simple geocoder tool from aus-emaps.com (select “Point” option and click on the map, reposition markers as required, copy to Notepad).

2. Convert a list of points in csv format to shp file format using instructions provided in a recent post titled “Converting csv files into shapefile”.

3. Convert shp file into KML using tools introduced in this post.

4. Convert KML file into KMZ format using Google Earth (open KML file and “Save As”…) [optional step]

5. Upload KML/KMZ file to your server and reference its URL address in an aus-emaps.com free map service.

6. Embed the map in you web page using HTML’s iframe element.

The process is a bit convoluted, I admit. You could take a shortcut by pasting latitude and longitude coordinates directly into KML “shell” but you will need the right template for this in the first place. Using Google Map’s MyMap drawing and embedding functionality is a simpler option but you will not be able to create complex polygons (eg multi-part and/or doughnut type).

I am currently working on a public tool to generate custom polygons from administrative boundaries in a few simple steps. Coming up in 2011! But I will also have to put some though into creating a dedicated KMZ generation tool so you can do it all from a single web page... added to the list of “Things to do!”.


Related posts:
Free map service preview
Converting csv data into shapefile
Colours for thematic mapping
Free GIS Tools – Google Map
Free Address Validation Tool
Manual geocoder for 70 countries