WP3- Setup and design of Spatial Data Infrastructure to support project
This is a Work Package as part of the OSSTIP project.
Inputs:
- Open source geospatial software (both server-oriented and interactive),
- sample datasets and formats from OSSTIP/WP1,
- report of required software dependencies from OSSTIP/WP2.
Outputs:
- Saved install files for chosen tools,
- documentation for setup,
- (possibly) a Linux Virtual Machine (VM) with necessary tools pre-installed.
Estimated Time: Small-medium
Status: In Progress, largely complete (2014-02-07)
Requirements Summary[edit | edit source]
The goal of this work package is to design and document how to set up a method for effectively storing and working with the GIS spatial data involved in the project. This is known as a Spatial Data Infrastructure (SDI).
In the case of this project, we'd want the simplest possible system that still provides necessary good management of the data for a ~6 month project involving transport network data analysis. Emphasis should be given to a system that's feasible to maintain with as little specialist System Administration skills as possible – at least after initial setup. However, supporting robustness and good practices of data maintenance (such as storing Metadata) also need to be considered.
The three key components of the system would be:
- A preferred tool to interactively edit and view GIS data on local PCs. For example, Quantum GIS (QGIS) or GRASS.
- An approach, and/or software infrastructure, for storing key GIS data so that multiple people can use and modify it during the project – along with necessary Metadata. This could be as simple as a common storage area for shapefiles (e.g. on Dropbox) and convention for storing updated versions – through to a specialised GIS server application (such as GeoServer) that incorporates this functionality and provides Web Services access to the data.
- A method or platform for displaying transport network GIS data on the web, as part of indicator platforms or other analysis. Again, simpler approaches for this might be converting the data to KMZ format for display in Google Earth, whereas a software approach to automate this could be to use GeoServer and integrated software such as OpenLayers to display the data.
An important factor in determining the best system should be integration with the GIS-T software evaluated in OSSTIP/WP3 in mind. For example, OpenTripPlanner is developed by the same organisation that supported GeoServer's early development, so is likely to integrate well with this platform.
Added notes:
- An Australian project to support the rollout of such a system on a larger scale is the AuScope SISS project - http://web.archive.org/web/20210115153909/http://siss.auscope.org/
Results[edit | edit source]
2013-10-24 update[edit | edit source]
We haven't yet formalised an overall system for both managing project work and display of final data. Expect this WP to be progressively updated throughout the project. Key aspects are:-
- A shared Dropbox folder, shared from Pat S's dropbox with all project participants at PhD-TechnicalProjectWork/OSSTIP_Common, containing:
- General planning notes
- Relevant technical notes for each WP, more detailed and/or draft form than on this site;
- Original forms of key data (as discussed in OSSTIP/WP1);
- Smaller data-handling and conversion scripts;
- Some of the project's GIS files and qGIS 'projects' to bring them together.
There is also a Dropbox for each participating organizations with more details relevant just to their projects:-
- PhD-TechnicalProjectWork/OSSTIP_BZE (Includes some Netview notes/data)
- PhD-TechnicalProjectWork/OSSTIP_PTUA
There are also several projects on Github relevant to the project, such as:
- https://github.com/PatSunter/OSAT - Pat's clone of the OSAT project, with the intention of updating to work with the latest OpenTripPlanner.
- https://github.com/PatSunter/OTP-Routing-Tools - Saved processing scripts relevant to doing routing processing in OTP.
- https://github.com/hsenot/nemtools - tools to convert between shapefiles and Netview's NEM format.
We are all de-facto standardising on QGIS as our GIS tool as well.
So things that need thinking about semi-urgently:-
- A better approach to store the conversion and post-processing scripts from OpenTripPlanner to help analysis and display in QGIS that Pat's produced for OSSTIP/WPBZE2. Perhaps saving on a Github repository, as per nemtools, would be a good idea?
- 24/10/2013 - done - https://github.com/PatSunter/OTP-Routing-Tools
- Setting up some Virtual Machines, especially for a 'production' OTP environment for the web. Probably using VirtualBox as the FOSS solution.
Longer-term issues:-
- Should we formalise the handling of GIS data even more?
- A workflow for saving all valuable exploratory info/examples of relevant Transport Informatics tools onto VMs for distribution (see OSSTIP/WP2. This is probably going to involve a repository or server space setup beforehand.
2014-02-07 update[edit | edit source]
No significant progress here since the last update - have generally been keeping data files on the Dropbox reasonably well, and storing important source code where relevant in Github repositories. Links to those posted on individual pages.
We don't need to formalise this too much more in the scope of this PhD project I believe. What may be useful is:
- A little bit more organised convention for storing different iterations of development of the public transport network shapefiles, and associated data.
- Some brief documentation of this.
- Storing some Virtual Machine exports of running systems at different points in time may actually be really useful. Could also assist with sharing the platform around different machines as well. Suggest using a FOSS VM solution such as VirtualBox.
2014-03-19 update[edit | edit source]
We have now created Virtual machines for web-based display of data in OpenStreetMap, using VirtualBox. These are based on the 12.10 distribution of Ubuntu Server. Have created two currently:
- ubuntu-12.10-server-opentripplanner (this one loaded with the BZE V1 bus network redesign)
- ubuntu-12.10-server-opentripplanner-ptv (this one loaded with all PTV networks and timetables)
I have configured them to be available over the "host network", which means they will be given a unique IP address internal to your local virtual network, and can then be queried via a web interface (even though the traffic will actually all be local to your machine). For example, to access a VM running at I.P. 192.168.56.101, you would use the address http://192.168.56.101:8080/opentripplanner-api-webapp.
What would be good to do next is:-
- Investigate if by using the https://github.com/opentripplanner/OpenTripPlanner/wiki/MultipleGraphs capability, we can create a VM that can serve both the existing networks, and various permutations of BZEs changes;
- Properly document the VM creation process;
- Also create a "development" Ubuntu installed, with something more like the full OSSTIP set of utilities and workflows installed that we're using at this stage. I.E., something like a cut-down version of the OSGeo Live CD.
Also, we need to ensure to archive the appropriate completed VMs at Beyond Zero Emissions offline somewhere.