OSSTIP/Project blog: Difference between revisions

Intro

This is a blog-style page by PatSunter about publicly-shareable learning, information and reflections arising from the OSSTIP project.

The idea is to reflect on and share useful ideas about the tools evaluated in the Open Source GIS-T Public Transport Tools Review, and how well they support my project collaboration partners (Beyond Zero Emissions and the Victorian Public Transport Users Association) in developing alternative paths forward for Melbourne's transport system. It is a component of the "Action-Research" aspect of my PhD.

If you'd like to comment at the end of entries :- that's fine, but please use Appropedia conventions of signing your posts using the --~~~~ formatting.

Entries

2013-07-18 :- Working with GTFS transport schedules, and promising start with OpenTripPlanner's capabilities

Last week was a good week working on the project with Beyond Zero Emissions (BZE). They are working in parallel still on their high-speed rail study, but also have a couple of people already working on their Melbourne case study of options for de-carbonising transport in the Melbourne Metro region.

Learning about GTFS, and relevant tools like TransitFeed

Thus far, my focus has been on setting up and evaluating Open Source tools that can effectively work with public transport networks and timetables (in GTFS form), and perform analysis thereof (OSSTIP/WP3) - and the data these tools need/use/produce (OSSTIP/WP1).

In this I had the much appreciated support of a friend who has developed Python scripts to transform the the database of Melbourne's public transit we had access to into GTFS format. His scripts used some open-source libraries and tools, TransitFeed. The latter include some nice validation and data exploration tools, like the ScheduleViewer web app (see screenshot below).

The data still needs some work, especially the bus network - but it was enough to start the next step. I would say that just by using the tool for a while, it helped get a better handle on the geography and layout of Melbourne's public transport system. This chimes in well with one of developer Andrew Byrd's stated goals for OpenTripPlanner Analyst, of being able to "better grasp the syntax of the urban environment and factors favoring spatial legibility for those of us navigating it via shared, scheduled and non-motorized transport."

This included for me realising just how complex Melbourne's current bus network is :- as one example, I noticed several route numbers are duplicated between areas of Melbourne and Victoria's regional towns. So improving the bus network does seem a worthwhile area to focus effort moving forward. I realise given Melbourne's history of privately-owned buses, the institutional aspects of this will be tricky, as my GAMUT colleagues like to point out ;) However, at least with this project we should be able to propose some useful ideas for the network itself.

First tests of OpenTripPlanner

The next step was exploring which OS tools best supported analysis operations on the data such as routing and calculation of isochrones etc, as per project WP2 :- Open Source GIS-T Public Transport Tools Review.

In this respect, OpenTripPlanner has offered the best mix of performance/usability so far. I did check out GraphServer, which might require less dependencies - but does seem more of a research, "DIY-tool", without too much user docs or 'hand-holding' to get you up and running. In contrast, OpenTripPlanner, while not having a perfect manual, has put in quite a lot of effort it seems e.g. by having an entire distribution you can download and install in one step, and also they provide "2-minute intro" and "5-minute detailed dive in" tutes on the project Wiki (see either of those pages for links to the download). For an Open Source project, I like this approach, as it helps people get over the initial barrier of using the software.

Note :- you are going to want to have a good internet connection though for this - the binary distribution is ~60megs, and the sample OpenStreetMap and graph files are 100mb+.

By mid last week I was able to get it to calculate and visualise travel time maps based on Melbourne's current PT network – e.g. screenshot attached of trip isochrones to Monash Uni Clayton.

My next step will be to investigate the ability of OpenTripPlanner, or possibly the OSAT plug-in developed on top of it by a US researcher (https://github.com/Noblis/OSAT), to do TAZ-based trip distribution calculations using OpenTripPlanner. While OTP is Java-based and is oriented towards presenting results on the Web using a REST design, it can be scripted to run in batch mode for these types of calculations. We are keen to test performance and see how it stacks up against the other tool the BZE guys are using for this.

1.6 GPM or less low-flow shower heads

1.8 Ford indirect injection with Bosch rotary pump

11 core principles for Sharing Cities

12 Lead EKG Interpretation

12 Lead EKG Interpretation Quiz

120V Photovoltaic Microinverter Literature Review

12NA

1973 energy crisis

2 FT Prosthetics

2-stroke Diesel engine

2009 IEEE Toronto International Conference Science and Technology for Humanity

2009 Victorian bushfires

2010 Xinfa aluminum plant protest

215 Accordion photovoltaic portable power

215 Appropedia

215 Cassiopeia solar apocalypse computer

215 DIY solar power box fan air filter

215 Emergency photovoltaic box

215 Mahana solar window fan

215 Solar for low power medical equipment

More results

Data Conversions and Github

Notably, a lot of the 'Open Source' work so far has involved scripts to transform data from one format to another – which gels with what full-time GIS project people tell me as a major time-absorber.

I was pleasantly surprised though that another BZE volunteer working on some of these already added them to a version controlled Github repository without my intervention – so BZE's first open source project was actually endogenous ;)

2013-08-29 :- Making some progress with OTP Batch Analyst for calculating OD Matrices and more advanced visualisations

Map of Melbourne's Train Station Accessibility from Other Train Stations

Probably the best test so far on OSSTIP/WPBZE2 to test OpenTripPlanner's batch routing capabilities is this example of displaying Melbourne's train station accessibility from other train stations. The values vary from between just over 40 minutes, to a bit over 2 hours.

(It's a very simple metric thus far, as it takes no account of the 'value' of the other destinations, either for their 'centrality', or job/population catchment).

I used QGIS to create the visualisation :- it's really quite a capable tool once you get over the learning curve.

My Email about OTP's default maxTransfers value

(Moved to OSSTIP/OpenTripPlanner Issues#Concerns about maxTransfers value too low by default).

2014-03-18 :- Updates on last ~6 months of work with BZE

Progress on using OpenTripPlanner, reflecting on the learning process

The last six months work on trialling the OSSTIP platform for use within Beyond Zero Emissions's (BZEs) Zero Carbon Australia Transport project (ZCATP) have been both busy, and productive.

As detailed in some of the Work Package notes like OSSTIP/WP2 and OSSTIP/WP4, we've settled on using OpenTripPlanner (OTP) as the main journey planning and network analysis tool for purposes of analysing the effectiveness of BZE's network and schedule improvement proposals. OTP's core routing capabilities are good - when you delve a little into the relevant literature of routing on large timetabled networks, it is really quite a challenging task - so it is a big help we can use the concentrated capabilities of the collective contributors to OTP over the years who've developed a workable approach to these issues.

Of course, learning how to use OTP certainly hasn't been without its challenges - I for example have had to re-familarise myself with the Eclipse Integrated Development Environment as the recommended way for working with the codebase, but getting started instructions here are reasonably well-documented on the OTP wiki. As someone whose main programming background was in developing either high-performance computing server applications and frameworks in the C or C++ languages (Underworld and StGermain), and then later various workflow scripting, analysis and workflow tools in Python for scientific computing (CREDO). Fortunately, one of my latter projects at my employer before I began the PhD, wikipedia:VPAC, included further development of a Java Tomcat-based combined web and server application with at least some similarities of underlying structure to OpenTripPlanner, so that gave some useful experience in 'wrangling' this kind of codebase to do what you'd like to do.

As I've started to use OTP more regularly, I've started to follow the OTP-user and OTP-developer mailing lists more closely, to follow the project's progress. There are several interesting things happening to the project as they work towards a 1.0 release.

• One of these is that the 1.0 software architecture has been significantly redesigned compared to the 'stable' branch that I've been using in OSSTIP work, 0.9.2. There have also been discussions recently about a continued and perhaps more radical simplification of the configuration and publishing aspects of the application in future, to try and reduce some of the complexities involved given its initial design as a Java Tomcat servlet and build tools like Maven. There's also been talk of a more advanced API for representations of mode change requirements during travel, as well as ways to be more efficient with RAM memory while still maintaining speed to allow dealing with ever-larger graphs.
• I think what all of this illustrates is the fact that large open source codes are complex beasts, and that successfully dealing with complexity is a big challenge and the ways developed to do that successfully has been one of the most interesting things to come out of the free-and-open-source software world. As with all [Open source software], there is a challenge between stability and merging new features and improvements, and the governance of this given different interests of different participants in the project. Free, or low-cost, configuration management and issue-tracking tools and platforms like Github are definitely a big part of this picture of allowing global collaboration, but each project has its own culture to understand and work with too. This is one of the things discussed in the book Understanding Open Source Software Development that I'm currently reading, as well as Karl Fogel's Producing Open Source software book - the latter freely available over the web and published under a Creative Commons linense.
• Of course, as initially "users" of OpenTripPlanner in this project we've tried to abstract ourselves from some of this development complexity, and have to make decisions on how to best do this, e.g. choosing to use a stable 0.9.2 version instead of the latest development 'trunk' version. However, as we increasingly become 'power users' of the code pushing its capabilities in areas like batch routing and analysis, and potentially even developers of new features, we do need to get increasingly involved in these kinds of discussions. That's going OK in the context of the BZE work so far, but is definitely a challenge to balance a close involvement with the internals of the code with 'getting things done' in the kind of context needed by BZE and other civil society organisations. One of the things I'd like to reflect upon later in the project is whether BZE, myself, and other local stakeholders in OSSTIP would need to work together in some kind of collaboration to be able to keep working on this kind of thing longer term.

An interesting governance and 'mode of production' change in the OTP project over the last 6 months has been the spinning out of the project's from its former institutional 'home' as part of the organisation OpenPlans, (originally the 'Open Planning Project'), into a new dual structure, as initially announced in this mailing list post:

• The transfer of the rights to the code to the Software Freedom Conservancy (SFC), an organisation set up expressly to create a legal structure for management of FOSS codes long-term;
• Simultaneously, the set up of a new consultancy company focusing on transportation software development Conveyal, 'an open data / open source focused transportation

consultancy that has taken on the work we began together at OpenPlans'.

While this is a complex issue, one preliminary opinion I'd proffer is that this approach to long-term sustainability of the project does compare favourably to that adopted by the UK CSO MySociety in their handling of Mapumental, which after an initial period of Open Source work chose to continue work on the codebase in a proprietary manner in a spin-off Mapumental company. I understand there were some grant conditions on their initial funding that constrained this choice though, so it is a complex issue.

At any rate, thus far OTP's change of governance structure hasn't had any impact on the code's continued availability for use and further development.

Work on displaying 'Virtual Future Timetables' in OpenTripPlanner, and feedback on this as a communication tool

Of course our main aim in using OpenTripPlanner is as part of a larger workflow analysing and communicating the potential benefit of revisions to Melbourne's public transport network, structure, and frequency.

In this respect, the BZE transport team have been making extensive use of Quantum GIS as our desktop GIS tool to manage designing modifications to networks, lines, and interchanges. This has probably been one of the significant aspects of the project in terms of user time.

Then, with a lot of help from one of the other BZE project volunteers, we developed a workflow to convert these networks via a set of stages into a new system-wide timetable, written mostly in Python and drawing on other GIS software. There is some description of this process in OSSTIP/WP5. N.B.: - Calling the repository of scripts I created for this process 'SimpleGTFSCreator' was deliberate, as the timetable creating capabilities are currently pretty simplified, e.g. currently we don't support issues like express services.

Nevertheless, it was exciting to be able to start checking the routes and times in the Google Transit Data Feed ScheduleViewer tool (see fig below), and then to start comparing how journey patterns would change in this future 'Virtual Timetable' as compared to in the current network (example included below). This means that task OSSTIP/WPBZE1 is now largely complete, bar better documentation.

Myself and BZE have now demonstrated this kind of functionality in two talks, one a collaborative talk to the Melbourne Metropolitan Transport Forum in December 2013, and a second by BZE at the 2014 Sustainable Living Festival. See OSSTIP#Talks, Media, and Academic Publications Related to the Project. Preliminary indications suggest this will be a valuable way of communicating the impact of timetable and network changes to stakeholders and the public.

OpenTripPlanner has also proved useful in assessing how effective the network redesign would be in serving different trips over the Melbourne Metro area - using a "batch routing" capability extended from work in OSSTIP/WPBZE2. It suggested that in many areas trip times were reduced on the redesigned network, while there were some places where trip times increased (mainly concentrated in the outer Northern suburbs, where it seems we made the bus network too sparse).

So, work is already underway at BZE to create a "Version 2" bus network re-design that aims to address some of these issues - as well as different permutations that include light and heavy rail network upgrades. More on this next update.

Virtual Machines, and Working towards a launch of an OTP server with BZE's work

TODO :- (Virtual machines)

... moving towards more elegant maps is a future task and goal .... OpenStreetMap aspect ...

And while I remain impressed with the Analyst Isochrones capability, doing some detailed testing has revealed some issues with the results that I've raised with other OTP developers to work on resolving.

Some issues with OTP Isochrone calculations at different times of day I've raised with the OTP community

Per OSSTIP/OpenTripPlanner_Issues#Concerns with accuracy of OTP Analyst Isochrones in late evening or overnight - want to resolve these as accurately as possible in coming months ...