From Open Data to Big Data

At all levels, governments around the world are opening their data in order to better engage their citizenry and foster the development of new solutions to long-standing problems. If governments are serious about this mission, and if they want to harness the true power of open data, they must release with intent.

Since Philadelphia Mayor Michael Nutter adopted a formal open data policy in April of 2012, we at the City of Philadelphia have taken steps to build an effective and durable open data infrastructure. Working with partners in both the private and public sector, we make our data available not only on static datastores, but via dynamic channels, allowing for continued feedback and collaboration. OpenDataPhilly, the portal for open data in Philadelphia, allows users to offer comments, suggest and nominate new datasets, as well as submit project examples. More recently, we have begun to leverage the public software code sharing platform GitHub as a new platform for our open data. As we continue to utilize new future, citizens will be able to see how data grows and changes, allowing for another level of transparency and engagement.

The true power of government data is unlocked when it is used to build new solutions, services, and visualizations. Rather than simply dumping giant sets of data into public repositories, our releases are designed with specific intent – to be readily accessible and digestible so that they can be easily incorporated into any number of different civic projects. Some of our most dynamic datasets are available through a number of Application Protocol Interfaces (APIs), allowing users to quickly retrieve data in as little time as it takes to write a query. These interfaces provide real-time updates, allowing developers to build apps that reflect changes in the data over time.

AxisPhilly's Crime Change App is a great example of how city data can be leveraged by the public.

AxisPhilly’s Crime Change App is a great example of how city data can be leveraged by the public.

Released in concert with the Philadelphia Police Department, the Philadelphia Crime Data API has proven to be one of our most popular offerings, with the data serving as the platform for solutions built by both government and outside developers from around the Philadelphia region. Axis Philly’s Crime Change Map, built by developer Casey Thomas, shows percent change for different types of crimes by neighborhood over time, allowing for both temporal and spatial comparison. Harnessing the dynamic nature of the API, developer David Walk put together PHL Crime Mapper, which allows users to search for all crime incidents by extent and time, with results as current as two days and as old as three years prior. Coming from outside of the city and highlighting the API’s utility to civic hackers, Boston coder Nick Doiron created a mashup of crime hotspots and Google routing data to show the relative danger of a user-defined walking route. Solutions like these highlight the power of data when it is released with the specific goal of being used.

Uses of city data such as those mentioned above not only create valuable services for our citizenry, but also greatly inform the work we do in city government. Outside developers and researchers coming from diverse sectors and backgrounds can see new uses for data that we may not have thought of, as those closest to a problem may often miss opportunities for innovation.

As more of our data is released with the intent that outside developers will make use of it, an added – almost unanticipated – benefit of open data is that we are starting to see opportunities for new internal uses for our data. Our work to make data more usable benefits not only those outside, but data users in other departments and agencies of government. This helps underscore the connection between the work being done on open data and the promise of “big data” to help enable better decisions and investments by government.

Open data programs help governments think more strategically about how their data is used. This will have ramifications for how those outside government make use of data, and for how we use it ourselves.

2 thoughts on “From Open Data to Big Data

  1. Pingback: Announcing New Spatial Web Services | PhillyInnovates

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s