Adding Socrata to the Mix

Four years ago on April 25, 2011, Philadelphia’s technology community launched OpenDataPhilly, a central clearinghouse of datasets pertaining to Philadelphia produced by government, nonprofit organizations, universities, and others. A year later, Mayor Nutter signed an executive order on open government and prioritized open data within the administration.

Housing the new open data program within the Office of Innovation & Technology presented multiple options for publishing data, but the City decided to continue publishing its data through the community-run portal instead of building or buying its own. Today we remain the first and only big city with a community-run open data portal. This collaboration is important to us, so we had to figure out how to implement the Strategic Plan we published in October while maintaining this commitment.

Specifically, how to:

  • Automate the refresh of datasets – only 13 of the 171 datasets we publish are currently automated. The rest are refreshed manually or not at all.
  • Make every dataset available for download in standard formats – some of our datasets are only available as an API. Making them available for download currently means manually generating each file format (CSV, SHP, GeoJSON, etc.).
  • Provide an API automatically for each dataset – Over 100 of our datasets are only available for bulk download, and creating custom APIs for every one is not manageable.
  • Connect the general public to open data through visualizations – currently we have to build visualizations individually, so most datasets don’t have them. As a result, users have to know what to do with a CSV file to interact with open data (or someone else has to build a visualization).

Philly already has a great portal to browse datasets that are available, particularly since February’s relaunch. What we needed was a way to improve the quality and accessibility of our data releases.

To that end, we’re excited to announce a new partnership with Socrata, a national leader in open data software. We will leverage their new infrastructure to store (and automatically refresh) the datasets we release, then push the links to OpenDataPhilly. As a result, users browsing OpenDataPhilly will be able to (1) download each dataset in multiple bulk formats, (2) work with the data via a powerful API, and (3) interact with the data through Socrata’s new visualization tool, DataLens.


Because these changes are all beneath the data portal layer, most users won’t even notice they’re interacting with a new product (in fact, we’ve disabled the typical Socrata landing page); they’ll just notice more consistent options in City datasets at OpenDataPhilly and a really nice way to visualize the datasets. But internally, this enables us to meet the goals described above and leap ahead to broader objectives.

Today’s debut coincides with the release of a major dataset from the Finance Department’s Bureau of Administrative Adjudication: Parking Violations. This release allows us to illustrate the benefits of this platform:

  • All 4.8 million records are available for bulk download in CSV and GeoJSON formats


Data Lens Demo

You can expect these features for future data releases, and we’ll work on getting the already-released datasets in here as well. We look forward to your feedback as we continue to roll out this platform, and to the great things you’ll do with open data.

4 thoughts on “Adding Socrata to the Mix

  1. Pingback: City’s Open Data Advisory Group Meets | PhillyInnovates

  2. Hello! Just wondering if my earlier comment is still being considered or its been decided it will not be published at all. Thanks, Joel

  3. Pingback: Philadelphia Visualization Tool Boosting Open Data - EfficientGov

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s