Four years ago on April 25, 2011, Philadelphia’s technology community launched OpenDataPhilly, a central clearinghouse of datasets pertaining to Philadelphia produced by government, nonprofit organizations, universities, and others. A year later, Mayor Nutter signed an executive order on open government and prioritized open data within the administration.

Housing the new open data program within the Office of Innovation & Technology presented multiple options for publishing data, but the City decided to continue publishing its data through the community-run portal instead of building or buying its own. Today we remain the first and only big city with a community-run open data portal. This collaboration is important to us, so we had to figure out how to implement the Strategic Plan we published in October while maintaining this commitment.

Specifically, how to:

• Automate the refresh of datasets – only 13 of the 171 datasets we publish are currently automated. The rest are refreshed manually or not at all.
• Make every dataset available for download in standard formats – some of our datasets are only available as an API. Making them available for download currently means manually generating each file format (CSV, SHP, GeoJSON, etc.).
• Provide an API automatically for each dataset – Over 100 of our datasets are only available for bulk download, and creating custom APIs for every one is not manageable.
• Connect the general public to open data through visualizations – currently we have to build visualizations individually, so most datasets don’t have them. As a result, users have to know what to do with a CSV file to interact with open data (or someone else has to build a visualization).

Philly already has a great portal to browse datasets that are available, particularly since February’s relaunch. What we needed was a way to improve the quality and accessibility of our data releases.

To that end, we’re excited to announce a new partnership with Socrata, a national leader in open data software. We will leverage their new infrastructure to store (and automatically refresh) the datasets we release, then push the links to OpenDataPhilly. As a result, users browsing OpenDataPhilly will be able to (1) download each dataset in multiple bulk formats, (2) work with the data via a powerful API, and (3) interact with the data through Socrata’s new visualization tool, DataLens.

Because these changes are all beneath the data portal layer, most users won’t even notice they’re interacting with a new product (in fact, we’ve disabled the typical Socrata landing page); they’ll just notice more consistent options in City datasets at OpenDataPhilly and a really nice way to visualize the datasets. But internally, this enables us to meet the goals described above and leap ahead to broader objectives.

Today’s debut coincides with the release of a major dataset from the Finance Department’s Bureau of Administrative Adjudication: Parking Violations. This release allows us to illustrate the benefits of this platform:

• All 4.8 million records are available for bulk download in CSV and GeoJSON formats

• There’s a powerful, well-documented API out-of-the-box to use this data in other applications. For example, you can:
  • retrieve the census tracts with the most tickets written,
  • find the most common types of violations since January 2015, or
  • get a list of all the tickets written for out-of-state vehicles within 1 mile of City Hall
  • (On those links, tick the “I’m not a robot” box and click the Launch button)
• You don’t need to know what to do with a CSV file to consume the data

You can expect these features for future data releases, and we’ll work on getting the already-released datasets in here as well. We look forward to your feedback as we continue to roll out this platform, and to the great things you’ll do with open data.