A project done as part of the Twin Cities Code For America Hackathon
See The Meetup for more details on the event.

The short term goal was to extract crime data from the weekly Excel file produced by the St Paul police department. See our map for a map of this data displayed on the CartoDB service. Ultimately, we want to provide this as a data feed for timely crime alerts and maps.

There is a separate scraperwiki project that combs through the St Paul web site and finds the URL for all of the spreadsheets. This list is the basis for the processing done by the StPaulCrimeStat scraperwiki.

We did not get time to deal with two data quality issues. First, some addresses have street names that we can’t geocode because there are missing spaces, such as “OLDHUDSON RD”. Second, some address are an intersection and only the first part is shown in the report, such as “MARION ST &”

Update: Google’s geocoding service seems to handle these better than the arcgisonline geocoding service we were trying. If you take the data and import them into a Google Fusion table, it seems to handle most of the strange addresses.

There is a java application hosted on GitHub that illustrates how to query the fusion table and provide the user with a list of crimes near an address.

Forked from ScraperWiki

Contributors michaelaltmann

This scraper has not yet been run

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (0 Bytes) Use the API

rows 10 / 54264

DISP CODE DESCRIPTION INCIDENT_DESCRIPTION DC RAWADDRESS GRID ADDRESS DATE CN NEIGH_
A
1800
Narcotics
Narcotics
1.0
36X PEDERSEN ST
120
360 PEDERSEN ST , St Paul, MN
2011-02-17T10:45:06
11031987
A
1800
Narcotics
Narcotics
1.0
215X WILSON AV
120
2150 WILSON AV , St Paul, MN
2011-02-17T16:15:45
11032218
RR
710
Auto Theft
Motor Vehicle Theft, Automobile
1.0
206X MARGARET ST
100
2060 MARGARET ST , St Paul, MN
2011-02-18T04:00:00
11032589
RR
861
Other Asasult Dom.
Assault, Domestic, Opposite Sex
1.0
29X BIRMINGHAM ST
117
290 BIRMINGHAM ST , St Paul, MN
2011-02-18T16:00:00
11032915
RR
631
Theft
Theft, Shoplifting, Under $500
1.0
213X OLDHUDSON RD
120
2130 OLDHUDSON RD , St Paul, MN
2011-02-18T17:20:00
11032969
R
1400
Vandalism
Criminal Damage to Property
1.0
177X OLDHUDSON RD
118
1770 OLDHUDSON RD , St Paul, MN
2011-02-18T18:02:17
11033002
RR
651
Theft
Theft, Auto Accessories, Under $500
1.0
17X MCKNIGHT RD N
140
170 MCKNIGHT RD N, St Paul, MN
2011-02-18T18:33:00
11033026
RR
430
Agg. Assault
Aggravated Assault, Other Dangerous Weapons
1.0
215X WILSON AV
120
2150 WILSON AV , St Paul, MN
2011-02-18T18:47:00
11033042
RR
861
Other Asasult Dom.
Assault, Domestic, Opposite Sex
1.0
47X HAZEL ST N
119
470 HAZEL ST N, St Paul, MN
2011-02-18T19:14:00
11033061
RR
210
Rape
Rape, By Force
1.0
2XX KIPLING ST
160
200 KIPLING ST , St Paul, MN
2011-02-19T03:00:00
11033337

Statistics

Total run time: less than 5 seconds

Total cpu time used: less than 5 seconds

Total disk space used: 26 KB

History

Scraper code

Python

st_paul_crime_stat / scraper.py