philipashlock / DemocracyMap_US_Counties

US County profiles and officials

Scrapes explorer.naco.org

Explore your county through over 500 indicators and nearly 70 datasets.


This is a scraper that runs on Morph. To get started see the documentation

Contributors philipashlock

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to /app/.heroku/src/scraperwiki  Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 8))  Downloading lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1 (from -r /tmp/build/requirements.txt (line 9))  Downloading cssselect-0.9.1.tar.gz  Collecting beautifulsoup4 (from -r /tmp/build/requirements.txt (line 12))  Downloading beautifulsoup4-4.6.0-py2-none-any.whl (86kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)  Collecting idna<2.7,>=2.5 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading idna-2.6-py2.py3-none-any.whl (56kB)  Collecting urllib3<1.23,>=1.21.1 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading urllib3-1.22-py2.py3-none-any.whl (132kB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading certifi-2018.1.18-py2.py3-none-any.whl (151kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)  Installing collected packages: dumptruck, idna, urllib3, certifi, chardet, requests, scraperwiki, lxml, cssselect, beautifulsoup4  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for cssselect: started  Running setup.py install for cssselect: finished with status 'done'  Successfully installed beautifulsoup4-4.6.0 certifi-2018.1.18 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 idna-2.6 lxml-3.4.4 requests-2.18.4 scraperwiki urllib3-1.22   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Traceback (most recent call last): File "scraper.py", line 88, in <module> prepare_counties() File "scraper.py", line 13, in prepare_counties get_counties(url, state); File "scraper.py", line 29, in get_counties get_county_data(state, county_name, county_data_url) File "scraper.py", line 45, in get_county_data if (rows.select('th')[0].get_text().strip() == 'Website:'): IndexError: list index out of range

Data

Downloaded 78 times by philipashlock MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (7.45 MB) Use the API

rows 10 / 3072

url cite_source address_locality address_postcode phone name address_region address_2 address_1
Petersburg, AK
99833-0329
(907) 772-4519
Petersburg Census Area
AK
PO BOX 329
Broomfield, CO
80020-2495
(303)438-6300
Broomfield County
CO
1 Descombes Dr
Denver, CO
80202-5337
(720)913-4900
Denver County
CO
City And County Building
1437 Bannock St
New York, NY
10007-1602
(212)788-3000
New York County
NY
1 Centre St Bldg Municipal
Prattville, AL
36067
334358-6700
Autauga County
AL
County Courthouse
135 N. Court Street, Suite B
Bay Minette, AL
36507-4809
(251)937-0264
Baldwin County
AL
County Administration Building
322 Courthouse Sq
Clayton, AL
36016-0398
(334)775-3203
Barbour County
AL
County Courthouse
PO BOX 398
Centreville, AL
35042-2277
(205)926-3114
Bibb County
AL
County Courthouse
157 Sw Davidson Dr
Oneonta, AL
35121-1716
205-625-4160
Blount County
AL
County Courthouse
220 2Nd Ave E
Union Springs, AL
36089-0472
(334)738-3883
Bullock County
AL
County Courthouse
PO BOX 472

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (7.45 MB) Use the API

rows 10 / 42966

government_name cite_source state name position
Autauga County
AL
James Johnson
County Sheriff
Autauga County
AL
Joan Smith
Revenue Commissioner
Cleburne County
AL
Joe Jacks
County Sheriff
Cleburne County
AL
Jerry Owen
Circuit Clerk
Colbert County
AL
Ronnie May
County Sheriff
Conecuh County
AL
Edwin Booker
Sheriff
Covington County
AL
Janice Hart
Revenue Commissioner
Geneva County
AL
Greg Ward
Sheriff
Hale County
AL
Charlesy Tubbs
County Tax Collector
Hale County
AL
Betty Pate
County Circuit Clerk

Statistics

Average successful run time: about 1 hour

Total run time: 25 days

Total cpu time used: about 9 hours

Total disk space used: 7.47 MB

History

  • Auto ran revision 8b819bbb and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 8b819bbb and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 8b819bbb and failed .
    nothing changed in the database
  • Auto ran revision 8b819bbb and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 8b819bbb and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

DemocracyMap_US_Counties / scraper.py