wdiv-scrapers / DC-PollingStations-Barnet

Polling station and district data from Barnet Borough Council

Scrapes open.barnet.gov.uk

Barnet Open Data


Scrape polling station and district data from Barnet Borough Council

Contributors chris48s

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 3 is python-3.6.2 (you are using python-3.4.3, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-3.6.2).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-3.4.3 -----> Installing pip -----> Installing requirements with pip  Collecting git+https://github.com/wdiv-scrapers/dc-base-scrapers.git (from -r /tmp/build/requirements.txt (line 1))  Cloning https://github.com/wdiv-scrapers/dc-base-scrapers.git to /tmp/pip-req-build-6oult44q  Collecting lxml==3.6.4 (from dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/3a/2d/953dfc8496cc9d5add66c0fd468d786d4e4706d263285bc4bb1785485149/lxml-3.6.4-cp34-cp34m-manylinux1_x86_64.whl (4.2MB)  Collecting retry==0.9.2 (from dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/4b/0d/53aea75710af4528a25ed6837d71d117602b01946b307a3912cb3cfcbcba/retry-0.9.2-py2.py3-none-any.whl  Collecting scraperwiki==0.5.1 (from dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/30/84/d874847baad89f03e6984fcd87505a37bf924b66519d1e07bf76e2369af0/scraperwiki-0.5.1.tar.gz  Collecting arcgis2geojson==1.2.0 (from dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/83/47/a065e6610a997096eb2db3b052dcfa094b8e149dda5f9b3b6d94358275c2/arcgis2geojson-1.2.0.tar.gz  Collecting commitment==2.0.0 (from dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/5b/f1/4422969e6c438568c486d727876addbc96abe0958ba0c54cad414b466209/commitment-2.0.0.tar.gz  Collecting decorator>=3.4.2 (from retry==0.9.2->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/bc/bb/a24838832ba35baf52f32ab1a49b906b5f82fb7c76b2f6a7e35e140bac30/decorator-4.3.0-py2.py3-none-any.whl  Collecting py<2.0.0,>=1.4.26 (from retry==0.9.2->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/67/a5/f77982214dd4c8fd104b066f249adea2c49e25e8703d284382eb5e9ab35a/py-1.5.3-py2.py3-none-any.whl (84kB)  Collecting requests (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)  Collecting six (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl  Collecting sqlalchemy (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/c1/c8/392fcd2d01534bc871c65cb964e0b39d59feb777e51649e6eaf00f6377b5/SQLAlchemy-1.2.7.tar.gz (5.6MB)  Collecting alembic (from scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/89/03/756d5b8e1c90bf283c3f435766aa3f20208d1c3887579dd8f2122e01d5f4/alembic-0.9.9.tar.gz (1.0MB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Collecting idna<2.7,>=2.5 (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)  Collecting urllib3<1.23,>=1.21.1 (from requests->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)  Collecting Mako (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz (564kB)  Collecting python-editor>=0.3 (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/65/1e/adf6e000ea5dc909aa420352d6ba37f16434c8a3c2fa030445411a1ed545/python-editor-1.0.3.tar.gz  Collecting python-dateutil (from alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB)  Collecting MarkupSafe>=0.9.2 (from Mako->alembic->scraperwiki==0.5.1->dc-base-scrapers==0.0.0->-r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz  Installing collected packages: lxml, decorator, py, retry, certifi, chardet, idna, urllib3, requests, six, sqlalchemy, MarkupSafe, Mako, python-editor, python-dateutil, alembic, scraperwiki, arcgis2geojson, commitment, dc-base-scrapers  Running setup.py install for sqlalchemy: started  Running setup.py install for sqlalchemy: finished with status 'done'  Running setup.py install for MarkupSafe: started  Running setup.py install for MarkupSafe: finished with status 'done'  Running setup.py install for Mako: started  Running setup.py install for Mako: finished with status 'done'  Running setup.py install for python-editor: started  Running setup.py install for python-editor: finished with status 'done'  Running setup.py install for alembic: started  Running setup.py install for alembic: finished with status 'done'  Running setup.py install for scraperwiki: started  Running setup.py install for scraperwiki: finished with status 'done'  Running setup.py install for arcgis2geojson: started  Running setup.py install for arcgis2geojson: finished with status 'done'  Running setup.py install for commitment: started  Running setup.py install for commitment: finished with status 'done'  Running setup.py install for dc-base-scrapers: started  Running setup.py install for dc-base-scrapers: finished with status 'done'  Successfully installed Mako-1.0.7 MarkupSafe-1.0 alembic-0.9.9 arcgis2geojson-1.2.0 certifi-2018.4.16 chardet-3.0.4 commitment-2.0.0 dc-base-scrapers-0.0.0 decorator-4.3.0 idna-2.6 lxml-3.6.4 py-1.5.3 python-dateutil-2.7.3 python-editor-1.0.3 requests-2.18.4 retry-0.9.2 scraperwiki-0.5.1 six-1.11.0 sqlalchemy-1.2.7 urllib3-1.22   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... found 1 polling-stations-in-barnet resources

Data

Downloaded 13 times by wdiv-scrapers chris48s

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (72 KB) Use the API

rows 10 / 299

dataset created revision_id url format
polling-stations-in-barnet
2016-05-26T13:11:06.153768
0ac57e30-fc72-4021-912a-a91fad2e1b5a
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
CSV
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv
polling-stations-in-barnet
2016-05-26T13:11:06.581Z
https://files.datapress.com/barnet/dataset/polling-stations-in-barnet/2016-05-26T13:10:59/Polling Stations 2016.csv
csv

Statistics

Average successful run time: half a minute

Total run time: about 4 hours

Total cpu time used: 6 minutes

Total disk space used: 94.6 KB

History

  • Auto ran revision 0ec16e8d and completed successfully .
    1 record added in the database
    2 pages scraped
  • Auto ran revision 0ec16e8d and completed successfully .
    1 record added in the database
    17 pages scraped
  • Auto ran revision 7c1eaaf0 and completed successfully .
    1 record added in the database
    2 pages scraped
  • Auto ran revision 7c1eaaf0 and completed successfully .
    1 record added in the database
    64 pages scraped
  • Auto ran revision 7c1eaaf0 and completed successfully .
    1 record added in the database
    3 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

DC-PollingStations-Barnet / scraper.py