TMMV / Parquimetros

Scrapes www.cm-porto.pt

O sítio oficial da Câmara Municipal do Porto pretende ser um canal de comunicação com o cidadão em geral e com os munícipes portuenses em particular. Disponibiliza informações genéricas sobre a autarquia portuense,a história, os diversos equipamentos culturais e desportivos, o turismo e a actividade da cidade. Permite ainda, de forma interactiva, facilitar o contacto com os cidadãos que tenham que recorrer aos diversos serviços municipais


A scrapper running on morph.io saving the streets with parking meters in Porto. The idea is to have an idea of how they have spread across the city in a couple of months.

Contributors TMMV

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.10, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.10 -----> Installing pip -----> Installing requirements with pip  Collecting beautifulsoup4==4.5.1 (from -r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/dc/27/8919b7a505b00f410c1c113c6c93b3253784375797976272e4198f4e455a/beautifulsoup4-4.5.1-py2-none-any.whl (83kB)  Collecting ftfy==4.2.0 (from -r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/ba/6f/f82c1b6fdd0e06ef6c456409aa87064cf9253c0159761500cc97e11b89c7/ftfy-4.2.0.tar.gz  Collecting requests==2.11.1 (from -r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/ea/03/92d3278bf8287c5caa07dbd9ea139027d5a3592b0f4d14abf072f890fab2/requests-2.11.1-py2.py3-none-any.whl (514kB)  Collecting wheel==0.26. (from -r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/d9/6f/cfb45d489703e4843b30c28e01ab39c87b0648bfa11a6c4a959f426fe217/wheel-0.26.0-py2.py3-none-any.whl (63kB)  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 5))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 5))  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Installing collected packages: beautifulsoup4, ftfy, requests, wheel, dumptruck, scraperwiki  Running setup.py install for ftfy: started  Running setup.py install for ftfy: finished with status 'done'  Found existing installation: wheel 0.31.1  Uninstalling wheel-0.31.1:  Successfully uninstalled wheel-0.31.1  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Successfully installed beautifulsoup4-4.5.1 dumptruck-0.1.6 ftfy-4.2.0 requests-2.11.1 scraperwiki wheel-0.26.0   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 14 times by TMMV

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (9.06 MB) Use the API

rows 10 / 69352

longitude latitude street date
-8.600393
41.146406
Rua Visconde de Bóbeda
01-10-2016
-8.615738
41.152722
Rua dos Bragas
01-10-2016
-8.604749
41.153478
Rua Santa Catarina
01-10-2016
-8.638357
41.162341
Rua de João de Deus
01-10-2016
-8.604775
41.145956
Rua de Campinho
01-10-2016
-8.628688
41.153318
Rua do Bom Sucesso
01-10-2016
-8.613438
41.14144
Rua de São João
01-10-2016
-8.608222
41.153516
Rua Gonçalo Cristóvão
01-10-2016
-8.620291
41.146615
Rua Alberto Aires de Gouveia
01-10-2016
-8.614138
41.145971
Rua de S. Filipe de Nery
01-10-2016

Statistics

Average successful run time: less than a minute

Total run time: about 6 hours

Total cpu time used: 7 minutes

Total disk space used: 9.1 MB

History

  • Auto ran revision 9bdba6c0 and completed successfully .
    166 records added in the database
    1 page scraped
  • Auto ran revision 9bdba6c0 and completed successfully .
    166 records added in the database
    1 page scraped
  • Auto ran revision 9bdba6c0 and completed successfully .
    166 records added in the database
    1 page scraped
  • Auto ran revision 9bdba6c0 and completed successfully .
    166 records added in the database
    1 page scraped
  • Auto ran revision 9bdba6c0 and completed successfully .
    166 records added in the database
    1 page scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

Parquimetros / scraper.py