This is a scraper that runs on Morph. To get started see the documentation

Contributors duncanparkes

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 8))  Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1 (from -r /tmp/build/requirements.txt (line 9))  Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz  Collecting PyExecJS==1.1.0 (from -r /tmp/build/requirements.txt (line 10))  Downloading https://files.pythonhosted.org/packages/0d/6b/5183948f2a12b7df0d597273f941a26baa35520505ec8d670f4c9b568749/PyExecJS-1.1.0.tar.gz  Collecting awesome-slugify==1.6.4 (from -r /tmp/build/requirements.txt (line 11))  Downloading https://files.pythonhosted.org/packages/2f/2a/adc6ce1178b938286c9ecc81b99c60dcac3e66c1b3f0fa72255af03a5cf6/awesome-slugify-1.6.4.tar.gz  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)  Collecting regex (from awesome-slugify==1.6.4->-r /tmp/build/requirements.txt (line 11))  Downloading https://files.pythonhosted.org/packages/9f/79/adbe79373ec3c56ae61670bbed30609fe4a9b6af835478fa12bf8795cdec/regex-2018.07.11.tar.gz (628kB)  Collecting Unidecode<0.05,>=0.04.14 (from awesome-slugify==1.6.4->-r /tmp/build/requirements.txt (line 11))  Downloading https://files.pythonhosted.org/packages/01/a1/9d7f3138ee3d79a1ab865a2cb38200ca778d85121db19fe264c76c981184/Unidecode-0.04.21-py2.py3-none-any.whl (228kB)  Collecting idna<2.8,>=2.5 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)  Collecting urllib3<1.24,>=1.21.1 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Installing collected packages: dumptruck, idna, certifi, urllib3, chardet, requests, scraperwiki, lxml, cssselect, PyExecJS, regex, Unidecode, awesome-slugify  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for cssselect: started  Running setup.py install for cssselect: finished with status 'done'  Running setup.py install for PyExecJS: started  Running setup.py install for PyExecJS: finished with status 'done'  Running setup.py install for regex: started  Running setup.py install for regex: finished with status 'done'  Running setup.py install for awesome-slugify: started  Running setup.py install for awesome-slugify: finished with status 'done'  Successfully installed PyExecJS-1.1.0 Unidecode-0.4.21 awesome-slugify-1.6.4 certifi-2018.4.16 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 idna-2.7 lxml-3.4.4 regex-2018.7.11 requests-2.19.1 scraperwiki urllib3-1.23   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 100 times by everypolitician duncanparkes tmtmtmtm

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (33 KB) Use the API

rows 10 / 88

term_id image area name details_url party id email
2013
Jean-Charles ALLAVENA
HM Horizon Monaco
Jean-Charles-ALLAVENA
jc.allavena@conseil-national.mc
2013
Christian BARILARO
HM Horizon Monaco
Christian-BARILARO
c.barilaro@conseil-national.mc
2013
Claude BOISSON
HM Horizon Monaco
Claude-BOISSON
c.boisson@conseil-national.mc
2013
Marc BURINI
HM Horizon Monaco
Marc-BURINI
m.burini@conseil-national.mc
2013
Philippe CLERISSI
HM Horizon Monaco
Philippe-CLERISSI
p.clerissi@conseil-national.mc
2013
Thierry CROVETTO
HM Horizon Monaco
Thierry-CROVETTO
t.crovetto@conseil-national.mc
2013
Jean-Michel CUCCHI
HM Horizon Monaco
Jean-Michel-CUCCHI
jm.cucchi@conseil-national.mc
2013
Alain FICINI
HM Horizon Monaco
Alain-FICINI
a.ficini@conseil-national.mc
2013
Sophie LAVAGNA
HM Horizon Monaco
Sophie-LAVAGNA
s.lavagna@conseil-national.mc
2013
Laurent NOUVION
HM Horizon Monaco
Laurent-NOUVION
lnp@conseil-national.mc

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (33 KB) Use the API

rows 3 / 3

start_date name id end_date
2013-02-21
2013
2013
2018
2008
2008-2013
2008
2013
2003
2003-2008
2003
2008

Statistics

Average successful run time: 3 minutes

Total run time: about 1 month

Total cpu time used: 31 minutes

Total disk space used: 65.7 KB

History

  • Auto ran revision 915510bf and completed successfully .
    72 records added, 72 records removed in the database
  • Auto ran revision 915510bf and completed successfully .
    72 records added, 72 records removed in the database
    27 pages scraped
  • Auto ran revision 915510bf and completed successfully .
    72 records added, 72 records removed in the database
    27 pages scraped
  • Auto ran revision 915510bf and completed successfully .
    72 records added, 72 records removed in the database
  • Auto ran revision 915510bf and completed successfully .
    72 records added, 72 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

monaco / scraper.py