This is a scraper that runs on Morph. To get started see the documentation

Contributors duncanparkes

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting lxml==3.4.4  Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1  Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz  Collecting PyExecJS==1.1.0  Downloading https://files.pythonhosted.org/packages/0d/6b/5183948f2a12b7df0d597273f941a26baa35520505ec8d670f4c9b568749/PyExecJS-1.1.0.tar.gz  Collecting awesome-slugify==1.6.4  Downloading https://files.pythonhosted.org/packages/2f/2a/adc6ce1178b938286c9ecc81b99c60dcac3e66c1b3f0fa72255af03a5cf6/awesome-slugify-1.6.4.tar.gz  Collecting dumptruck>=0.1.2  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Collecting requests  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)  Collecting regex  Downloading https://files.pythonhosted.org/packages/fc/1d/13cc7d174cd2d05808abac3f5fb37433e30c4cd93b152d2a9c09c926d7e8/regex-2019.11.1.tar.gz (669kB)  Collecting Unidecode<0.05,>=0.04.14  Downloading https://files.pythonhosted.org/packages/01/a1/9d7f3138ee3d79a1ab865a2cb38200ca778d85121db19fe264c76c981184/Unidecode-0.04.21-py2.py3-none-any.whl (228kB)  Collecting certifi>=2017.4.17  Downloading https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl (154kB)  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1  Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB)  Collecting idna<2.9,>=2.5  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)  Collecting chardet<3.1.0,>=3.0.2  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Building wheels for collected packages: lxml, cssselect, PyExecJS, awesome-slugify, dumptruck, regex  Building wheel for lxml (setup.py): started  Building wheel for lxml (setup.py): still running...  Building wheel for lxml (setup.py): finished with status 'done'  Created wheel for lxml: filename=lxml-3.4.4-cp27-cp27m-linux_x86_64.whl size=2989840 sha256=26d9d8ddf83db3fb174c65e7449bc840f67bb25db9e414b468d592b07de0a472  Stored in directory: /tmp/pip-ephem-wheel-cache-DxsaUG/wheels/f6/df/7b/af9cace9baf95a6e4a2b5790e30da55fc780ddee598314d1ed  Building wheel for cssselect (setup.py): started  Building wheel for cssselect (setup.py): finished with status 'done'  Created wheel for cssselect: filename=cssselect-0.9.1-cp27-none-any.whl size=26994 sha256=62de20045a3ec180dba58e6dada89ce5ac6fad7d8d531dbaa91e638faa8a00ed  Stored in directory: /tmp/pip-ephem-wheel-cache-DxsaUG/wheels/45/25/d7/5a3b06d22b1ffb616f868a74729a5a002bcc04d45109b4f223  Building wheel for PyExecJS (setup.py): started  Building wheel for PyExecJS (setup.py): finished with status 'done'  Created wheel for PyExecJS: filename=PyExecJS-1.1.0-cp27-none-any.whl size=9431 sha256=974b38edde6ff48a248dedc133580984a884650aad227f5c678ce916d58fea58  Stored in directory: /tmp/pip-ephem-wheel-cache-DxsaUG/wheels/9e/e4/21/d78fd3ddb7cee84c8d37c9dc4c57837f08b012cd0045cf36f7  Building wheel for awesome-slugify (setup.py): started  Building wheel for awesome-slugify (setup.py): finished with status 'done'  Created wheel for awesome-slugify: filename=awesome_slugify-1.6.4-cp27-none-any.whl size=8334 sha256=3cbe6f3a2fa9ecb9ad9763a5a31ffe417c2278f5b357f0b0666f2a1f2dcd76d8  Stored in directory: /tmp/pip-ephem-wheel-cache-DxsaUG/wheels/71/bf/7f/2a1535926ab6419a15cd28eee5f3f6107c88fd2fe2c99dfa08  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Created wheel for dumptruck: filename=dumptruck-0.1.6-cp27-none-any.whl size=11845 sha256=6e0da0c816058ac2558c983c7cb46c553be8c5bbf99d703f8aa52cc6cfa33870  Stored in directory: /tmp/pip-ephem-wheel-cache-DxsaUG/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e  Building wheel for regex (setup.py): started  Building wheel for regex (setup.py): finished with status 'done'  Created wheel for regex: filename=regex-2019.11.1-cp27-cp27m-linux_x86_64.whl size=627208 sha256=e6fb12276da7b4e8938c533b390796e126374b38c5ae08d56cc84db9bc401c22  Stored in directory: /tmp/pip-ephem-wheel-cache-DxsaUG/wheels/5c/c6/c1/0bc8d16ea38c44536a82dd1bec665996e5af37489fa88826b6  Successfully built lxml cssselect PyExecJS awesome-slugify dumptruck regex  Installing collected packages: dumptruck, certifi, urllib3, idna, chardet, requests, scraperwiki, lxml, cssselect, PyExecJS, regex, Unidecode, awesome-slugify  Running setup.py develop for scraperwiki  Successfully installed PyExecJS-1.1.0 Unidecode-0.4.21 awesome-slugify-1.6.4 certifi-2019.9.11 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 idna-2.8 lxml-3.4.4 regex-2019.11.1 requests-2.22.0 scraperwiki urllib3-1.25.7 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support    -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Traceback (most recent call last): File "scraper.py", line 40, in <module> mailto_script = member_root.xpath("//h4[contains(., 'Contact Mail')]")[0].getnext().getchildren()[0].text_content() IndexError: list index out of range

Data

Downloaded 100 times by everypolitician duncanparkes tmtmtmtm

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (33 KB) Use the API

rows 10 / 88

term_id image area name details_url party id email
2013
Jean-Charles ALLAVENA
HM Horizon Monaco
Jean-Charles-ALLAVENA
jc.allavena@conseil-national.mc
2013
Christian BARILARO
HM Horizon Monaco
Christian-BARILARO
c.barilaro@conseil-national.mc
2013
Claude BOISSON
HM Horizon Monaco
Claude-BOISSON
c.boisson@conseil-national.mc
2013
Marc BURINI
HM Horizon Monaco
Marc-BURINI
m.burini@conseil-national.mc
2013
Philippe CLERISSI
HM Horizon Monaco
Philippe-CLERISSI
p.clerissi@conseil-national.mc
2013
Thierry CROVETTO
HM Horizon Monaco
Thierry-CROVETTO
t.crovetto@conseil-national.mc
2013
Jean-Michel CUCCHI
HM Horizon Monaco
Jean-Michel-CUCCHI
jm.cucchi@conseil-national.mc
2013
Alain FICINI
HM Horizon Monaco
Alain-FICINI
a.ficini@conseil-national.mc
2013
Sophie LAVAGNA
HM Horizon Monaco
Sophie-LAVAGNA
s.lavagna@conseil-national.mc
2013
Laurent NOUVION
HM Horizon Monaco
Laurent-NOUVION
lnp@conseil-national.mc

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (33 KB) Use the API

rows 3 / 3

start_date name id end_date
2013-02-21
2013
2013
2018
2008
2008-2013
2008
2013
2003
2003-2008
2003
2008

Statistics

Average successful run time: 3 minutes

Total run time: about 1 month

Total cpu time used: about 1 hour

Total disk space used: 65.7 KB

History

  • Auto ran revision 915510bf and failed .
    nothing changed in the database
  • Auto ran revision 915510bf and failed .
    nothing changed in the database
  • Auto ran revision 915510bf and failed .
    nothing changed in the database
  • Auto ran revision 915510bf and failed .
    nothing changed in the database
  • Auto ran revision 915510bf and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

monaco / scraper.py