everypolitician-scrapers / gibraltar-parliament

README.md

This is a scraper that runs on Morph. To get started see the documentation

Contributors andylolz chrismytton

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 2))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting beautifulsoup4==4.4.0 (from -r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/9d/c8/cd70aabb46af8f30ed83c15287c3d8b1455ba7ee923b03870ee0cdb6ec4f/beautifulsoup4-4.4.0-py2-none-any.whl (81kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl (157kB)  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl (150kB)  Collecting idna<2.9,>=2.5 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Building wheels for collected packages: dumptruck  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Stored in directory: /tmp/pip-ephem-wheel-cache-2MgwB1/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e  Successfully built dumptruck  Installing collected packages: dumptruck, certifi, urllib3, idna, chardet, requests, scraperwiki, beautifulsoup4  Running setup.py develop for scraperwiki  Successfully installed beautifulsoup4-4.4.0 certifi-2019.6.16 chardet-3.0.4 dumptruck-0.1.6 idna-2.8 requests-2.22.0 scraperwiki urllib3-1.25.3 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... J Bossano (term 3) is probably the same person as J J Bossano (term 2) J Bossano (term 7) is probably the same person as J J Bossano (term 6) R Rhoda (term 8) is probably the same person as R R Rhoda (term 10) J J Gabay (term 8) is probably the same person as J Gabay (term 8) F J Vinet (term 11) is probably the same person as F Vinet (term 10) C G Beltran (term 11) is probably the same person as C Beltran (term 10)

Data

Downloaded 2211 times by everypolitician

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 KB) Use the API

rows 10 / 13

name end_date start_date id
First House of Assembly
1972
1969
1
Second House of Assembly
1976
1972
2
Third House of Assembly
1980
1976
3
Fourth House of Assembly
1984
1980
4
Fifth House of Assembly
1988
1984
5
Sixth House of Assembly
1992
1988
6
Seventh House of Assembly
1996
1992
7
Eighth House of Assembly
2000
1996
8
Ninth House of Assembly
2003-10-24
2000-02-23
9
Tenth House of Assembly
2007-09-07
2003-12-18
10

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 KB) Use the API

rows 10 / 225

term side role start_date end_date honorific_suffix id honorific_prefix name
1
Government
Chief Minister
1
Major
R J Peliza
1
Government
Minister for Labour and Social Security
2
M Xiberras
1
Government
Minister for Information, Port Trade & Industry
3
Major
A J Gache
1
Government
Minister for Medical and Health Services
4
J Caruana
1
Government
Minister for Tourism and Municipal Services
5
W M Isola
1
Government
Minister for Public Works and Housing
6
Miss
C Anes
1
Government
Minister for Education and Recreation
7
L Devincenzi
1
Government
Attorney General
CBE QC
8
C B O’Beirne
1
Government
Financial & Development Secretary
CMG OBE
9
E H Davies
1
Government
OBE
10
P J Isola

Statistics

Average successful run time: less than a minute

Total run time: about 7 hours

Total cpu time used: 8 minutes

Total disk space used: 58 KB

History

  • Auto ran revision a3fb342d and completed successfully .
    238 records added, 238 records removed in the database
  • Auto ran revision a3fb342d and completed successfully .
    238 records added, 238 records removed in the database
  • Auto ran revision a3fb342d and completed successfully .
    238 records added, 238 records removed in the database
  • Auto ran revision a3fb342d and completed successfully .
    238 records added, 238 records removed in the database
  • Auto ran revision a3fb342d and completed successfully .
    238 records added, 238 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

gibraltar-parliament / scraper.py