jamezpolley / parlinfo_people

Parliamentarian data from Parlinfo


This is a scraper that runs on Morph. To get started see the documentation

Contributors jamezpolley

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.  Collecting scraperwiki  Downloading scraperwiki-0.5.1.tar.gz (7.7 kB)  Collecting requests  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)  Collecting six  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)  Collecting sqlalchemy  Downloading SQLAlchemy-1.4.36-cp27-cp27m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.6 MB)  Collecting alembic  Downloading alembic-1.6.5-py2.py3-none-any.whl (164 kB)  Collecting idna<3,>=2.5; python_version < "3"  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)  Collecting chardet<5,>=3.0.2; python_version < "3"  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)  Collecting urllib3<1.27,>=1.21.1  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)  Collecting importlib-metadata; python_version < "3.8"  Downloading importlib_metadata-2.1.3-py2.py3-none-any.whl (10 kB)  Collecting python-editor>=0.3  Downloading python_editor-1.0.4-py2-none-any.whl (4.9 kB)  Collecting Mako  Downloading Mako-1.1.6-py2.py3-none-any.whl (75 kB)  Collecting python-dateutil  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)  Collecting contextlib2; python_version < "3"  Downloading contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)  Collecting pathlib2; python_version < "3"  Downloading pathlib2-2.3.7.post1-py2.py3-none-any.whl (18 kB)  Collecting configparser>=3.5; python_version < "3"  Downloading configparser-4.0.2-py2.py3-none-any.whl (22 kB)  Collecting zipp>=0.5  Downloading zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)  Collecting MarkupSafe>=0.9.2  Downloading MarkupSafe-1.1.1-cp27-cp27m-manylinux1_x86_64.whl (24 kB)  Collecting scandir; python_version < "3.5"  Downloading scandir-1.10.0.tar.gz (33 kB)  Collecting typing; python_version < "3.5"  Downloading typing-3.10.0.0-py2-none-any.whl (26 kB)  Building wheels for collected packages: scraperwiki, scandir  Building wheel for scraperwiki (setup.py): started  Building wheel for scraperwiki (setup.py): finished with status 'done'  Created wheel for scraperwiki: filename=scraperwiki-0.5.1-py2-none-any.whl size=6544 sha256=a357ef1f056e2e99b3612312d75f118eb37373fa8f8ebef6ad12e7c8fa15f766  Stored in directory: /tmp/pip-ephem-wheel-cache-ZXq3A9/wheels/b6/f7/aa/dee52fcb718a083cd3cf348dfff25a8945f5e7aed0a27de6c4  Building wheel for scandir (setup.py): started  Building wheel for scandir (setup.py): finished with status 'done'  Created wheel for scandir: filename=scandir-1.10.0-cp27-cp27m-linux_x86_64.whl size=39293 sha256=54d0d6c6cf4ee062450b101f496aa28e91c7000644300f0cf58336cb4a499e94  Stored in directory: /tmp/pip-ephem-wheel-cache-ZXq3A9/wheels/58/2c/26/52406f7d1f19bcc47a6fbd1037a5f293492f5cf1d58c539edb  Successfully built scraperwiki scandir  Installing collected packages: idna, certifi, chardet, urllib3, requests, six, contextlib2, scandir, typing, pathlib2, configparser, zipp, importlib-metadata, sqlalchemy, python-editor, MarkupSafe, Mako, python-dateutil, alembic, scraperwiki  Successfully installed Mako-1.1.6 MarkupSafe-1.1.1 alembic-1.6.5 certifi-2021.10.8 chardet-4.0.0 configparser-4.0.2 contextlib2-0.6.0.post1 idna-2.10 importlib-metadata-2.1.3 pathlib2-2.3.7.post1 python-dateutil-2.8.2 python-editor-1.0.4 requests-2.27.1 scandir-1.10.0 scraperwiki-0.5.1 six-1.16.0 sqlalchemy-1.4.36 typing-3.10.0.0 urllib3-1.26.9 zipp-1.2.0 DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.    -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... ['person count', 'aph id', 'name', 'birthday', 'alt name'] ['person count', 'aph id', 'name'] Traceback (most recent call last): File "scraper.py", line 54, in <module> read_people_csv() File "scraper.py", line 49, in read_people_csv person["oa_id"], person["aph_id"], person["name"] = row[0:3] NameError: global name 'person' is not defined

Data

Downloaded 1 time by edmiglobal

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3.27 MB) Use the API

rows 10 / 7072

ShowEmail State Gender MPID Representing TwitterAddress ShowContactForm Chamber ImageUrl Party EmailAddress FullName FormerMember date_scraped SurnameLetter Titles FacebookAddress PersonalWebsite DateElected RepresentingTitle
false
NSW
MALE
EZ5
Warringah, New South Wales
false
House of Representatives
Liberal Party of Australia
Tony.Abbott.MP@aph.gov.au
Hon Tony Abbott MP
false
2019-05-27
A
1995-03-25
Member
false
TAS
MALE
N26
Tasmania
true
Senate
Liberal Party of Australia
senator.abetz@aph.gov.au
Senator the Hon Eric Abetz
false
2019-05-27
A
Chair of Foreign Affairs, Defence and Trade Legislation Committee; Deputy Chair of Foreign Affairs, Defence and Trade References Committee
1994-02-21
Senator
true
NSW
MALE
R36
Grayndler, New South Wales
false
House of Representatives
Australian Labor Party
A.Albanese.MP@aph.gov.au
Hon Anthony Albanese MP
false
2019-05-27
A
1996-03-01
Member
false
NSW
MALE
M3M
Bennelong, New South Wales
true
House of Representatives
Liberal Party of Australia
John.Alexander.MP@aph.gov.au
Mr John Alexander OAM, MP
false
2019-05-27
A
2017-12-20
Member
false
WA
FEMALE
13050
Cowan, Western Australia
false
House of Representatives
Australian Labor Party
Anne.Aly.MP@aph.gov.au
Dr Anne Aly MP
false
2019-05-27
A
2016-07-28
Member
true
QLD
FEMALE
230886
McPherson, Queensland
false
House of Representatives
Liberal Party of Australia
Karen.Andrews.MP@aph.gov.au
Hon Karen Andrews MP
false
2019-05-27
A
Minister for Industry; Science and Technology
2010-08-20
Member
true
VIC
MALE
HK5
Menzies, Victoria
false
House of Representatives
Liberal Party of Australia
menzies@aph.gov.au
Hon Kevin Andrews MP
false
2019-05-27
A
Chair of Joint Standing Committee on the National Disability Insurance Scheme
1991-05-10
Member
true
QLD
MALE
273829
Queensland
false
Senate
Independent
senator.anning@aph.gov.au
Senator Fraser Anning
false
2019-05-27
A
2017-11-09
Senator
false
TAS
FEMALE
281558
Tasmania
true
Senate
Liberal Party of Australia
senator.askew@aph.gov.au
Senator Wendy Askew
false
2019-05-27
A
2019-03-05
Senator
true
VIC
MALE
M3C
Melbourne, Victoria
false
House of Representatives
Australian Greens
Adam.Bandt.MP@aph.gov.au
Mr Adam Bandt MP
false
2019-05-27
B
2010-08-20
Member

Statistics

Average successful run time: 1 minute

Total run time: about 19 hours

Total cpu time used: 11 minutes

Total disk space used: 3.63 MB

History

  • Auto ran revision e5b90ef9 and failed .
    nothing changed in the database
  • Auto ran revision e5b90ef9 and failed .
    nothing changed in the database
  • Auto ran revision e5b90ef9 and failed .
    nothing changed in the database
  • Auto ran revision e5b90ef9 and failed .
    nothing changed in the database
  • Auto ran revision e5b90ef9 and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

parlinfo_people / scraper.py