duncanparkes / namibia

Scrape the members of the Namibian parliament.


This is a scraper that runs on Morph. To get started see the documentation

Contributors duncanparkes

The scraper is running. It was queued automatically .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting lxml==3.4.4  Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1  Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz  Collecting PyExecJS==1.1.0  Downloading https://files.pythonhosted.org/packages/0d/6b/5183948f2a12b7df0d597273f941a26baa35520505ec8d670f4c9b568749/PyExecJS-1.1.0.tar.gz  Collecting awesome-slugify==1.6.4  Downloading https://files.pythonhosted.org/packages/2f/2a/adc6ce1178b938286c9ecc81b99c60dcac3e66c1b3f0fa72255af03a5cf6/awesome-slugify-1.6.4.tar.gz  Collecting dumptruck>=0.1.2  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Collecting requests  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)  Collecting regex  Downloading https://files.pythonhosted.org/packages/fc/1d/13cc7d174cd2d05808abac3f5fb37433e30c4cd93b152d2a9c09c926d7e8/regex-2019.11.1.tar.gz (669kB)  Collecting Unidecode<0.05,>=0.04.14  Downloading https://files.pythonhosted.org/packages/01/a1/9d7f3138ee3d79a1ab865a2cb38200ca778d85121db19fe264c76c981184/Unidecode-0.04.21-py2.py3-none-any.whl (228kB)  Collecting certifi>=2017.4.17  Downloading https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl (154kB)  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1  Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB)  Collecting idna<2.9,>=2.5  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)  Collecting chardet<3.1.0,>=3.0.2  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Building wheels for collected packages: lxml, cssselect, PyExecJS, awesome-slugify, dumptruck, regex  Building wheel for lxml (setup.py): started  Building wheel for lxml (setup.py): still running...  Building wheel for lxml (setup.py): finished with status 'done'  Created wheel for lxml: filename=lxml-3.4.4-cp27-cp27m-linux_x86_64.whl size=2989915 sha256=157d403630a57e228d5a661d2a97b9af42d2d9317e1efc673b3ccdb5f9645d5d  Stored in directory: /tmp/pip-ephem-wheel-cache-1P6O4z/wheels/f6/df/7b/af9cace9baf95a6e4a2b5790e30da55fc780ddee598314d1ed  Building wheel for cssselect (setup.py): started  Building wheel for cssselect (setup.py): finished with status 'done'  Created wheel for cssselect: filename=cssselect-0.9.1-cp27-none-any.whl size=26994 sha256=1e13bd418550e708fd0a11f66498441db4d667ce54942c5a2653274901577fc7  Stored in directory: /tmp/pip-ephem-wheel-cache-1P6O4z/wheels/45/25/d7/5a3b06d22b1ffb616f868a74729a5a002bcc04d45109b4f223  Building wheel for PyExecJS (setup.py): started  Building wheel for PyExecJS (setup.py): finished with status 'done'  Created wheel for PyExecJS: filename=PyExecJS-1.1.0-cp27-none-any.whl size=9431 sha256=853e0637398d7dd16c4db384ceb595f5cd3bb917c1676df01116b77c88c3f99a  Stored in directory: /tmp/pip-ephem-wheel-cache-1P6O4z/wheels/9e/e4/21/d78fd3ddb7cee84c8d37c9dc4c57837f08b012cd0045cf36f7  Building wheel for awesome-slugify (setup.py): started  Building wheel for awesome-slugify (setup.py): finished with status 'done'  Created wheel for awesome-slugify: filename=awesome_slugify-1.6.4-cp27-none-any.whl size=8334 sha256=d2a67c9d5aaa9477210a029a3651c0568ed64bcf1e1af4ef1d74e6df07d1af5e  Stored in directory: /tmp/pip-ephem-wheel-cache-1P6O4z/wheels/71/bf/7f/2a1535926ab6419a15cd28eee5f3f6107c88fd2fe2c99dfa08  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Created wheel for dumptruck: filename=dumptruck-0.1.6-cp27-none-any.whl size=11845 sha256=2b03ef3fb2a6157097104b791d3e5ae9c7660fd547741fdd4caa2e76bd8aeb8d  Stored in directory: /tmp/pip-ephem-wheel-cache-1P6O4z/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e  Building wheel for regex (setup.py): started

Data

Downloaded 1451 times by everypolitician duncanparkes tmtmtmtm MikeRalphson Created4

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (441 KB) Use the API

rows 10 / 12

term_number name start_date id end_date
5
5th National Council: 2015 - 2020
2015
5th National Council: 2015 - 2020
2020
4
4th National Council: 2010 - 2015
2010
4th National Council: 2010 - 2015
2015
3
3rd National Council: 2004 - 2010
2004
3rd National Council: 2004 - 2010
2010
2
2nd National Council: 1998 - 2004
1998
2nd National Council: 1998 - 2004
2004
1
1st National Council: 1993 - 1998
1993
1st National Council: 1993 - 1998
1998
6
6th National Assembly: 2015 - 2020
2015
6th National Assembly: 2015 - 2020
2020
5
5th National Assembly: 2010 - 2015
2010
5th National Assembly: 2010 - 2015
2015
4
4th National Assembly: 2005 - 2010
2005
4th National Assembly: 2005 - 2010
2010
3
3rd National Assembly: 2000 - 2005
2000
3rd National Assembly: 2000 - 2005
2005
2
2nd National Assembly: 1995 - 2000
1995
2nd National Assembly: 1995 - 2000
2000

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (441 KB) Use the API

rows 10 / 749

image party term details_url chamber area name term_id id email note
DTA
6th National Assembly: 2015 - 2020
National Assembly
Van den Heever, Jenniffer Muriel
6th National Assembly: 2015 - 2020
Van-den-Heever-Jenniffer-Muriel
jennifer@dtaofnamibia.org
Appointed by President
5th National Council: 2015 - 2020
National Council
Shangula, Kalumbi
5th National Council: 2015 - 2020
Shangula-Kalumbi
RDP
5th National Assembly: 2010 - 2015
National Assembly
Nehova "Kandy" Shafodino Hinananye
5th National Assembly: 2010 - 2015
Nehova-Kandy-Shafodino-Hinananye
SWAPO
1st National Assembly: 1990 - 1995
National Assembly
Mweshihange Peter
1st National Assembly: 1990 - 1995
Mweshihange-Peter
SWAPO
Constituent Assembly: 1989 - 1990
National Assembly
Meroro David
Constituent Assembly: 1989 - 1990
Meroro-David
SWAPO
1st National Council: 1993 - 1998
National Council
Kaiyamo Hosea Nico
1st National Council: 1993 - 1998
Kaiyamo-Hosea-Nico
SWAPO
6th National Assembly: 2015 - 2020
National Assembly
Shapwa, Lidwina Ndeshimona
6th National Assembly: 2015 - 2020
Shapwa-Lidwina-Ndeshimona
lshapwa@moj.gov.na
DTA
1st National Assembly: 1990 - 1995
National Assembly
Gende Allois
1st National Assembly: 1990 - 1995
Gende-Allois
SWAPO
3rd National Assembly: 2000 - 2005
National Assembly
Amathila Appolus Libertine
3rd National Assembly: 2000 - 2005
Amathila-Appolus-Libertine
SWAPO
4th National Assembly: 2005 - 2010
National Assembly
Geingob Hage
4th National Assembly: 2005 - 2010
Geingob-Hage

Statistics

Average successful run time: about 1 hour

Total run time: 3 months

Total cpu time used: about 9 hours

Total disk space used: 478 KB

History

  • Auto ran revision 92bb4aa6 and failed .
    nothing changed in the database
  • Auto ran revision 92bb4aa6 and failed .
    nothing changed in the database
  • Auto ran revision 92bb4aa6 and failed .
    nothing changed in the database
  • Auto ran revision 92bb4aa6 and failed .
    nothing changed in the database
  • Auto ran revision 92bb4aa6 and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

namibia / scraper.py