duncanparkes / namibia

Scrape the members of the Namibian parliament.


This is a scraper that runs on Morph. To get started see the documentation

Contributors duncanparkes

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 8))  Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1 (from -r /tmp/build/requirements.txt (line 9))  Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz  Collecting PyExecJS==1.1.0 (from -r /tmp/build/requirements.txt (line 11))  Downloading https://files.pythonhosted.org/packages/0d/6b/5183948f2a12b7df0d597273f941a26baa35520505ec8d670f4c9b568749/PyExecJS-1.1.0.tar.gz  Collecting awesome-slugify==1.6.4 (from -r /tmp/build/requirements.txt (line 13))  Downloading https://files.pythonhosted.org/packages/2f/2a/adc6ce1178b938286c9ecc81b99c60dcac3e66c1b3f0fa72255af03a5cf6/awesome-slugify-1.6.4.tar.gz  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)  Collecting regex (from awesome-slugify==1.6.4->-r /tmp/build/requirements.txt (line 13))  Downloading https://files.pythonhosted.org/packages/e1/57/21c7a8e0af95c31d76845d3feeabe2cc794c91f9be5db2d37cea2cbc6d36/regex-2018.06.20.tar.gz (632kB)  Collecting Unidecode<0.05,>=0.04.14 (from awesome-slugify==1.6.4->-r /tmp/build/requirements.txt (line 13))  Downloading https://files.pythonhosted.org/packages/01/a1/9d7f3138ee3d79a1ab865a2cb38200ca778d85121db19fe264c76c981184/Unidecode-0.04.21-py2.py3-none-any.whl (228kB)  Collecting idna<2.8,>=2.5 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)  Collecting urllib3<1.24,>=1.21.1 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Installing collected packages: dumptruck, idna, certifi, urllib3, chardet, requests, scraperwiki, lxml, cssselect, PyExecJS, regex, Unidecode, awesome-slugify  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for cssselect: started  Running setup.py install for cssselect: finished with status 'done'  Running setup.py install for PyExecJS: started  Running setup.py install for PyExecJS: finished with status 'done'  Running setup.py install for regex: started  Running setup.py install for regex: finished with status 'done'  Running setup.py install for awesome-slugify: started  Running setup.py install for awesome-slugify: finished with status 'done'  Successfully installed PyExecJS-1.1.0 Unidecode-0.4.21 awesome-slugify-1.6.4 certifi-2018.4.16 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 idna-2.7 lxml-3.4.4 regex-2018.6.20 requests-2.19.1 scraperwiki urllib3-1.23   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... http://www.parliament.na/index.php?option=com_contact&view=category&id=108&Itemid=1573 http://www.parliament.na/index.php?option=com_contact&view=category&id=108&Itemid=1573&limitstart=20 http://www.parliament.na/index.php?option=com_contact&view=category&id=108&Itemid=1573&limitstart=40 http://www.parliament.na/index.php?option=com_contact&view=category&id=136&Itemid=1376 http://www.parliament.na/index.php?option=com_contact&view=category&id=136&Itemid=1376&limitstart=20 http://www.parliament.na/index.php?option=com_contact&view=category&id=124&Itemid=1377 http://www.parliament.na/index.php?option=com_contact&view=category&id=124&Itemid=1377&limitstart=20 http://www.parliament.na/index.php?option=com_contact&view=category&id=122&Itemid=1378 http://www.parliament.na/index.php?option=com_contact&view=category&id=122&Itemid=1378&limitstart=20 http://www.parliament.na/index.php?option=com_contact&view=category&id=119&Itemid=1379 http://www.parliament.na/index.php?option=com_contact&view=category&id=119&Itemid=1379&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=104&Itemid=1516 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=104&Itemid=1516&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=104&Itemid=1516&limitstart=40 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=104&Itemid=1516&limitstart=60 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=104&Itemid=1516&limitstart=80 No image found for Munsu, Norah Lute in 6th National Assembly: 2015 - 2020 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=104&Itemid=1516&limitstart=100 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=129&Itemid=1370 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=129&Itemid=1370&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=129&Itemid=1370&limitstart=40 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=129&Itemid=1370&limitstart=60 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=129&Itemid=1370&limitstart=80 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=125&Itemid=1371 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=125&Itemid=1371&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=125&Itemid=1371&limitstart=40 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=125&Itemid=1371&limitstart=60 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=123&Itemid=1372 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=123&Itemid=1372&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=123&Itemid=1372&limitstart=40 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=123&Itemid=1372&limitstart=60 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=123&Itemid=1372&limitstart=80 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=121&Itemid=1373 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=121&Itemid=1373&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=121&Itemid=1373&limitstart=40 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=121&Itemid=1373&limitstart=60 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=120&Itemid=1374 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=120&Itemid=1374&limitstart=20 Duplicate (name, term) pair ignored: (Muyongo Mishake, 1st National Assembly: 1990 - 1995) http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=120&Itemid=1374&limitstart=40 No image found for Hubschle Michaela in 1st National Assembly: 1990 - 1995 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=120&Itemid=1374&limitstart=60 No image found for Herrigel Otto in 1st National Assembly: 1990 - 1995 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=120&Itemid=1374&limitstart=80 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=118&Itemid=1375 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=118&Itemid=1375&limitstart=20 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=118&Itemid=1375&limitstart=40 http://www.parliament.gov.na/index.php?option=com_contact&view=category&id=118&Itemid=1375&limitstart=60

Data

Downloaded 1259 times by everypolitician duncanparkes tmtmtmtm MikeRalphson Created4

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (441 KB) Use the API

rows 10 / 12

term_number name start_date id end_date
5
5th National Council: 2015 - 2020
2015
5th National Council: 2015 - 2020
2020
4
4th National Council: 2010 - 2015
2010
4th National Council: 2010 - 2015
2015
3
3rd National Council: 2004 - 2010
2004
3rd National Council: 2004 - 2010
2010
2
2nd National Council: 1998 - 2004
1998
2nd National Council: 1998 - 2004
2004
1
1st National Council: 1993 - 1998
1993
1st National Council: 1993 - 1998
1998
6
6th National Assembly: 2015 - 2020
2015
6th National Assembly: 2015 - 2020
2020
5
5th National Assembly: 2010 - 2015
2010
5th National Assembly: 2010 - 2015
2015
4
4th National Assembly: 2005 - 2010
2005
4th National Assembly: 2005 - 2010
2010
3
3rd National Assembly: 2000 - 2005
2000
3rd National Assembly: 2000 - 2005
2005
2
2nd National Assembly: 1995 - 2000
1995
2nd National Assembly: 1995 - 2000
2000

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (441 KB) Use the API

rows 10 / 745

image party term details_url chamber area name term_id id email note
DTA
6th National Assembly: 2015 - 2020
National Assembly
Van den Heever, Jenniffer Muriel
6th National Assembly: 2015 - 2020
Van-den-Heever-Jenniffer-Muriel
jennifer@dtaofnamibia.org
SWAPO
2nd National Council: 1998 - 2004
National Council
Shinohamba Karlous Marx
2nd National Council: 1998 - 2004
Shinohamba-Karlous-Marx
SWAPO
4th National Assembly: 2005 - 2010
National Assembly
Tsheehama A.T. Peter
4th National Assembly: 2005 - 2010
Tsheehama-A-T-Peter
SWAPO
3rd National Assembly: 2000 - 2005
National Assembly
Wentworth J.W. Buddy
3rd National Assembly: 2000 - 2005
Wentworth-J-W-Buddy
SWAPO
5th National Assembly: 2010 - 2015
National Assembly
Angula Nahas
5th National Assembly: 2010 - 2015
Angula-Nahas
SWAPO
4th National Assembly: 2005 - 2010
National Assembly
Gurirab Theo Ben
4th National Assembly: 2005 - 2010
Gurirab-Theo-Ben
SWAPO
2nd National Assembly: 1995 - 2000
National Assembly
Shoombe Pashukeni
2nd National Assembly: 1995 - 2000
Shoombe-Pashukeni
SWAPO
2nd National Assembly: 1995 - 2000
National Assembly
!Naruseb Alpheus
2nd National Assembly: 1995 - 2000
Naruseb-Alpheus
SWAPO
4th National Council: 2010 - 2015
National Council
Nhinda Ruth Kepawa
4th National Council: 2010 - 2015
Nhinda-Ruth-Kepawa
r.nhinda@parliament.na
DECEASED
Appointed by President
6th National Assembly: 2015 - 2020
National Assembly
Hanse-Himarwa, Katrina
6th National Assembly: 2015 - 2020
Hanse-Himarwa-Katrina

Statistics

Average successful run time: about 1 hour

Total run time: 2 months

Total cpu time used: about 6 hours

Total disk space used: 478 KB

History

  • Auto ran revision 92bb4aa6 and completed successfully .
    756 records added, 756 records removed in the database
  • Auto ran revision 92bb4aa6 and completed successfully .
    756 records added, 756 records removed in the database
  • Auto ran revision 92bb4aa6 and completed successfully .
    756 records added, 756 records removed in the database
    791 pages scraped
  • Auto ran revision 92bb4aa6 and failed .
    nothing changed in the database
  • Auto ran revision 92bb4aa6 and completed successfully .
    756 records added, 756 records removed in the database
    791 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

namibia / scraper.py