technicallynotdeaf / SA_members_for_OA_parser

Scraper part #2, SA Lower House Member Data

Contributors

Last run completed successfully 2017-10-19.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Compiling Ruby [1G-----> Using Ruby version: ruby-2.0.0 [1G-----> Installing dependencies using bundler 1.13.7 [1G Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment [1G Fetching gem metadata from https://rubygems.org/........ [1G Fetching version metadata from https://rubygems.org/. [1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git [1G Rubygems 2.0.14.1 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation. [1G Installing unf_ext 0.0.7.1 with native extensions [1G Installing httpclient 2.6.0.1 [1G Installing mime-types 2.5 [1G Installing net-http-digest_auth 1.4 [1G Installing net-http-persistent 2.9.4 [1G Installing mini_portile 0.6.2 [1G Installing ntlm-http 0.1.1 [1G Installing webrobots 0.1.1 [1G Installing sqlite3 1.3.10 with native extensions [1G Using bundler 1.13.7 [1G Installing unf 0.1.4 [1G Installing nokogiri 1.6.6.2 with native extensions [1G Installing sqlite_magic 0.0.3 [1G Installing domain_name 0.5.24 [1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176) [1G Installing http-cookie 1.0.2 [1G Installing mechanize 2.7.3 [1G Bundle complete! 2 Gemfile dependencies, 17 gems now installed. [1G Gems in the groups development and test were not installed. [1G Bundled gems are installed into ./vendor/bundle. [1G Bundle completed (21.33s) [1G Cleaning up the bundler cache. [1G-----> Detecting rake tasks [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... (+ upper house Senator ignored) Name: Hon Michael Atkinson {:member_count=>"531", :id__saparl=>"531", :full_name=>"Hon Michael Atkinson", :division=>"Croydon", :state=>"SA", :start_date=>"25.11.1989", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Ms Frances Bedford {:member_count=>"543", :id__saparl=>"543", :full_name=>"Ms Frances Bedford", :division=>"Florey", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"IND"} Name: Mr Troy Bell {:member_count=>"4844", :id__saparl=>"4844", :full_name=>"Mr Troy Bell", :division=>"Mount Gambier", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"IND"} Name: Hon Zoe Bettison {:member_count=>"4623", :id__saparl=>"4623", :full_name=>"Hon Zoe Bettison", :division=>"Ramsay", :state=>"SA", :start_date=>"11.02.2012", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Hon Leon Bignell {:member_count=>"3117", :id__saparl=>"3117", :full_name=>"Hon Leon Bignell", :division=>"Mawson", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Hon Geoff Brock {:member_count=>"3533", :id__saparl=>"3533", :full_name=>"Hon Geoff Brock", :division=>"Frome", :state=>"SA", :start_date=>"17.01.2009", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"IND"} (+ upper house Senator ignored) Name: Hon Paul Caica {:member_count=>"1802", :id__saparl=>"1802", :full_name=>"Hon Paul Caica", :division=>"Colton", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Ms Vickie Chapman {:member_count=>"1804", :id__saparl=>"1804", :full_name=>"Ms Vickie Chapman", :division=>"Bragg", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Hon Susan Close {:member_count=>"4622", :id__saparl=>"4622", :full_name=>"Hon Susan Close", :division=>"Port Adelaide", :state=>"SA", :start_date=>"11.02.2012", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Ms Nat Cook {:member_count=>"4978", :id__saparl=>"4978", :full_name=>"Ms Nat Cook", :division=>"Fisher", :state=>"SA", :start_date=>"16.12.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) (+ upper house Senator ignored) Name: Mrs Annabel Digance {:member_count=>"4838", :id__saparl=>"4838", :full_name=>"Mrs Annabel Digance", :division=>"Elder", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Mr Sam Duluk {:member_count=>"4991", :id__saparl=>"4991", :full_name=>"Mr Sam Duluk", :division=>"Davenport", :state=>"SA", :start_date=>"31.01.2015", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} (+ upper house Senator ignored) (+ upper house Senator ignored) Name: Mr John Gardner {:member_count=>"4343", :id__saparl=>"4343", :full_name=>"Mr John Gardner", :division=>"Morialta", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} (+ upper house Senator ignored) Name: Mr Jon Gee {:member_count=>"4845", :id__saparl=>"4845", :full_name=>"Mr Jon Gee", :division=>"Napier", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Mr Mark Goldsworthy {:member_count=>"1805", :id__saparl=>"1805", :full_name=>"Mr Mark Goldsworthy", :division=>"Kavel", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Steven Griffiths {:member_count=>"3118", :id__saparl=>"3118", :full_name=>"Mr Steven Griffiths", :division=>"Goyder", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Hon Martin Hamilton-Smith {:member_count=>"631", :id__saparl=>"631", :full_name=>"Hon Martin Hamilton-Smith", :division=>"Waite", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"IND"} (+ upper house Senator ignored) Name: Hon Katrine Hildyard {:member_count=>"4846", :id__saparl=>"4846", :full_name=>"Hon Katrine Hildyard", :division=>"Reynell", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) Name: Mr Eddie Hughes {:member_count=>"4839", :id__saparl=>"4839", :full_name=>"Mr Eddie Hughes", :division=>"Giles", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) Name: Hon Tom Kenyon {:member_count=>"3119", :id__saparl=>"3119", :full_name=>"Hon Tom Kenyon", :division=>"Newland", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Hon Steph Key {:member_count=>"539", :id__saparl=>"539", :full_name=>"Hon Steph Key", :division=>"Ashford", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Mr Stephan Knoll {:member_count=>"4847", :id__saparl=>"4847", :full_name=>"Mr Stephan Knoll", :division=>"Schubert", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Hon Tom Koutsantonis {:member_count=>"633", :id__saparl=>"633", :full_name=>"Hon Tom Koutsantonis", :division=>"West Torrens", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) (+ upper house Senator ignored) (+ upper house Senator ignored) (+ upper house Senator ignored) (+ upper house Senator ignored) Name: Mr Steven Marshall {:member_count=>"4338", :id__saparl=>"4338", :full_name=>"Mr Steven Marshall", :division=>"Dunstan", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Dr Duncan McFetridge {:member_count=>"1807", :id__saparl=>"1807", :full_name=>"Dr Duncan McFetridge", :division=>"Morphett", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"IND"} (+ upper house Senator ignored) Name: Hon Stephen Mullighan {:member_count=>"4842", :id__saparl=>"4842", :full_name=>"Hon Stephen Mullighan", :division=>"Lee", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) Name: Mr Lee Odenwalder {:member_count=>"4335", :id__saparl=>"4335", :full_name=>"Mr Lee Odenwalder", :division=>"Little Para", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) Name: Mr Adrian Pederick {:member_count=>"3120", :id__saparl=>"3120", :full_name=>"Mr Adrian Pederick ", :division=>"Hammond", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Michael Pengilly {:member_count=>"3121", :id__saparl=>"3121", :full_name=>"Mr Michael Pengilly ", :division=>"Finniss", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Hon Tony Piccolo {:member_count=>"3123", :id__saparl=>"3123", :full_name=>"Hon Tony Piccolo", :division=>"Light", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Hon Chris Picton {:member_count=>"4841", :id__saparl=>"4841", :full_name=>"Hon Chris Picton", :division=>"Kaurna", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Mr David Pisoni {:member_count=>"3124", :id__saparl=>"3124", :full_name=>"Mr David Pisoni", :division=>"Unley", :state=>"SA", :start_date=>"18.03.2006", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Hon Jennifer Rankine {:member_count=>"614", :id__saparl=>"614", :full_name=>"Hon Jennifer Rankine", :division=>"Wright", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Hon John Rau {:member_count=>"1810", :id__saparl=>"1810", :full_name=>"Hon John Rau", :division=>"Enfield", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Ms Isobel Redmond {:member_count=>"1813", :id__saparl=>"1813", :full_name=>"Ms Isobel Redmond", :division=>"Heysen", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} (+ upper house Senator ignored) Name: Ms Rachel Sanderson {:member_count=>"4340", :id__saparl=>"4340", :full_name=>"Ms Rachel Sanderson", :division=>"Adelaide", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Jack Snelling {:member_count=>"627", :id__saparl=>"627", :full_name=>"Mr Jack Snelling", :division=>"Playford", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Mr David Speirs {:member_count=>"4837", :id__saparl=>"4837", :full_name=>"Mr David Speirs", :division=>"Bright", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} (+ upper house Senator ignored) Name: Mr Vincent Tarzia {:member_count=>"4840", :id__saparl=>"4840", :full_name=>"Mr Vincent Tarzia", :division=>"Hartley", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Peter Treloar {:member_count=>"4341", :id__saparl=>"4341", :full_name=>"Mr Peter Treloar", :division=>"Flinders", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Dan van Holst Pellekaan {:member_count=>"4342", :id__saparl=>"4342", :full_name=>"Mr Dan van Holst Pellekaan", :division=>"Stuart", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} (+ upper house Senator ignored) Name: Ms Leesa Vlahos {:member_count=>"4336", :id__saparl=>"4336", :full_name=>"Ms Leesa Vlahos ", :division=>"Taylor", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored) Name: Hon Jay Weatherill {:member_count=>"1812", :id__saparl=>"1812", :full_name=>"Hon Jay Weatherill", :division=>"Cheltenham", :state=>"SA", :start_date=>"09.02.2002", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} Name: Mr Tim Whetstone {:member_count=>"4339", :id__saparl=>"4339", :full_name=>"Mr Tim Whetstone", :division=>"Chaffey", :state=>"SA", :start_date=>"20.03.2010", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Mitch Williams {:member_count=>"546", :id__saparl=>"546", :full_name=>"Mr Mitch Williams", :division=>"MacKillop", :state=>"SA", :start_date=>"11.10.1997", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Mr Corey Wingard {:member_count=>"4843", :id__saparl=>"4843", :full_name=>"Mr Corey Wingard", :division=>"Mitchell", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"LIB"} Name: Ms Dana Wortley {:member_count=>"4848", :id__saparl=>"4848", :full_name=>"Ms Dana Wortley", :division=>"Torrens", :state=>"SA", :start_date=>"15.03.2014", :election_type=>"", :end_date=>"", :reason_left=>"still_in_office", :party=>"ALP"} (+ upper house Senator ignored)

Statistics

Average successful run time: 2 minutes

Total run time: 17 minutes

Total cpu time used: half a minute

Total disk space used: 40 KB

History

Manually ran revision 90aa1921 and completed successfully 2017-10-19.

run time 2 minutes

47 records added, 47 records removed in the database
Manually ran revision 90aa1921 and completed successfully 2016-10-29.

run time 3 minutes

47 records added, 47 records removed in the database

48 pages scraped
Manually ran revision 90aa1921 and failed 2016-09-22.

run time 18 s

5 records added, 5 records removed in the database

6 pages scraped
Manually ran revision 90aa1921 and completed successfully 2016-09-22.

run time 2 minutes

47 records added, 47 records removed in the database

48 pages scraped
Manually ran revision 9c00567a and failed 2016-09-22.

run time 7 s

nothing changed in the database

2 pages scraped
...
Created on morph.io 2016-09-15

Show complete history

Scraper code

SA_members_for_OA_parser

git clone URL