richygit / qld_mp_data

Scrapes www.parliament.qld.gov.au

Queensland Parliament


Contributors richygit

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching rake 10.4.2  Fetching Ascii85 1.0.2  Fetching afm 0.2.2  Installing Ascii85 1.0.2  Installing afm 0.2.2  Using bundler 1.15.2  Fetching unf_ext 0.0.7.1  Installing rake 10.4.2  Fetching hashery 2.1.1  Installing hashery 2.1.1  Fetching httpclient 2.6.0.1  Fetching mime-types 2.6.1  Installing unf_ext 0.0.7.1 with native extensions  Installing httpclient 2.6.0.1  Installing mime-types 2.6.1  Fetching net-http-digest_auth 1.4  Installing net-http-digest_auth 1.4  Fetching net-http-persistent 2.9.4  Fetching mini_portile 0.6.2  Installing net-http-persistent 2.9.4  Installing mini_portile 0.6.2  Fetching ntlm-http 0.1.1  Fetching webrobots 0.1.1  Installing ntlm-http 0.1.1  Fetching ruby-rc4 0.1.5  Installing webrobots 0.1.1  Installing ruby-rc4 0.1.5  Fetching ttfunk 1.4.0  Fetching ruby-ole 1.2.11.8  Installing ttfunk 1.4.0  Installing ruby-ole 1.2.11.8  Fetching sqlite3 1.3.10  Fetching titleize 1.3.0  Installing titleize 1.3.0  Fetching nokogiri 1.6.6.2  Installing sqlite3 1.3.10 with native extensions  Installing nokogiri 1.6.6.2 with native extensions  Fetching pdf-reader 1.3.3  Installing pdf-reader 1.3.3  Fetching spreadsheet 1.0.4  Installing spreadsheet 1.0.4  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.24  Installing domain_name 0.5.24  Fetching http-cookie 1.0.2  Installing http-cookie 1.0.2  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Fetching scraperwiki 3.0.2  Installing scraperwiki 3.0.2  Fetching mechanize 2.7.3  Installing mechanize 2.7.3  Bundle complete! 15 Gemfile dependencies, 27 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from pdf-reader:  ********************************************  v1.0.0 of PDF::Reader introduced a new page-based API. There are extensive  examples showing how to use it in the README and examples directory.  For detailed documentation, check the rdocs for the PDF::Reader,  PDF::Reader::Page and PDF::Reader::ObjectHash classes.  The old API is marked as deprecated but will continue to work with no  visible warnings for now.  ********************************************  Bundle completed (25.78s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... I, [2018-04-19T09:35:51.957154 #17] INFO -- : Scraping XLS I, [2018-04-19T09:35:54.913700 #17] INFO -- : Scraping web I, [2018-04-19T09:36:48.978891 #17] INFO -- : ### Saving Leeanne Enoch ### Saving Leeanne Enoch I, [2018-04-19T09:36:48.995155 #17] INFO -- : ### Saving Bart Mellish ### Saving Bart Mellish I, [2018-04-19T09:36:48.999422 #17] INFO -- : ### Saving Christopher Whiting ### Saving Christopher Whiting I, [2018-04-19T09:36:49.002687 #17] INFO -- : ### Saving Craig Crawford ### Saving Craig Crawford I, [2018-04-19T09:36:49.005568 #17] INFO -- : ### Saving Samuel O'Connor ### Saving Samuel O'Connor I, [2018-04-19T09:36:49.009235 #17] INFO -- : ### Saving David Crisafulli ### Saving David Crisafulli I, [2018-04-19T09:36:49.012520 #17] INFO -- : ### Saving Brent Mickelberg ### Saving Brent Mickelberg I, [2018-04-19T09:36:49.015699 #17] INFO -- : ### Saving Dianne Farmer ### Saving Dianne Farmer I, [2018-04-19T09:36:49.032655 #17] INFO -- : ### Saving David Batt ### Saving David Batt I, [2018-04-19T09:36:49.036386 #17] INFO -- : ### Saving Jo-Ann Miller ### Saving Jo-Ann Miller I, [2018-04-19T09:36:49.039656 #17] INFO -- : ### Saving Dale Last ### Saving Dale Last I, [2018-04-19T09:36:49.043011 #17] INFO -- : ### Saving Michael Hart ### Saving Michael Hart I, [2018-04-19T09:36:49.046856 #17] INFO -- : ### Saving Stephen Bennett ### Saving Stephen Bennett I, [2018-04-19T09:36:49.050210 #17] INFO -- : ### Saving Michael Healy ### Saving Michael Healy I, [2018-04-19T09:36:49.053646 #17] INFO -- : ### Saving Colin Boyce ### Saving Colin Boyce I, [2018-04-19T09:36:49.061545 #17] INFO -- : ### Saving Mark McArdle ### Saving Mark McArdle I, [2018-04-19T09:36:49.067160 #17] INFO -- : ### Saving Donald Brown ### Saving Donald Brown I, [2018-04-19T09:36:49.070485 #17] INFO -- : ### Saving Steven Minnikin ### Saving Steven Minnikin I, [2018-04-19T09:36:49.073849 #17] INFO -- : ### Saving Timothy Nicholls ### Saving Timothy Nicholls I, [2018-04-19T09:36:49.077583 #17] INFO -- : ### Saving Patrick Weir ### Saving Patrick Weir I, [2018-04-19T09:36:49.082017 #17] INFO -- : ### Saving Cynthia Lui ### Saving Cynthia Lui I, [2018-04-19T09:36:49.085609 #17] INFO -- : ### Saving Michael Crandon ### Saving Michael Crandon I, [2018-04-19T09:36:49.088653 #17] INFO -- : ### Saving Kate Jones ### Saving Kate Jones I, [2018-04-19T09:36:49.091093 #17] INFO -- : ### Saving Janet Stuckey ### Saving Janet Stuckey I, [2018-04-19T09:36:49.093135 #17] INFO -- : ### Saving Timothy Mander ### Saving Timothy Mander I, [2018-04-19T09:36:49.095372 #17] INFO -- : ### Saving Mark Furner ### Saving Mark Furner I, [2018-04-19T09:36:49.097791 #17] INFO -- : ### Saving Meaghan Scanlon ### Saving Meaghan Scanlon I, [2018-04-19T09:36:49.099840 #17] INFO -- : ### Saving Glenn Butcher ### Saving Glenn Butcher I, [2018-04-19T09:36:49.102101 #17] INFO -- : ### Saving Andrew Powell ### Saving Andrew Powell I, [2018-04-19T09:36:49.108408 #17] INFO -- : ### Saving Joseph Kelly ### Saving Joseph Kelly I, [2018-04-19T09:36:49.116421 #17] INFO -- : ### Saving Lachlan Millar ### Saving Lachlan Millar I, [2018-04-19T09:36:49.119258 #17] INFO -- : ### Saving Anthony Perrett ### Saving Anthony Perrett I, [2018-04-19T09:36:49.122203 #17] INFO -- : ### Saving Edward Sorensen ### Saving Edward Sorensen I, [2018-04-19T09:36:49.123851 #17] INFO -- : ### Saving Shane Knuth ### Saving Shane Knuth I, [2018-04-19T09:36:49.126106 #17] INFO -- : ### Saving Nicholas Dametto ### Saving Nicholas Dametto I, [2018-04-19T09:36:49.129669 #17] INFO -- : ### Saving Annastacia Palaszczuk ### Saving Annastacia Palaszczuk I, [2018-04-19T09:36:49.160495 #17] INFO -- : ### Saving Jennifer Howard ### Saving Jennifer Howard I, [2018-04-19T09:36:49.173657 #17] INFO -- : ### Saving James Madden ### Saving James Madden I, [2018-04-19T09:36:49.186088 #17] INFO -- : ### Saving Charis Mullen ### Saving Charis Mullen I, [2018-04-19T09:36:49.189272 #17] INFO -- : ### Saving Jarrod Bleijie ### Saving Jarrod Bleijie I, [2018-04-19T09:36:49.192604 #17] INFO -- : ### Saving Brittany Lauga ### Saving Brittany Lauga I, [2018-04-19T09:36:49.195482 #17] INFO -- : ### Saving Shane King ### Saving Shane King I, [2018-04-19T09:36:49.198490 #17] INFO -- : ### Saving James McDonald ### Saving James McDonald I, [2018-04-19T09:36:49.203415 #17] INFO -- : ### Saving Linus Power ### Saving Linus Power I, [2018-04-19T09:36:49.205613 #17] INFO -- : ### Saving Joan Pease ### Saving Joan Pease I, [2018-04-19T09:36:49.208368 #17] INFO -- : ### Saving Melissa McMahon ### Saving Melissa McMahon I, [2018-04-19T09:36:49.210402 #17] INFO -- : ### Saving Julieanne Gilbert ### Saving Julieanne Gilbert I, [2018-04-19T09:36:49.212944 #17] INFO -- : ### Saving Michael Berkman ### Saving Michael Berkman I, [2018-04-19T09:36:49.216327 #17] INFO -- : ### Saving Corrine McMillan ### Saving Corrine McMillan I, [2018-04-19T09:36:49.219835 #17] INFO -- : ### Saving Fiona Simpson ### Saving Fiona Simpson I, [2018-04-19T09:36:49.222675 #17] INFO -- : ### Saving Bruce Saunders ### Saving Bruce Saunders I, [2018-04-19T09:36:49.226358 #17] INFO -- : ### Saving Grace Grace ### Saving Grace Grace I, [2018-04-19T09:36:49.230970 #17] INFO -- : ### Saving Raymond Stevens ### Saving Raymond Stevens I, [2018-04-19T09:36:49.251379 #17] INFO -- : ### Saving Mark Bailey ### Saving Mark Bailey I, [2018-04-19T09:36:49.271636 #17] INFO -- : ### Saving Stephen Andrew ### Saving Stephen Andrew I, [2018-04-19T09:36:49.279971 #17] INFO -- : ### Saving Christian Rowan ### Saving Christian Rowan I, [2018-04-19T09:36:49.282183 #17] INFO -- : ### Saving Mark Ryan ### Saving Mark Ryan I, [2018-04-19T09:36:49.285707 #17] INFO -- : ### Saving Jessica Pugh ### Saving Jessica Pugh I, [2018-04-19T09:36:49.288536 #17] INFO -- : ### Saving Rosslyn Bates ### Saving Rosslyn Bates I, [2018-04-19T09:36:49.290945 #17] INFO -- : ### Saving Curtis Pitt ### Saving Curtis Pitt I, [2018-04-19T09:36:49.293033 #17] INFO -- : ### Saving Coralee O'Rourke ### Saving Coralee O'Rourke I, [2018-04-19T09:36:49.295618 #17] INFO -- : ### Saving Dr Miles ### Saving Dr Miles I, [2018-04-19T09:36:49.298506 #17] INFO -- : ### Saving Deborah Frecklington ### Saving Deborah Frecklington I, [2018-04-19T09:36:49.300549 #17] INFO -- : ### Saving Martin Hunt ### Saving Martin Hunt I, [2018-04-19T09:36:49.303181 #17] INFO -- : ### Saving Daniel Purdie ### Saving Daniel Purdie I, [2018-04-19T09:36:49.305507 #17] INFO -- : ### Saving Sandra Bolton ### Saving Sandra Bolton I, [2018-04-19T09:36:49.307831 #17] INFO -- : ### Saving Leanne Linard ### Saving Leanne Linard I, [2018-04-19T09:36:49.309341 #17] INFO -- : ### Saving Mark Robinson ### Saving Mark Robinson I, [2018-04-19T09:36:49.322542 #17] INFO -- : ### Saving Nikki Boyd ### Saving Nikki Boyd I, [2018-04-19T09:36:49.325515 #17] INFO -- : ### Saving Simone Wilson ### Saving Simone Wilson I, [2018-04-19T09:36:49.328809 #17] INFO -- : ### Saving Yvette D'Ath ### Saving Yvette D'Ath I, [2018-04-19T09:36:49.332563 #17] INFO -- : ### Saving Kim Richards ### Saving Kim Richards I, [2018-04-19T09:36:49.336195 #17] INFO -- : ### Saving Barry O'Rourke ### Saving Barry O'Rourke I, [2018-04-19T09:36:49.342415 #17] INFO -- : ### Saving Stirling Hinchliffe ### Saving Stirling Hinchliffe I, [2018-04-19T09:36:49.346980 #17] INFO -- : ### Saving Jon Krause ### Saving Jon Krause I, [2018-04-19T09:36:49.349932 #17] INFO -- : ### Saving Jacklyn Trad ### Saving Jacklyn Trad I, [2018-04-19T09:36:49.354024 #17] INFO -- : ### Saving James Lister ### Saving James Lister I, [2018-04-19T09:36:49.361062 #17] INFO -- : ### Saving Robert Molhoek ### Saving Robert Molhoek I, [2018-04-19T09:36:49.374070 #17] INFO -- : ### Saving Michael de Brenni ### Saving Michael de Brenni I, [2018-04-19T09:36:49.378897 #17] INFO -- : ### Saving Dr Lynham ### Saving Dr Lynham I, [2018-04-19T09:36:49.385626 #17] INFO -- : ### Saving Duncan Pegg ### Saving Duncan Pegg I, [2018-04-19T09:36:49.390181 #17] INFO -- : ### Saving John-Paul Langbroek ### Saving John-Paul Langbroek I, [2018-04-19T09:36:49.394241 #17] INFO -- : ### Saving Mark Boothman ### Saving Mark Boothman I, [2018-04-19T09:36:49.397843 #17] INFO -- : ### Saving Aaron Harper ### Saving Aaron Harper I, [2018-04-19T09:36:49.400031 #17] INFO -- : ### Saving Peter Russo ### Saving Peter Russo I, [2018-04-19T09:36:49.403287 #17] INFO -- : ### Saving Trevor Watts ### Saving Trevor Watts I, [2018-04-19T09:36:49.407095 #17] INFO -- : ### Saving David Janetzki ### Saving David Janetzki I, [2018-04-19T09:36:49.410100 #17] INFO -- : ### Saving Scott Stewart ### Saving Scott Stewart I, [2018-04-19T09:36:49.412834 #17] INFO -- : ### Saving Robert Katter ### Saving Robert Katter I, [2018-04-19T09:36:49.415934 #17] INFO -- : ### Saving Ann Leahy ### Saving Ann Leahy I, [2018-04-19T09:36:49.418763 #17] INFO -- : ### Saving Shannon Fentiman ### Saving Shannon Fentiman I, [2018-04-19T09:36:49.425596 #17] INFO -- : ### Saving Jason Costigan ### Saving Jason Costigan I, [2018-04-19T09:36:49.429315 #17] INFO -- : ### Saving Cameron Dick ### Saving Cameron Dick

Data

Downloaded 2 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (27 KB) Use the API

rows 10 / 105

last_name office_address office_suburb office_state office_postcode party first_name parliament_phone parliament_fax email electorate
Boothman
PO Box 1255
Beenleigh
QLD
4207
LNP
Mark
(07) 3445 0100
(07) 3445 0109
Albert@parliament.qld.gov.au
Albert
Jones
Shop 2, 230 Waterworks Road
Ashgrove
QLD
4060
ALP
Kate
(07) 3554 9100
(07) 3554 9109
Ashgrove@parliament.qld.gov.au
Ashgrove
Krause
PO Box 656
Beaudesert
QLD
4285
LNP
Jon
(07) 5515 1100
(07) 5515 1109
Beaudesert@parliament.qld.gov.au
Beaudesert
Grace
Unit 2, Hill House 541 Boundary Street (Corner Bradley Street)
Spring Hill
QLD
4000
ALP
Grace
(07) 3145 9100
(07) 3145 9109
Brisbane.Central@parliament.qld.gov.au
Brisbane Central
Robinson
PO Box 1399
Cleveland
QLD
4163
LNP
Mark
(07) 3446 9100
(07) 3446 9109
Cleveland@parliament.qld.gov.au
Cleveland
Knuth
Stock Exchange Arcade 2/76 Mosman Street
Charters Towers
QLD
4820
KAP
Shane
(07) 4235 7100
(07) 4235 7109
Dalrymple@parliament.qld.gov.au
Dalrymple
Emerson
Suite 1, 49 Station Road
Indooroopilly
QLD
4068
LNP
Scott
(07) 3737 4100
(07) 3737 4109
Indooroopilly@parliament.qld.gov.au
Indooroopilly
King
PO Box 629
Kallangur
QLD
4503
ALP
Shane
(07) 3448 9100
(07) 3448 9109
Kallangur@parliament.qld.gov.au
Kallangur
Miles
PO Box 1549
Milton
QLD
4064
ALP
Dr
(07) 3703 2100
07 3703 2109
Mount.Coot-tha@parliament.qld.gov.au
Mount Coot-tha
Katter
PO Box 1968
Mount Isa
QLD
4825
KAP
Robert
(07) 4730 1100
(07) 4730 1109
Mount.Isa@parliament.qld.gov.au
Mount Isa

Statistics

Average successful run time: 2 minutes

Total run time: 15 days

Total cpu time used: about 1 hour

Total disk space used: 34.4 MB

History

  • Auto ran revision b61f7ee6 and completed successfully .
    93 records added, 93 records removed in the database
    95 pages scraped
  • Auto ran revision b61f7ee6 and completed successfully .
    93 records added, 93 records removed in the database
    95 pages scraped
  • Auto ran revision b61f7ee6 and completed successfully .
    93 records added, 93 records removed in the database
    95 pages scraped
  • Auto ran revision b61f7ee6 and completed successfully .
    93 records added, 93 records removed in the database
    95 pages scraped
  • Auto ran revision b61f7ee6 and completed successfully .
    93 records added, 93 records removed in the database
    95 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

qld_mp_data / scraper.rb