morph-test-scrapers / theyvoteforyou_policy_positions

The number of Australian MPs in each policy position on TheyVoteForYou.org.au

Scrapes theyvoteforyou.org.au

Discover how your MP votes on the issues that matter to you.


The purpose of this scraper is to track basic data on how MP positions for each policy in They Vote For You change over time.

The They Vote For You API provides more detailed data on MP positions for the current time, but this scraper allows you to see how many MPs were in each category on a specific date.

This information is useful for me for an experiment I want to hack on, and I thought it might be an interesting thing to look back on one day.

Contributors equivalentideas

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... {:name=>"same-sex marriage equality", :id=>1, :datetime_scraped=>"2015-07-25T02:38:58+00:00", :scrape_id=>"1_20150725", :provisional=>"false", :voted_very_strongly_for=>34, :voted_moderately_for=>8, :voted_a_mixture_of_for_and_against=>19, :voted_moderately_against=>20, :voted_strongly_against=>8, :voted_very_strongly_against=>59, "division_count"=>15} /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/openssl/ssl.rb:232:in `post_connection_check': hostname "theyvoteforyou.org.au" does not match the server certificate (OpenSSL::SSL::SSLError) from /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/http.rb:923:in `connect' from /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/http.rb:862:in `do_start' from /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/http.rb:857:in `start' from /app/vendor/bundle/ruby/2.0.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start' from /app/vendor/bundle/ruby/2.0.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:631:in `connection_for' from /app/vendor/bundle/ruby/2.0.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request' from /app/vendor/bundle/ruby/2.0.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:259:in `fetch' from /app/vendor/bundle/ruby/2.0.0/gems/mechanize-2.7.3/lib/mechanize.rb:440:in `get' from scraper.rb:22:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3 KB) Use the API

rows 1 / 1

name id datetime_scraped scrape_id provisional voted_very_strongly_for voted_moderately_for voted_a_mixture_of_for_and_against voted_moderately_against voted_strongly_against voted_very_strongly_against division_count
same-sex marriage equality
1
2015-07-25T02:38:58+00:00
1_20150725
false
34
8
19
20
8
59
15

Statistics

Total run time: less than a minute

Total cpu time used: less than 5 seconds

Total disk space used: 26.5 KB

History

  • Manually ran revision e6559e64 and failed .
    1 record added in the database
    2 pages scraped
  • Manually ran revision e6559e64 and failed .
    nothing changed in the database
    1 page scraped
  • Created on morph.io