everypolitician-scrapers / welsh-assembly-p39s

Contributors tmtmtmtm

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.4.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.5). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/..  Fetching dependency metadata from https://rubygems.org/.  Fetching https://github.com/everypolitician/scraped.git  Fetching https://github.com/openc/sqlite_magic.git  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching https://github.com/everypolitician/wikidata-fetcher.git  Fetching rake 12.3.2  Fetching ast 2.4.0  Using bundler 1.15.2  Fetching coderay 1.1.2  Installing ast 2.4.0  Installing rake 12.3.2  Installing coderay 1.1.2  Fetching unf_ext 0.0.7.6  Fetching excon 0.64.0  Fetching multipart-post 2.0.0  Installing multipart-post 2.0.0  Fetching field_serializer 0.3.0  Installing excon 0.64.0  Installing field_serializer 0.3.0  Fetching hashie 3.6.0  Installing hashie 3.6.0  Installing unf_ext 0.0.7.6 with native extensions  Fetching httpclient 2.8.3  Fetching jaro_winkler 1.5.2  Installing jaro_winkler 1.5.2 with native extensions  Installing httpclient 2.8.3  Fetching json 2.2.0  Installing json 2.2.0 with native extensions  Fetching method_source 0.9.2  Installing method_source 0.9.2  Fetching mime-types-data 3.2019.0331  Installing mime-types-data 3.2019.0331  Fetching mini_portile2 2.4.0  Installing mini_portile2 2.4.0  Fetching netrc 0.11.0  Installing netrc 0.11.0  Fetching parallel 1.17.0  Installing parallel 1.17.0  Fetching psych 3.1.0  Installing psych 3.1.0 with native extensions  Fetching rainbow 3.0.0  Installing rainbow 3.0.0  Fetching require_all 2.0.0  Installing require_all 2.0.0  Fetching ruby-progressbar 1.10.0  Installing ruby-progressbar 1.10.0  Fetching unicode-display_width 1.5.0  Installing unicode-display_width 1.5.0  Fetching sqlite3 1.4.0  Installing sqlite3 1.4.0 with native extensions  Fetching yajl-ruby 1.4.1  Installing yajl-ruby 1.4.1 with native extensions  Fetching parser 2.6.2.1  Installing parser 2.6.2.1  Fetching faraday 0.15.4  Installing faraday 0.15.4  Fetching pry 0.12.2  Installing pry 0.12.2  Fetching mime-types 3.2.2  Installing mime-types 3.2.2  Fetching nokogiri 1.10.3  Installing nokogiri 1.10.3 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Using sqlite_magic 0.0.6 from https://github.com/openc/sqlite_magic.git (at master@4df975e)  Fetching rubocop 0.67.2  Installing rubocop 0.67.2  Fetching faraday_middleware 0.13.1  Installing faraday_middleware 0.13.1  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Using scraped 0.6.2 from https://github.com/everypolitician/scraped.git (at master@7fd4391)  Fetching wikidata-client 0.0.12  Fetching http-cookie 1.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching wikisnakker 0.9.1  Installing http-cookie 1.0.3  Installing wikisnakker 0.9.1  Fetching faraday-cookie_jar 0.0.6  Fetching rest-client 2.0.2  Installing faraday-cookie_jar 0.0.6  Fetching mediawiki_api 0.7.1  Installing rest-client 2.0.2  Installing wikidata-client 0.0.12  Installing mediawiki_api 0.7.1  Using wikidata-fetcher 0.24.0 from https://github.com/everypolitician/wikidata-fetcher.git (at master@cb919d1)  Bundle complete! 9 Gemfile dependencies, 43 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from rubocop:  Performance Cops will be removed from RuboCop 0.68. Use rubocop-performance gem instead.  Put this in your Gemfile.  gem 'rubocop-performance'  And then execute:  $ bundle install  Put this into your .rubocop.yml.  require: rubocop-performance  More information: https://github.com/rubocop-hq/rubocop-performance  Bundle completed (38.95s)  Cleaning up the bundler cache.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.5). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:363:in `open_http': 403 Forbidden (OpenURI::HTTPError) from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:741:in `buffer_open' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:212:in `block in open_loop' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:210:in `catch' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:210:in `open_loop' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:151:in `open_uri' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:721:in `open' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:35:in `open' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request/strategy/live_request.rb:10:in `response' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:29:in `block in first_successful_response' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:30:in `first' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:30:in `first_successful_response' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/scraped-7fd43913456a/lib/scraped/request.rb:13:in `response' from scraper.rb:76:in `<main>'

Data

Downloaded 249 times by everypolitician loudspeek-bot tmtmtmtm mashedkeyboard

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (97 KB) Use the API

rows 10 / 322

statement id name constituency_id constituency party_id party start_date end_date term
Q3337819-4718D48A-0E36-4E5C-AC2A-DD8B648718B3
Q3337819
Neil Hamilton
Q6841063
Mid and West Wales
Q10647
UK Independence Party
2016-05-05
5
Q6530486-69E77B2B-F72D-40B2-BF11-4F2F4C79D872
Q6530486
Lesley Griffiths
Q8038114
Wrexham
Q832685
Welsh Labour
2016-05-05
5
Q24041564-D368B903-9B51-4964-B6D4-1CEECC0D87EB
Q24041564
Neil McEvoy
Q7568764
South Wales Central
Q10691
Plaid Cymru
2016-05-05
2018-01-16
5
Q24041564-efda1289-4fd7-2724-f268-152d519df21b
Q24041564
Neil McEvoy
Q7568764
South Wales Central
Q327591
independent politician
2018-01-16
5
Q46999598-b48153f8-46c1-4323-dcda-69c30452c8a0
Q46999598
Mandy Jones
Q7057143
North Wales
Q10647
UK Independence Party
2017-12-28
5
Q4762396-A7B4832B-FF8B-4EDF-B000-A110CCC06580
Q4762396
Angela Burns
Q5043210
Carmarthen West and South Pembrokeshire
Q7981882
Welsh Conservative Party
2016-05-05
5
Q4766480-CC07B6CA-FBA2-4976-A2F6-EBF477BA230A
Q4766480
Ann Jones
Q7909529
Vale of Clwyd
Q832685
Welsh Labour
2016-05-05
5
Q5236695-B09A7788-0243-4A96-856E-9E624A4BBA4E
Q5236695
David Lloyd
Q7568795
South Wales West
Q10691
Plaid Cymru
2016-05-05
5
Q6236400-A5610134-FCD8-4D8F-ABCC-7A690F62E847
Q6236400
John Griffiths
Q7018848
Newport East
Q832685
Welsh Labour
2016-05-05
5
Q6297656-1F30E75C-C979-40D8-9F7D-D40BB70E0787
Q6297656
Joyce Watson
Q6841063
Mid and West Wales
Q832685
Welsh Labour
2016-05-05
5

Statistics

Average successful run time: 1 minute

Total run time: about 11 hours

Total cpu time used: 12 minutes

Total disk space used: 130 KB

History

  • Auto ran revision 7fcbedcc and failed .
    nothing changed in the database
  • Auto ran revision 7fcbedcc and failed .
    nothing changed in the database
  • Auto ran revision 7fcbedcc and failed .
    nothing changed in the database
  • Auto ran revision 7fcbedcc and failed .
    nothing changed in the database
  • Auto ran revision 7fcbedcc and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

welsh-assembly-p39s / scraper.rb