mashedkeyboard / wales-AMs-wikidata

README.md

This is a scraper that runs on Morph. To get started see the documentation

Contributors tmtmtmtm

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:363:in `open_http': 429 Too many requests (620f469) (OpenURI::HTTPError) from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:741:in `buffer_open' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:212:in `block in open_loop' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:210:in `catch' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:210:in `open_loop' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:151:in `open_uri' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:721:in `open' from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/open-uri.rb:35:in `open' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:50:in `get' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:18:in `block in initialize' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:17:in `each' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:17:in `each_slice' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:17:in `each' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:17:in `map' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:17:in `initialize' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:8:in `new' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/lookup.rb:8:in `find' from /app/vendor/bundle/ruby/2.4.0/gems/wikisnakker-0.9.1/lib/wikisnakker/item.rb:4:in `find' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/wikidata-fetcher-e14ebbff9c91/lib/wikidata/fetcher.rb:15:in `find' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/wikidata-fetcher-e14ebbff9c91/lib/wikidata.rb:136:in `block in scrape_wikidata' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/wikidata-fetcher-e14ebbff9c91/lib/wikidata.rb:134:in `each' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/wikidata-fetcher-e14ebbff9c91/lib/wikidata.rb:134:in `each_slice' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/wikidata-fetcher-e14ebbff9c91/lib/wikidata.rb:134:in `scrape_wikidata' from scraper.rb:10:in `<main>'

Data

Downloaded 11 times by mashedkeyboard

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (195 KB) Use the API

rows 0 / 0

Statistics

Average successful run time: less than a minute

Total run time: about 15 hours

Total cpu time used: about 1 hour

Total disk space used: 220 KB

History

  • Auto ran revision f989c45d and failed .
    161 records removed in the database
  • Auto ran revision f989c45d and completed successfully .
    159 records updated in the database
  • Auto ran revision f989c45d and completed successfully .
    161 records added in the database
  • Auto ran revision f989c45d and failed .
    161 records removed in the database
  • Auto ran revision f989c45d and completed successfully .
    160 records updated in the database
  • ...
  • Created on { hostname }

Show complete history

Scraper code

Ruby

wales-AMs-wikidata / scraper.rb