Contributors tmtmtmtm chrismytton ondenman

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... `Scraped::Response::Decorator::AbsoluteUrls` has been deprecated. Use `Scraped::Response::Decorator::CleanUrls` instead. `Scraped::Response::Decorator::AbsoluteUrls` has been deprecated. Use `Scraped::Response::Decorator::CleanUrls` instead. The 'scraped_page_archive' gem wants to store the scraped pages in a git repo,but it cannot determine which git repo it should use. See https://github.com/everypolitician/scraped_page_archive#usage for details of how to specify the repo. Found 449 members The 'scraped_page_archive' gem wants to store the scraped pages in a git repo,but it cannot determine which git repo it should use. See https://github.com/everypolitician/scraped_page_archive#usage for details of how to specify the repo. /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/uri/rfc2396_parser.rb:305:in `gsub': invalid byte sequence in UTF-8 (ArgumentError) from /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/uri/rfc2396_parser.rb:305:in `escape' from /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/uri/common.rb:100:in `escape' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator/clean_urls.rb:16:in `to_s' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator/clean_urls.rb:40:in `absolute_url' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator/clean_urls.rb:33:in `block (2 levels) in body' from /app/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.7.1/lib/nokogiri/xml/node_set.rb:187:in `block in each' from /app/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.7.1/lib/nokogiri/xml/node_set.rb:186:in `upto' from /app/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.7.1/lib/nokogiri/xml/node_set.rb:186:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator/clean_urls.rb:33:in `block in body' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator/clean_urls.rb:31:in `tap' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator/clean_urls.rb:31:in `body' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response/decorator.rb:10:in `decorated_response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response_decorator.rb:15:in `block in response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response_decorator.rb:9:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response_decorator.rb:9:in `reduce' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/response_decorator.rb:9:in `response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/document.rb:36:in `response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-c458dd2601a7/lib/scraped/document.rb:40:in `url' from /app/lib/member_page.rb:9:in `block in <class:MemberPage>' from /app/vendor/bundle/ruby/2.3.0/gems/field_serializer-0.3.0/lib/field_serializer.rb:26:in `block in to_h' from /app/vendor/bundle/ruby/2.3.0/gems/field_serializer-0.3.0/lib/field_serializer.rb:25:in `map' from /app/vendor/bundle/ruby/2.3.0/gems/field_serializer-0.3.0/lib/field_serializer.rb:25:in `to_h' from scraper.rb:31:in `block in <main>' from scraper.rb:30:in `each' from scraper.rb:30:in `<main>'

Statistics

Average successful run time: about 6 hours

Total run time: about 1 month

Total cpu time used: 18 days

Total disk space used: 83.1 MB

History

  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 9e7817be and failed .
    nothing changed in the database
  • Manually ran revision 9e7817be and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    nothing changed in the database
  • Auto ran revision 6b2fcf6e and failed .
    452 records removed in the database
  • Manually ran revision ded789cd and completed successfully .
    256 records added in the database
  • Auto ran revision ded789cd and failed .
    143 records added in the database
  • Auto ran revision ded789cd and failed .
    261 records removed in the database
  • Auto ran revision ded789cd and failed .
    204 records added in the database
  • Auto ran revision ded789cd and failed .
    14 records added in the database
  • Auto ran revision ded789cd and failed .
    356 records removed in the database
  • Auto ran revision ded789cd and completed successfully .
    270 records added in the database
  • Auto ran revision ded789cd and failed .
    270 records removed in the database
  • Auto ran revision ded789cd and completed successfully .
    363 records added in the database
  • Auto ran revision ded789cd and failed .
    363 records removed in the database
  • Auto ran revision ded789cd and completed successfully .
    nothing changed in the database
  • Auto ran revision ded789cd and completed successfully .
    nothing changed in the database
  • Auto ran revision ded789cd and completed successfully .
    nothing changed in the database
  • Auto ran revision ded789cd and completed successfully .
    142 records added in the database
  • Auto ran revision ded789cd and failed .
    142 records removed, 310 records updated in the database
  • Auto ran revision 07fc21ab and completed successfully .
    120 records added in the database
  • Auto ran revision 07fc21ab and failed .
    120 records removed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    194 records added, 1 record updated in the database
  • Auto ran revision 07fc21ab and failed .
    194 records removed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Manually ran revision 07fc21ab and completed successfully .
    96 records added in the database
  • Auto ran revision 07fc21ab and failed .
    96 records removed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    1 record updated in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    1 record updated in the database
  • Auto ran revision 07fc21ab and completed successfully .
    283 records added in the database
  • Auto ran revision 07fc21ab and failed .
    283 records removed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    216 records added in the database
  • Auto ran revision 07fc21ab and failed .
    216 records removed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    1 record updated in the database
  • Auto ran revision 07fc21ab and completed successfully .
    3 records added in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and failed .
    99 records added in the database
  • Auto ran revision 07fc21ab and failed .
    102 records removed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    2 records updated in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    1 record updated in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    2 records added, 154 records updated in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1806 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1806 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1806 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    1 record updated in the database
    1806 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    2 records removed, 219 records updated in the database
    1806 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    nothing changed in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    348 records added in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and failed .
    348 records removed in the database
    423 pages scraped
  • Auto ran revision 07fc21ab and completed successfully .
    290 records added in the database
    1814 pages scraped
  • Auto ran revision 07fc21ab and failed .
    287 records removed, 161 records updated in the database
    8664 pages scraped
  • Auto ran revision 07fc21ab and failed .
    7287 pages scraped
  • Auto ran revision 07fc21ab and failed .
    nothing changed in the database
    6073 pages scraped
  • Auto ran revision 07fc21ab and failed .
    nothing changed in the database
    3183 pages scraped
  • Auto ran revision aebe9b8d and completed successfully .
    nothing changed in the database
    1802 pages scraped
  • Manually ran revision aebe9b8d and completed successfully .
    449 records updated in the database
    1802 pages scraped
  • Auto ran and failed .
  • Auto ran revision 7abfe4ec and completed successfully .
    367 records updated in the database
    1802 pages scraped
  • Auto ran revision 7abfe4ec and failed .
    nothing changed in the database
  • Auto ran revision 7abfe4ec and completed successfully .
    nothing changed in the database
    1802 pages scraped
  • Auto ran revision 7abfe4ec and failed .
    nothing changed in the database
  • Auto ran revision 7abfe4ec and failed .
    nothing changed in the database
  • Auto ran revision 7abfe4ec and failed .
    nothing changed in the database
  • Auto ran revision 7abfe4ec and completed successfully .
    2 records updated in the database
    1803 pages scraped
  • Auto ran revision 7abfe4ec and failed .
    nothing changed in the database
    1281 pages scraped
  • Auto ran revision 7abfe4ec and completed successfully .
    nothing changed in the database
    1802 pages scraped
  • Auto ran revision 7abfe4ec and completed successfully .
    344 records added in the database
    1802 pages scraped
  • Auto ran revision 7abfe4ec and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 7abfe4ec and failed .
    344 records removed in the database
    429 pages scraped
  • Auto ran revision 7abfe4ec and completed successfully .
    446 records added in the database
    1802 pages scraped
  • Auto ran revision 7abfe4ec and failed .
    3 records added, 452 records removed in the database
    18 pages scraped
  • Auto ran revision 661414de and failed .
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and failed .
    nothing changed in the database
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and failed .
    353 records added, 353 records removed in the database
    1420 pages scraped
  • Auto ran revision 661414de and failed .
    408 records added, 408 records removed in the database
    1639 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and failed .
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 448 records removed in the database
    1802 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    1798 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    1798 pages scraped
  • Auto ran revision 661414de and failed .
    148 records added, 148 records removed in the database
    599 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    1799 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    1798 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    3597 pages scraped
  • Auto ran revision 661414de and failed .
    253 records added, 253 records removed in the database
    1019 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    1798 pages scraped
  • Auto ran revision 661414de and completed successfully .
    448 records added, 448 records removed in the database
    1798 pages scraped
  • Auto ran revision 661414de and failed .
    352 records added, 352 records removed in the database
    1415 pages scraped
  • Auto ran revision 661414de and completed successfully .
    5 records added, 5 records removed in the database
    27 pages scraped
  • Auto ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
  • Auto ran revision 661414de and completed successfully .
    nothing changed in the database
  • Auto ran revision 661414de and failed .
    nothing changed in the database
    219 pages scraped
  • Auto ran revision 661414de and completed successfully .
    nothing changed in the database
    1802 pages scraped
  • Manually ran revision 661414de and completed successfully .
    449 records added, 449 records removed in the database
    1802 pages scraped
  • Auto ran revision 1beafedd and failed .
    61 records added, 61 records removed in the database
    375 pages scraped
  • Auto ran revision 1beafedd and failed .
    64 records added, 64 records removed in the database
  • Auto ran revision 1beafedd and failed .
    404 records added, 403 records removed in the database
    2433 pages scraped
  • Auto ran revision 1beafedd and failed .
    54 records added, 54 records removed in the database
    333 pages scraped
  • Auto ran revision 1beafedd and failed .
    323 records added, 323 records removed in the database
    1947 pages scraped
  • Auto ran revision 1beafedd and failed .
    75 records added, 75 records removed in the database
    459 pages scraped
  • Auto ran revision 1beafedd and failed .
    91 records added, 91 records removed in the database
    555 pages scraped
  • Auto ran revision 1beafedd and failed .
    56 records added, 56 records removed in the database
    345 pages scraped
  • Auto ran revision 1beafedd and failed .
    59 records added, 59 records removed in the database
    363 pages scraped
  • Auto ran revision 1beafedd and failed .
    91 records added, 91 records removed in the database
    555 pages scraped
  • Auto ran revision 1beafedd and failed .
    nothing changed in the database
    686 pages scraped
  • Auto ran revision 1beafedd and failed .
    147 records added, 147 records removed in the database
  • Auto ran revision 1beafedd and failed .
    nothing changed in the database
  • Auto ran revision 1beafedd and failed .
    230 records added, 230 records removed in the database
    1389 pages scraped
  • Auto ran revision 1beafedd and failed .
    263 records added, 263 records removed in the database
    1587 pages scraped
  • Auto ran revision 1beafedd and failed .
    69 records added, 69 records removed in the database
    423 pages scraped
  • Auto ran revision 1beafedd and failed .
    63 records added, 63 records removed in the database
    387 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and failed .
    43 records added, 43 records removed in the database
    267 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    5388 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    448 records added, 448 records removed in the database
    2694 pages scraped
  • Auto ran revision 1beafedd and failed .
    nothing changed in the database
    459 pages scraped
  • Auto ran revision 1beafedd and failed .
    nothing changed in the database
    704 pages scraped
  • Auto ran revision 1beafedd and failed .
    nothing changed in the database
    2617 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    449 records added, 449 records removed in the database
    2700 pages scraped
  • Auto ran revision 1beafedd and failed .
    275 records added, 275 records removed in the database
    1659 pages scraped
  • Auto ran revision 1beafedd and failed .
    9 records added, 9 records removed in the database
    61 pages scraped
  • Auto ran revision 1beafedd and completed successfully .
    450 records added, 450 records removed in the database
    2706 pages scraped
  • Auto ran revision 1beafedd and failed .
    349 records added, 349 records removed in the database
    2102 pages scraped
  • Manually ran revision 1beafedd and completed successfully .
    450 records added, 450 records removed in the database
    2706 pages scraped
  • Manually ran revision e8e2a846 and completed successfully .
    450 records added, 11 records removed in the database
    451 pages scraped
  • Manually ran revision 7e8b3b07 and failed .
    11 records added in the database
    13 pages scraped
  • Created on morph.io

Scraper code

Ruby

russia-duma-2016 / scraper.rb