tmtmtmtm / croatia-parliament

Scrapes github.com

Build software better, together.


This is a scraper that runs on Morph. To get started see the documentation

Contributors tmtmtmtm octopusinvitro ondenman chrismytton

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... /app/vendor/bundle/ruby/2.3.0/gems/git-1.3.0/lib/git/lib.rb:937:in `command': git clone '--' 'https://c4748fd51c1d2185bdd5bce2fc6373a1afc27295@github.com/tmtmtmtm/croatia-parliament' '/tmp/d20180217-18-wlzmzt' 2>&1:Cloning into '/tmp/d20180217-18-wlzmzt'... (Git::GitExecuteError) from /app/vendor/bundle/ruby/2.3.0/gems/git-1.3.0/lib/git/lib.rb:76:in `clone' from /app/vendor/bundle/ruby/2.3.0/gems/git-1.3.0/lib/git/base.rb:29:in `clone' from /app/vendor/bundle/ruby/2.3.0/gems/git-1.3.0/lib/git.rb:97:in `clone' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-28f93d74b1c1/lib/scraped_page_archive/git_storage.rb:52:in `git' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-28f93d74b1c1/lib/scraped_page_archive/git_storage.rb:17:in `path' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-28f93d74b1c1/lib/scraped_page_archive.rb:35:in `record' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-28f93d74b1c1/lib/scraped_page_archive/open-uri.rb:9:in `open_uri' from /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/open-uri.rb:717:in `open' from /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/open-uri.rb:35:in `open' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request/strategy/live_request.rb:10:in `response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:29:in `block in first_successful_response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `each' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `first' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:30:in `first_successful_response' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped-23a9620a8b0c/lib/scraped/request.rb:13:in `response' from scraper.rb:18:in `scrape' from scraper.rb:22:in `scrape_list' from scraper.rb:30:in `<main>'

Statistics

Average successful run time: about 6 hours

Total run time: 5 months

Total cpu time used: 2 months

Total disk space used: 529 MB

History

  • Auto ran revision 0942cb39 and failed .
    503 records removed in the database
    2 pages scraped
  • Auto ran revision 0942cb39 and completed successfully .
    503 records added in the database
    2735 pages scraped
  • Auto ran revision 0942cb39 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 0942cb39 and failed .
    68 records removed in the database
    2 pages scraped
  • Auto ran revision 0942cb39 and failed .
    435 records removed in the database
    348 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

croatia-parliament / scraper.rb