tmtmtmtm / saint-lucia-assembly

Scrapes github.com and www.govt.lc

Build software better, together.


This is a scraper that runs on Morph. To get started see the documentation

Contributors tmtmtmtm ondenman

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.3 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/everypolitician/scraped.git  Fetching https://github.com/everypolitician/scraped_page_archive.git  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching rake 12.0.0  Fetching public_suffix 2.0.4  Fetching ast 2.3.0  Installing public_suffix 2.0.4  Installing ast 2.3.0  Using bundler 1.15.2  Fetching coderay 1.1.1  Installing rake 12.0.0  Fetching safe_yaml 1.0.4  Installing coderay 1.1.1  Installing safe_yaml 1.0.4  Fetching field_serializer 0.3.0  Fetching git 1.3.0  Installing field_serializer 0.3.0  Fetching hashdiff 0.3.0  Fetching httpclient 2.8.2.4  Installing hashdiff 0.3.0  Installing git 1.3.0  Fetching method_source 0.8.2  Fetching mini_portile2 2.1.0  Installing httpclient 2.8.2.4  Installing method_source 0.8.2  Installing mini_portile2 2.1.0  Fetching minitest 5.10.1  Fetching vcr 3.0.3  Installing minitest 5.10.1  Installing vcr 3.0.3  Fetching open-uri-cached 0.0.5  Fetching powerpack 0.1.1  Installing powerpack 0.1.1  Installing open-uri-cached 0.0.5  Fetching slop 3.6.0  Installing slop 3.6.0  Fetching rainbow 2.1.0  Fetching require_all 1.3.3  Installing require_all 1.3.3  Fetching ruby-progressbar 1.8.1  Installing ruby-progressbar 1.8.1  Installing rainbow 2.1.0  Fetching unicode-display_width 1.1.2  Fetching sqlite3 1.3.12  Fetching parser 2.3.3.1  Installing unicode-display_width 1.1.2  Fetching addressable 2.5.0  Installing sqlite3 1.3.12 with native extensions  Installing addressable 2.5.0  Fetching crack 0.4.3  Installing parser 2.3.3.1  Installing crack 0.4.3  Fetching nokogiri 1.6.8.1  Fetching minispec-metadata 2.0.0  Installing minispec-metadata 2.0.0  Fetching pry 0.10.4  Installing pry 0.10.4  Fetching rubocop 0.46.0  Installing rubocop 0.46.0  Fetching webmock 2.0.3  Installing webmock 2.0.3  Installing nokogiri 1.6.8.1 with native extensions  Fetching minitest-vcr 1.4.0  Installing minitest-vcr 1.4.0  Fetching vcr-archive 0.3.0  Installing vcr-archive 0.3.0  Using scraped_page_archive 0.5.0 from https://github.com/everypolitician/scraped_page_archive.git (at master@28f93d7)  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Using scraped 0.1.0 from https://github.com/everypolitician/scraped.git (at master@44f8c0b)  Bundle complete! 12 Gemfile dependencies, 36 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from webmock:  WebMock 2.0 has some breaking changes. Please check the CHANGELOG: https://goo.gl/piDGLu  Bundle completed (24.77s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Statistics

Average successful run time: less than a minute

Total run time: about 10 hours

Total cpu time used: 24 minutes

Total disk space used: 32.2 MB

History

  • Auto ran revision 90264b08 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 90264b08 and completed successfully .
    nothing changed in the database
  • Auto ran revision 90264b08 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 90264b08 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 90264b08 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saint-lucia-assembly / scraper.rb