tmhammer / i94_monthly_data

I-94 International Arrivals Data by Month


This is a scraper that runs on Morph. To get started see the documentation

Contributors tmhammer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.2.2 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/...  Fetching dependency metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Installing mini_portile2 2.1.0  Installing httpclient 2.8.0  Installing json 2.0.2 with native extensions  Installing pkg-config 1.1.7  Installing rubyzip 1.2.0  Installing ruby-ole 1.2.12  Using bundler 1.11.2  Installing sqlite3 1.3.11 with native extensions  Installing nokogiri 1.6.8 with native extensions  Installing spreadsheet 1.1.2  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Installing roo 2.4.0  Installing dropbox-sdk 1.6.5  Installing roo-xls 1.0.0  Bundle complete! 4 Gemfile dependencies, 15 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (23.01s)  Cleaning up the bundler cache.  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... /app/vendor/bundle/ruby/2.2.0/gems/dropbox-sdk-1.6.5/lib/dropbox_sdk.rb:103:in `rescue in do_http': SSL error connecting to Dropbox. There may be a problem with the set of certificates in "/app/vendor/bundle/ruby/2.2.0/gems/dropbox-sdk-1.6.5/lib/trusted-certs.crt". SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (DropboxError) from /app/vendor/bundle/ruby/2.2.0/gems/dropbox-sdk-1.6.5/lib/dropbox_sdk.rb:100:in `do_http' from /app/vendor/bundle/ruby/2.2.0/gems/dropbox-sdk-1.6.5/lib/dropbox_sdk.rb:179:in `do_http' from /app/vendor/bundle/ruby/2.2.0/gems/dropbox-sdk-1.6.5/lib/dropbox_sdk.rb:188:in `do_get' from /app/vendor/bundle/ruby/2.2.0/gems/dropbox-sdk-1.6.5/lib/dropbox_sdk.rb:1122:in `metadata' from /app/dropbox_client.rb:12:in `get_file_paths' from scraper.rb:12:in `main' from scraper.rb:83:in `<main>'

Statistics

Average successful run time: 2 minutes

Total run time: 33 minutes

Total cpu time used: 9 minutes

Total disk space used: 83.4 KB

History

  • Manually ran revision 0c3499ab and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 2b4f5978 and failed .
    nothing changed in the database
  • Manually ran revision 290cfbec and completed successfully .
    49560 records added, 49556 records removed in the database
  • Manually ran revision 290cfbec and completed successfully .
    49556 records added in the database
  • Manually ran revision 4d7b0652 and failed .
    nothing changed in the database
  • Manually ran revision 9c0416e5 and completed successfully .
    49556 records added, 46910 records removed in the database
  • Manually ran revision 9c0416e5 and completed successfully .
    46910 records added, 46500 records removed in the database
  • Manually ran revision 3834e77b and completed successfully .
    46500 records added in the database
  • Manually ran revision d334575b and completed successfully .
    46522 records added in the database
  • Manually ran revision d334575b and completed successfully .
    46522 records added in the database
  • Manually ran revision 8ea74a98 and completed successfully .
    45467 records added, 45205 records removed in the database
  • Manually ran revision 0e2cd8a3 and completed successfully .
    46260 records added in the database
  • Manually ran revision 0e2cd8a3 and completed successfully .
    46260 records added, 43212 records removed in the database
  • Manually ran revision 6e95733e and completed successfully .
    46260 records added, 45852 records removed in the database
  • Manually ran revision 9490028b and completed successfully .
    45852 records added, 42948 records removed in the database
  • Manually ran revision 45df5ef9 and completed successfully .
    42948 records added in the database
  • Manually ran revision d45692ee and failed .
    nothing changed in the database
  • Manually ran revision e0ec02d7 and completed successfully .
    5520 records added in the database
  • Manually ran revision e0ec02d7 and completed successfully .
    5520 records added, 5520 records removed in the database
  • Manually ran revision 7d708041 and failed .
    nothing changed in the database
  • Manually ran revision b970d471 and completed successfully .
    5544 records added, 5544 records removed in the database
  • Manually ran revision 212dcb4e and completed successfully .
    5544 records added in the database
  • Created on morph.io

Scraper code

Ruby

i94_monthly_data / scraper.rb