Contributors chrismytton

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.2.4 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/...  Fetching dependency metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Installing buftok 0.2.0  Installing coderay 1.1.1  Installing addressable 2.4.0  Installing dotenv 2.1.0  Installing unf_ext 0.0.7.2 with native extensions  Installing equalizer 0.0.10  Installing multipart-post 2.0.0  Installing http-form_data 1.0.1  Installing http_parser.rb 0.6.0 with native extensions  Installing httpclient 2.8.0  Installing json 1.8.3 with native extensions  Installing thread_safe 0.3.5  Installing method_source 0.8.2  Installing naught 1.1.0  Installing slop 3.6.0  Installing sqlite3 1.3.11 with native extensions  Installing simple_oauth 0.3.1  Using bundler 1.11.2  Installing faraday 0.9.2  Installing unf 0.1.4  Installing memoizable 0.4.2  Installing pry 0.10.3  Installing domain_name 0.5.20160310  Installing http-cookie 1.0.2  Installing http 1.0.4  Installing twitter 5.16.0  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Bundle complete! 4 Gemfile dependencies, 28 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (24.92s)  Cleaning up the bundler cache.   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.2.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 227 times by everypolitician chrismytton

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (93 KB) Use the API

rows 10 / 564

id name twitter image
474991141
Guto Bebb
GutoBebb
2613582806
Chris Davies MP
ChrisDavies_MP
303707744
Mike Freer MP
mikefreermp
1880622294
Derek Thomas
DerekThomas2015
2922923291
Richard Bacon
Bacon4SNorfolk
3390861040
Shailesh Vara
ShaileshVara
3346442313
Leader's Office
CommonsLeader
3318714389
Susan Elan Jones MP
susanelanjones
3306903539
Edward Timpson MP
edwardtimpson
3301636361
@MhairiBlack
Mhairi1921

Statistics

Average successful run time: less than a minute

Total run time: about 2 hours

Total cpu time used: 6 minutes

Total disk space used: 128 KB

History

  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and failed .
    nothing changed in the database
  • Manually ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran and failed .
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    77 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    102 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    559 records added, 559 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    560 records added, 560 records removed in the database
    28 pages scraped
  • Manually ran revision 42019e24 and completed successfully .
    560 records added, 560 records removed in the database
    28 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    561 records added, 561 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    561 records added, 561 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    100 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    47 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and failed .
    40 records added, 40 records removed in the database
    2 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and failed .
    500 records added, 500 records removed in the database
    25 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and failed .
    nothing changed in the database
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    52 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    26 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    nothing changed in the database
    27 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    563 records added, 563 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Auto ran revision 42019e24 and completed successfully .
    562 records added, 562 records removed in the database
    29 pages scraped
  • Manually ran revision 42019e24 and completed successfully .
    564 records added, 564 records removed in the database
  • Manually ran revision 0164c127 and completed successfully .
    564 records added in the database
  • Created on morph.io

Scraper code

Ruby

twitter-gov-uk-mps / scraper.rb