everypolitician-scrapers / russia-duma

https://morph.io/tmtmtmtm/russia-duma

Scrapes github.com and www.duma.gov.ru

Build software better, together.


This is a scraper that runs on Morph. To get started see the documentation

Contributors octopusinvitro tmtmtmtm

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.1 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/..........  Fetching version metadata from https://rubygems.org/...  Fetching dependency metadata from https://rubygems.org/..  Fetching https://github.com/everypolitician/scraped_page_archive  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Installing colorize 0.7.7  Installing public_suffix 2.0.4  Installing coderay 1.1.0  Installing execjs 2.5.2  Installing safe_yaml 1.0.4  Installing excon 0.45.4  Installing fuzzy_match 2.1.0  Installing git 1.3.0  Installing multipart-post 2.0.0  Installing hashdiff 0.3.1  Installing hashie 3.4.2  Installing method_source 0.8.2  Installing httpclient 2.6.0.1  Installing mini_portile 0.6.2  Installing slop 3.6.0  Installing open-uri-cached 0.0.5  Using bundler 1.11.2  Installing vcr 3.0.3  Installing sqlite3 1.3.10 with native extensions  Installing addressable 2.5.0  Installing faraday 0.9.1  Installing crack 0.4.3  Installing pry 0.10.1  Installing faraday_middleware 0.10.0  Installing webmock 2.0.3  Installing nokogiri 1.6.6.2 with native extensions  Installing wikidata-client 0.0.7  Installing vcr-archive 0.3.0  Using scraped_page_archive 0.5.0 from https://github.com/everypolitician/scraped_page_archive (at master@28f93d7)  Installing sqlite_magic 0.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Bundle complete! 9 Gemfile dependencies, 31 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from webmock:  WebMock 2.0 has some breaking changes. Please check the CHANGELOG: https://goo.gl/piDGLu  Bundle completed (19.54s)  Cleaning up the bundler cache.  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (335 KB) Use the API

rows 10 / 903

id name faction region image term source
131337
Абалаков Александр Николаевич
КПРФ
Новосибирская область
6
239916
Абасов Мамед Магарамович
«ЕДИНАЯ РОССИЯ»
Республика Дагестан
6
131273
Абрамов Иван Николаевич
ЛДПР
Республика Саха (Якутия), Камчатский край, Амурская область, Магаданская область, Чукотский автономный округ
6
131431
Абубакиров Ришат Фазлутдинович
«ЕДИНАЯ РОССИЯ»
Республика Татарстан (Татарстан)
6
131338
Авдеев Михаил Юрьевич
КПРФ
Московская область
6
131432
Агаев Бекхан Вахаевич
«ЕДИНАЯ РОССИЯ»
Удмуртская Республика
6
131339
Агаев Ваха Абуевич
КПРФ
Красноярский край
6
131221
Агеев Александр Александрович
«СПРАВЕДЛИВАЯ РОССИЯ»
Город Москва - Восточная, Город Москва - Донская, Город Москва - Люблинская
6
131222
Аксаков Анатолий Геннадьевич
«СПРАВЕДЛИВАЯ РОССИЯ»
Республика Марий Эл, Чувашская Республика - Чувашия
6
174437
Акулов Тимур Юрьевич
«ЕДИНАЯ РОССИЯ»
Республика Татарстан (Татарстан)
6

Statistics

Average successful run time: 1 minute

Total run time: about 2 hours

Total cpu time used: 5 minutes

Total disk space used: 7.1 MB

History

  • Auto ran revision 5ff24717 and completed successfully .
    447 records added, 447 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    447 records added, 447 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    447 records added, 447 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    7 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and failed .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    nothing changed in the database
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 5ff24717 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    449 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    nothing changed in the database
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision b4fbafc2 and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    76 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran and failed .
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    8 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    7 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    448 records added, 448 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    449 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 449 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    450 records added, 450 records removed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 1840446d and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Manually ran revision 1840446d and completed successfully .
    450 records added in the database
    6 pages scraped
  • Manually ran revision de5071f6 and failed .
    nothing changed in the database
  • Created on morph.io

Scraper code

Ruby

russia-duma / scraper.rb