This is a scraper that runs on Morph. To get started see the documentation

Contributors tmtmtmtm mashedkeyboard

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Compiling Ruby [1G-----> Using Ruby version: ruby-2.6.0 [1G-----> Installing dependencies using bundler version 1.17.2 [1G Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment [1G Fetching gem metadata from https://rubygems.org/.......... [1G Fetching https://github.com/everypolitician/combine_popolo_memberships.git [1G Fetching https://github.com/everypolitician/scraped.git [1G Fetching https://github.com/everypolitician/scraped_page_archive.git [1G Fetching https://github.com/everypolitician/scraper_test.git [1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git [1G Fetching https://github.com/everypolitician/table_unspanner.git [1G Fetching public_suffix 4.0.6 [1G Using bundler 1.17.2 [1G Fetching coderay 1.1.3 [1G Using combine_popolo_memberships 0.2.0 from https://github.com/everypolitician/combine_popolo_memberships.git (at master@5769841) [1G Fetching rexml 3.2.5 [1G Installing coderay 1.1.3 [1G Installing public_suffix 4.0.6 [1G Installing rexml 3.2.5 [1G Fetching field_serializer 0.3.0 [1G Installing field_serializer 0.3.0 [1G Fetching git 1.3.0 [1G Fetching hashdiff 1.0.1 [1G Fetching httpclient 2.8.3 [1G Installing git 1.3.0 [1G Installing httpclient 2.8.3 [1G Installing hashdiff 1.0.1 [1G Fetching method_source 1.0.0 [1G Fetching mini_portile2 2.7.1 [1G Installing method_source 1.0.0 [1G Fetching minitest 5.15.0 [1G Installing mini_portile2 2.7.1 [1G Installing minitest 5.15.0 [1G Fetching racc 1.6.0 [1G Installing racc 1.6.0 with native extensions [1G Fetching require_all 3.0.0 [1G Fetching vcr 3.0.3 [1G Installing require_all 3.0.0 [1G Installing vcr 3.0.3 [1G Fetching sqlite3 1.4.2 [1G Installing sqlite3 1.4.2 with native extensions [1G Fetching addressable 2.8.0 [1G Installing addressable 2.8.0 [1G Fetching crack 0.4.5 [1G Installing crack 0.4.5 [1G Fetching pry 0.14.1 [1G Installing pry 0.14.1 [1G Fetching webmock 2.0.3 [1G Installing webmock 2.0.3 [1G Fetching vcr-archive 0.3.0 [1G Installing vcr-archive 0.3.0 [1G Using scraper_test 0.1.0 from https://github.com/everypolitician/scraper_test.git (at master@9b4326c) [1G Using scraped_page_archive 0.5.0 from https://github.com/everypolitician/scraped_page_archive.git (at master@a998412) [1G Fetching nokogiri 1.13.0 (x86_64-linux) [1G Fetching sqlite_magic 0.0.6 [1G Installing sqlite_magic 0.0.6 [1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176) [1G Installing nokogiri 1.13.0 (x86_64-linux) [1G Using scraped 0.6.2 from https://github.com/everypolitician/scraped.git (at master@7fd4391) [1G Using table_unspanner 0.1.1 from https://github.com/everypolitician/table_unspanner.git (at master@238e7f9) [1G Bundle complete! 8 Gemfile dependencies, 28 gems now installed. [1G Gems in the groups development and test were not installed. [1G Bundled gems are installed into `./vendor/bundle` [1G Post-install message from webmock: [1G WebMock 2.0 has some breaking changes. Please check the CHANGELOG: https://goo.gl/piDGLu [1G Removing bundler (1.15.2) [1G Bundle completed (10.28s) [1G Cleaning up the bundler cache. [1G-----> Detecting rake tasks [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 435 times by loudspeek-bot mashedkeyboard asiamemo pysak tomgoodthings

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (176 KB) Use the API

rows 10 / 650

id name sort_name birth_date death_date gender party party_id constituency email phone fax website twitter facebook blog youtube flickr identifier__dods identifier__pims
172
Ms Diane Abbott
Abbott, Ms Diane
female
Labour
15
Hackney North and Stoke Newington
diane.abbott.office@parliament.uk
020 7219 4426
25790
3572
5131
Jack Abbott
Abbott, Jack
male
Labour (Co-op)
15
Ipswich
jack.abbott.mp@parliament.uk
6547
4212
Debbie Abrahams
Abrahams, Debbie
female
Labour
15
Oldham East and Saddleworth
abrahamsd@parliament.uk
020 7219 1041
80556
5905
5120
Shockat Adam
Adam, Shockat
male
Independent
8
Leicester South
shockat.adam.mp@parliament.uk
6536
5213
Dr Zubir Ahmed
Ahmed, Dr Zubir
male
Labour
15
Glasgow South West
zubir.ahmed.mp@parliament.uk
6629
5112
Luke Akehurst
Akehurst, Luke
male
Labour
15
North Durham
luke.akehurst.mp@parliament.uk
6528
5097
Mr Bayo Alaba
Alaba, Mr Bayo
male
Labour
15
Southend East and Rochford
bayo.alaba.mp@parliament.uk
6513
5172
Dan Aldridge
Aldridge, Dan
male
Labour
15
Weston-super-Mare
daniel.aldridge.mp@parliament.uk
6588
632
Mr Douglas Alexander
Alexander, Mr Douglas
male
Labour (Co-op)
15
Lothian East
douglas.alexander.mp@parliament.uk
25705
3629
4038
Heidi Alexander
Alexander, Heidi
female
Labour
15
Swindon South
heidi.alexander.mp@parliament.uk
83154
5563

Statistics

Average successful run time: 1 minute

Total run time: 1 day

Total cpu time used: 40 minutes

Total disk space used: 226 KB

History

  • Auto ran revision 55f449eb and completed successfully .
    nothing changed in the database
  • Auto ran revision 55f449eb and completed successfully .
    9 records updated in the database
  • Auto ran revision 55f449eb and completed successfully .
    12 records updated in the database
  • Auto ran revision 55f449eb and completed successfully .
    5 records updated in the database
  • Auto ran revision 55f449eb and completed successfully .
    1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

uk-parliament-members / scraper.rb