This is a scraper that runs on Morph. To get started see the documentation

Contributors tmtmtmtm

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.3 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/..  Fetching dependency metadata from https://rubygems.org/.  Fetching https://github.com/everypolitician/everypolitician-ruby.git  Fetching https://github.com/openc/sqlite_magic.git  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching https://github.com/everypolitician/wikisnakker.git  Fetching https://github.com/everypolitician/wikidata-fetcher.git  Fetching rake 12.0.0  Fetching activesupport-inflector 0.1.0  Fetching ast 2.3.0  Installing ast 2.3.0  Installing rake 12.0.0  Using bundler 1.15.2  Fetching coderay 1.1.1  Fetching colorize 0.8.1  Installing activesupport-inflector 0.1.0  Fetching diskcached 1.1.3  Installing colorize 0.8.1  Installing diskcached 1.1.3  Fetching unf_ext 0.0.7.4  Installing coderay 1.1.1  Fetching require_all 1.4.0  Installing require_all 1.4.0  Fetching excon 0.55.0  Fetching multipart-post 2.0.0  Installing multipart-post 2.0.0  Fetching hashie 3.5.5  Installing excon 0.55.0  Installing hashie 3.5.5  Installing unf_ext 0.0.7.4 with native extensions  Fetching httpclient 2.8.3  Fetching i18n 0.8.1  Installing httpclient 2.8.3  Installing i18n 0.8.1  Fetching json 2.1.0  Installing json 2.1.0 with native extensions  Fetching method_source 0.8.2  Installing method_source 0.8.2  Fetching mime-types-data 3.2016.0521  Installing mime-types-data 3.2016.0521  Fetching mini_portile2 2.1.0  Installing mini_portile2 2.1.0  Fetching netrc 0.11.0  Installing netrc 0.11.0  Fetching open-uri-cached 0.0.5  Installing open-uri-cached 0.0.5  Fetching powerpack 0.1.1  Installing powerpack 0.1.1  Fetching slop 3.6.0  Installing slop 3.6.0  Fetching ruby-progressbar 1.8.1  Installing ruby-progressbar 1.8.1  Fetching unicode-display_width 1.2.1  Installing unicode-display_width 1.2.1  Fetching sqlite3 1.3.13  Installing sqlite3 1.3.13 with native extensions  Fetching yajl-ruby 1.3.0  Installing yajl-ruby 1.3.0 with native extensions  Fetching parser 2.4.0.0  Installing parser 2.4.0.0  Fetching rainbow 2.2.2  Installing rainbow 2.2.2 with native extensions  Fetching everypolitician-popolo 0.8.0  Installing everypolitician-popolo 0.8.0  Fetching faraday 0.12.1  Installing faraday 0.12.1  Fetching mime-types 3.1  Fetching nokogiri 1.7.1  Installing mime-types 3.1  Fetching pry 0.10.4  Installing pry 0.10.4  Fetching unf 0.1.4  Using wikisnakker 0.9.1 from https://github.com/everypolitician/wikisnakker.git (at master@72dbb8b)  Using sqlite_magic 0.0.6 from https://github.com/openc/sqlite_magic.git (at master@4df975e)  Using everypolitician 0.20.0 from https://github.com/everypolitician/everypolitician-ruby.git (at master@20582b1)  Fetching faraday_middleware 0.11.0.1  Installing unf 0.1.4  Installing faraday_middleware 0.11.0.1  Fetching rubocop 0.48.1  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching wikidata-client 0.0.10  Installing rubocop 0.48.1  Installing wikidata-client 0.0.10  Fetching domain_name 0.5.20170404  Installing domain_name 0.5.20170404  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching faraday-cookie_jar 0.0.6  Installing faraday-cookie_jar 0.0.6  Fetching rest-client 2.0.2  Installing rest-client 2.0.2  Fetching mediawiki_api 0.7.1  Installing mediawiki_api 0.7.1  Installing nokogiri 1.7.1 with native extensions  Using wikidata-fetcher 0.20.2 from https://github.com/everypolitician/wikidata-fetcher.git (at master@6cbd151)  Bundle complete! 14 Gemfile dependencies, 47 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (36.76s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P6634 for Q3379784 ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P4434 for Q3124250 ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P5409 for Q3045518 ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P4434 for Q960015 ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P4828 for Q960015 ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P3767 for Q2829989 ⁇ Unknown property: https://www.wikidata.org/wiki/Property:P3765 for Q2829989

Data

Downloaded 1212 times by everypolitician chrismytton MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (47 KB) Use the API

rows 10 / 73

id name gender identifier__VIAF identifier__BNF birth_date given_name identifier__sycomore identifier__senatfr image family_name death_date name__en identifier__freebase identifier__senatDOTfr name__fr wikipedia__fr wikipedia__en identifier__SUDOC name__nl name__es name__nb name__da name__nn name__sv name__de name__pl name__ko name__ca name__ms wikipedia__ca wikipedia__ko wikipedia__ms wikipedia__pl name__el wikipedia__el identifier__FAST name__oc wikipedia__oc name__it name__hu name__pt name__cy wikipedia__pt identifier__ISNI wikipedia__de identifier__EuroparlMEP wikipedia__es wikipedia__no original_wikiname name__zh_hant name__zh_cn twitter occupation identifier__LCAuth wikipedia__commons name__als name__simple name__nds name__lt name__li name__lb name__gsw name__ht wikipedia__ht identifier__assemblee_nationale identifier__ULAN identifier__whoswho_france name__ar identifier__GND name__zh name__ru name__sl facebook linkedin identifier__NTA wikipedia__ast name__ja name__ast website name__zh_hans name__pt_br wikipedia__ar wikipedia__fa name__fa
Q2993506
Cono Hamu
male
1941-08-15
Cono
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
Cono Hamu
politician
Cono Hamu
Cono Hamu
Q3287696
Marc-Kanyan Case
male
1942-09-14
Marc
Marc-Kanyan Case
Marc-Kanyan Case
Marc Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc-Kanyan Case
Marc Kanyan Case
association football player
Marc-Kanyan Case
Marc-Kanyan Case
مارک-کانیان کاز
مارک-کانیان کاز
Q3336545
Nathalie Brizard
female
1973-05-25
Nathalie
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
Nathalie Brizard
politician
Nathalie Brizard
Nathalie Brizard
Q3104894
Ghislaine Arlie
female
313546647
1961-05-03
Ghislaine
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
Ghislaine Arlie
politician
Ghislaine Arlie
Ghislaine Arlie
Q3380376
Philippe Michel
male
313541855
1960-01-10
Philippe
Michel
Philippe Michel
Philippe Michel
Philippe Michel (homme politique)
Philippe Michel
Philippe Michel
Philippe Michel
Philippe Michel
Philippe Michel
Philippe Michel
Philippe Michel (homme politique)
politician
فيليب ميشيل
Филипп Мишель
Philippe Michel
Philippe Michel
Q3292216
Marie-Pierre Goyetche
female
1956-02-12
Marie-Pierre
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Marie-Pierre Goyetche
trade unionist
Marie-Pierre Goyetche
Marie-Pierre Goyetche
Q27707396
François Burck
male
1939-06-04
François
2014-10-09
François Burck
François Burck
François Burck
François Burck
0000 0004 0339 7998
François Burck
politician
François Burck
Q28365481
Nina Julié
female
1979-12-08
Nina
Nina Julié
Nina Julié
Nina Julié
Nina Julié
Nina Julié
Nina Julié
Nina Julié
politician
Nina Julié
Nina Julié
Q3262404
Louis Kotra Uregei
male
1951-02-04
Louis
Louis Kotra Uregei
/m/06w25jh
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
Louis Kotra Uregei
politician
Louis Kotra Uregei
Louis Kotra Uregei
Q3437790
Rock Wamytan
male
1950-12-13
Roch
Roch Wamytan
Rock Wamytan
Rock Wamytan
Roch Wamytan
Roch Wamytan
Roch Wamytan
Roch Wamytan
Rock Wamytan
Rock Wamytan
Roch Wamytan
Roch Wamytan
Rock Wamytan
politician
Rock Wamytan

Statistics

Average successful run time: 1 minute

Total run time: 1 day

Total cpu time used: about 1 hour

Total disk space used: 81.4 KB

History

  • Auto ran revision a7c674d5 and completed successfully .
    73 records updated in the database
  • Auto ran revision a7c674d5 and completed successfully .
    70 records updated in the database
  • Auto ran revision a7c674d5 and completed successfully .
    73 records updated in the database
  • Auto ran revision a7c674d5 and completed successfully .
    73 records updated in the database
  • Auto ran revision a7c674d5 and completed successfully .
    71 records updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

new-caledonia-mps-wp / scraper.rb