everypolitician-scrapers / malaysian_parliament-wikidata

Scrapes api.morph.io, en.wikipedia.org, ms.wikipedia.org, and 2 other domains

Get structured data out of the web. Code collaboration through GitHub. Run your scrapers in the cloud.


Contributors tmtmtmtm ondenman

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.0.0 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/...  Fetching dependency metadata from https://rubygems.org/..  Fetching https://github.com/openc/sqlite_magic.git  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching https://github.com/everypolitician/wikidata-fetcher.git  Fetching https://github.com/everypolitician/wikisnakker  Rubygems 2.0.14.1 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation.  Installing coderay 1.1.0  Installing colorize 0.8.1  Installing diskcached 1.1.2  Installing unf_ext 0.0.7.2 with native extensions  Installing excon 0.53.0  Installing multipart-post 2.0.0  Installing hashie 3.4.6  Installing httpclient 2.8.2.4  Installing json 2.0.2 with native extensions  Installing method_source 0.8.2  Installing mime-types-data 3.2016.0521  Installing mini_portile2 2.1.0  Installing netrc 0.11.0  Installing pkg-config 1.1.7  Installing open-uri-cached 0.0.5  Installing slop 3.6.0  Installing sqlite3 1.3.11 with native extensions  Using wikisnakker 0.7.0 from https://github.com/everypolitician/wikisnakker (at master@cc55a4f)  Using bundler 1.11.2  Installing unf 0.1.4  Installing faraday 0.9.2  Installing mime-types 3.1  Installing nokogiri 1.6.8 with native extensions  Installing pry 0.10.3  Using sqlite_magic 0.0.6 from https://github.com/openc/sqlite_magic.git (at master@4df975e)  Installing domain_name 0.5.20160826  Installing faraday_middleware 0.10.0  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Installing http-cookie 1.0.3  Installing wikidata-client 0.0.10  Installing faraday-cookie_jar 0.0.6  Installing rest-client 2.0.0  Installing mediawiki_api 0.7.0  Using wikidata-fetcher 0.17.1 from https://github.com/everypolitician/wikidata-fetcher.git (at master@2ab5ce3)  Bundle complete! 10 Gemfile dependencies, 34 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (47.03s)  Cleaning up the bundler cache.  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Can't find Wikidata IDs for: Mohamed Said Keruak in en Can't find Wikidata IDs for: Mohamed Said Keruak in en Can't find Wikidata IDs for: Mohamed Said Keruak in en ‼ Q21005935 is_instance_of Wikimedia list article. Skipping ⁇ Unknown claim: https://www.wikidata.org/wiki/Property:P3373 for Q16867453 ‼ Q24949579 is_instance_of Wikimedia list article. Skipping ‼ Q20716099 is_instance_of Wikimedia list article. Skipping ⁇ Unknown claim: https://www.wikidata.org/wiki/Property:P3373 for Q5575520 ‼ Q6814485 is_instance_of Wikimedia list article. Skipping ⁇ Unknown claim: https://www.wikidata.org/wiki/Property:P3373 for Q57675 ‼ Q12857255 is_instance_of Wikimedia list article. Skipping

Data

Downloaded 50 times by everypolitician ondenman

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (467 KB) Use the API

rows 10 / 503

id name__en name__zh name__ms name__es name__id name__de name__nb name__da name__nn name__sv name__nl name__hu name wikipedia__en wikipedia__id wikipedia__ms wikipedia__zh gender birth_date identifier__freebase original_wikiname name__fr image given_name name__cy wikipedia__de identifier__VIAF identifier__GND identifier__LCAuth death_date name__zh_hans name__zh_hant name__zh_hk name__zh_cn name__zh_sg name__zh_tw name__jv name__pl name__eu name__qu name__ta name__et name__my name__pt name__eo name__is name__ru name__tr name__hak name__mk name__fi name__uk name__be_tarask name__ne name__az name__tl name__kk_arab name__kk_cn name__kk_tr name__kk_kz name__kk_cyrl name__kk_latn name__nan name__be name__ko name__he name__lv name__it name__li name__ja name__vi name__yue name__sh name__sa name__hi name__ro name__th name__ca name__war name__cs name__bg name__fa name__mr name__km name__kk name__sr name__ar name__ky name__new name__sco name__si name__la name__mg name__bcl name__lzh name__pnb name__wuu name__yi name__ur name__ba name__ka name__ml name__sd name__tt name__vo name__hy name__io name__lb name__lo name__mn name__te name__gu name__bn name__gl name__ast name__el wikipedia__ar wikipedia__arwikiquote wikipedia__ast wikipedia__az wikipedia__ba wikipedia__bcl wikipedia__be_x_old wikipedia__be wikipedia__bg wikipedia__bn wikipedia__ca wikipedia__commons wikipedia__cs wikipedia__cy wikipedia__da wikipedia__dewikiquote wikipedia__el wikipedia__enwikinews wikipedia__enwikiquote wikipedia__eo wikipedia__es wikipedia__et wikipedia__eu wikipedia__fa wikipedia__fi wikipedia__fr wikipedia__gu wikipedia__hak wikipedia__he wikipedia__hi wikipedia__hu wikipedia__hy wikipedia__io wikipedia__is wikipedia__it wikipedia__ja wikipedia__jv wikipedia__ka wikipedia__kk wikipedia__km wikipedia__ko wikipedia__ky wikipedia__la wikipedia__lb wikipedia__li wikipedia__lo wikipedia__lv wikipedia__mg wikipedia__mk wikipedia__ml wikipedia__mn wikipedia__mr wikipedia__my wikipedia__ne wikipedia__new wikipedia__nl wikipedia__nn wikipedia__nnwikiquote wikipedia__no wikipedia__pl wikipedia__plwikiquote wikipedia__pnb wikipedia__pt wikipedia__qu wikipedia__ro wikipedia__ru wikipedia__ruwikinews wikipedia__ruwikiquote wikipedia__sa wikipedia__sco wikipedia__sd wikipedia__sh wikipedia__simple wikipedia__sr wikipedia__sv wikipedia__ta wikipedia__te wikipedia__th wikipedia__thwikiquote wikipedia__tl wikipedia__tr wikipedia__tt wikipedia__uk wikipedia__ukwikiquote wikipedia__ur wikipedia__vi wikipedia__vo wikipedia__war wikipedia__wuu wikipedia__yi wikipedia__zh_classical wikipedia__zh_min_nan wikipedia__zh_yue wikipedia__zhwikiquote identifier__ISNI identifier__BNF identifier__SUDOC identifier__IMDB identifier__NDL family_name identifier__NNDB identifier__CANTIC identifier__Munzinger identifier__FAST identifier__CSPAN identifier__wikitree identifier__NLA identifier__NTA identifier__NLAtrove name__nds name__vmf name__en_ca name__en_gb twitter facebook name__min wikipedia__min website name__so name__ckb name__bs name__ace wikipedia__ace wikipedia__bswikiquote wikipedia__ckb wikipedia__itwikiquote wikipedia__so wikipedia__trwikiquote identifier__MusicBrainz identifier__GTAA name__yo name__lt name__oc wikipedia__eowikinews wikipedia__lt wikipedia__yo name__de_ch name__pt_br wikipedia__oc identifier__Oxforddnb identifier__bigenc_ru name__br name__co name__ty identifier__OLID wikipedia__gl name__zh_mo name__lmo name__pa wikipedia__pa name__sk name__zh_my identifier__NLI identifier__britishmuseum identifier__NPG occupation identifier__google_knowledge identifier__nyt_topic wikipedia__kn name__kn
Q4670089
Abu Bakar Taib
巴都卡阿布巴卡
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
Abu Bakar Taib
巴都卡阿布巴卡
male
1943-03-21
/m/09rvtdb
Abu Bakar Taib
politician
Q6893881
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
Mohd Effendi Norwawi
male
Mohd Effendi Norwawi
Mohd
politician
Q5088773
Che Rosli Che Mat
仄罗斯里
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
Che Rosli Che Mat
仄罗斯里
male
1951-07-16
Che Rosli Che Mat
Che Rosli Che Mat
politician
Q6894002
Mohd Nizar Zakaria
莫哈末尼扎
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
Mohd Nizar Zakaria
莫哈末尼扎
male
1969
Mohd Nizar Zakaria
Mohd
politician
Q7282661
Rafidah Aziz
拉菲达
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
Rafidah Aziz
拉菲达
female
1943-11-04
Rafidah Aziz
Rafidah Aziz
50993853
n97905191
റാഫിദ അസീസ്
Rafidah Aziz
റാഫിദ അസീസ്
411709
politician
Q7333551
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
female
Fatimah Abdul Majid
Fatimah Abdul Majid
Fatimah Abdul Majid
2013-04-12
politician
Q8054967
Yong Khoon Seng
杨昆贤
Yong Khoon Hian @ Yong Khoon Seng
Yong Khoon Seng
Yong Khoon Seng
Yong Khoon Hian @ Yong Khoon Seng
Yong Khoon Hian @ Yong Khoon Seng
Yong Khoon Hian @ Yong Khoon Seng
Yong Khoon Hian @ Yong Khoon Seng
Yong Khoon Seng
Yong Khoon Seng
Yong Khoon Seng
Yong Khoon Seng
Yong Khoon Hian @ Yong Khoon Seng
杨昆贤
male
1941-12-22
/m/0bm9qgk
Yong Khoon Seng
politician
Q6893994
Mohd Nasir Zakaria
莫哈末纳斯查卡利亚
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
Mohd Nasir Zakaria
莫哈末纳斯查卡利亚
male
1976-02-09
Mohd Nasir Zakaria
Mohd
politician
Q16193557
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
Azizah Mohd Dun
female
1960-11-04
Azizah Mohd Dun
Azizah Mohd Dun
Azizah
politician
Q12687564
Harun Pilus
Harun Pilus
Harun Pilus
Harun Pilus

Statistics

Average successful run time: 4 minutes

Total run time: about 8 hours

Total cpu time used: 40 minutes

Total disk space used: 512 KB

History

  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    544 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    544 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    503 records added in the database
    544 pages scraped
  • Auto ran revision 0bf94504 and failed .
    503 records removed in the database
    138 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    503 records updated in the database
    544 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    544 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
  • Auto ran revision 0bf94504 and failed .
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    544 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    1 record added, 502 records updated in the database
  • Auto ran revision 0bf94504 and failed .
    nothing changed in the database
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    543 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    543 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    543 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
  • Auto ran revision 0bf94504 and completed successfully .
    500 records updated in the database
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran and failed .
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    498 records updated in the database
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    498 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    500 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    543 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    nothing changed in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    499 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    501 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records updated in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and completed successfully .
    502 records added in the database
    542 pages scraped
  • Auto ran revision 0bf94504 and failed .
    502 records removed in the database
    185 pages scraped
  • Manually ran revision 0bf94504 and completed successfully .
    502 records added in the database
    542 pages scraped
  • Manually ran revision 0bf94504 and failed .
    nothing changed in the database
    1 page scraped
  • Manually ran revision 0bf94504 and failed .
    nothing changed in the database
    1 page scraped
  • Created on morph.io