struan / spain_congreso_es

Details of members of the Spanish Congress from the official website congreso.es

Scrapes www.congreso.es and github.com

Congreso de los Diputados


This is a scraper that runs on Morph. To get started see the documentation

Contributors struan tmtmtmtm octopusinvitro chrismytton

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby/Rack -----> Using Ruby version: ruby-2.3.1 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/everypolitician/scraped_page_archive.git  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching addressable 2.4.0  Using bundler 1.15.2  Fetching mime-types-data 3.2016.0221  Fetching mini_portile2 2.0.0  Installing mini_portile2 2.0.0  Installing addressable 2.4.0  Fetching rack 1.6.4  Installing mime-types-data 3.2016.0221  Fetching cliver 0.3.2  Fetching coderay 1.1.1  Installing coderay 1.1.1  Installing cliver 0.3.2  Fetching colorize 0.7.7  Installing colorize 0.7.7  Fetching safe_yaml 1.0.4  Fetching git 1.3.0  Installing safe_yaml 1.0.4  Installing rack 1.6.4  Installing git 1.3.0  Fetching hashdiff 0.3.0  Fetching httpclient 2.7.1  Installing hashdiff 0.3.0  Fetching method_source 0.8.2  Installing method_source 0.8.2  Fetching multi_json 1.11.2  Installing httpclient 2.7.1  Installing multi_json 1.11.2  Fetching open-uri-cached 0.0.5  Installing open-uri-cached 0.0.5  Fetching websocket-extensions 0.1.2  Installing websocket-extensions 0.1.2  Fetching slop 3.6.0  Fetching vcr 3.0.3  Installing slop 3.6.0  Fetching sqlite3 1.3.11  Installing sqlite3 1.3.11 with native extensions  Installing vcr 3.0.3  Fetching nokogiri 1.6.7.2  Fetching mime-types 3.0  Installing mime-types 3.0  Fetching crack 0.4.3  Installing crack 0.4.3  Installing nokogiri 1.6.7.2 with native extensions  Fetching websocket-driver 0.6.3  Installing websocket-driver 0.6.3 with native extensions  Fetching rack-test 0.6.3  Installing rack-test 0.6.3  Fetching pry 0.10.3  Installing pry 0.10.3  Fetching webmock 2.0.3  Installing webmock 2.0.3  Fetching sqlite_magic 0.0.6  Fetching vcr-archive 0.3.0  Installing vcr-archive 0.3.0  Installing sqlite_magic 0.0.6  Using scraped_page_archive 0.4.1 from https://github.com/everypolitician/scraped_page_archive.git (at master@9d8a634)  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching xpath 2.0.0  Installing xpath 2.0.0  Fetching capybara 2.6.2  Installing capybara 2.6.2  Fetching poltergeist 1.9.0  Installing poltergeist 1.9.0  Bundle complete! 8 Gemfile dependencies, 33 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from webmock:  WebMock 2.0 has some breaking changes. Please check the CHANGELOG: https://goo.gl/piDGLu  Bundle completed (18.74s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... including Capybara::DSL in the global scope is not recommended! including Capybara::DSL in the global scope is not recommended! /app/vendor/bundle/ruby/2.3.0/gems/poltergeist-1.9.0/lib/capybara/poltergeist/server.rb:38:in `send': PhantomJS client died while processing {"id":"07c03100-1ce9-4da1-851b-506872652f07","name":"body","args":[]} (Capybara::Poltergeist::DeadClient) from /app/vendor/bundle/ruby/2.3.0/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:344:in `command' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-9d8a6347d122/lib/scraped_page_archive/capybara.rb:53:in `command' from /app/vendor/bundle/ruby/2.3.0/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:46:in `body' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-9d8a6347d122/lib/scraped_page_archive/capybara.rb:58:in `block in command' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-3.0.3/lib/vcr/util/variable_args_block_caller.rb:9:in `call_block' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-3.0.3/lib/vcr.rb:189:in `use_cassette' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-9d8a6347d122/lib/scraped_page_archive.rb:36:in `record' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-9d8a6347d122/lib/scraped_page_archive.rb:18:in `record' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-9d8a6347d122/lib/scraped_page_archive/capybara.rb:57:in `command' from /app/vendor/bundle/ruby/2.3.0/gems/poltergeist-1.9.0/lib/capybara/poltergeist/browser.rb:75:in `visible_text' from /app/vendor/bundle/ruby/2.3.0/gems/poltergeist-1.9.0/lib/capybara/poltergeist/node.rb:17:in `command' from /app/vendor/bundle/ruby/2.3.0/gems/poltergeist-1.9.0/lib/capybara/poltergeist/node.rb:50:in `visible_text' from /app/vendor/bundle/ruby/2.3.0/gems/capybara-2.6.2/lib/capybara/node/element.rb:61:in `block in text' from /app/vendor/bundle/ruby/2.3.0/gems/capybara-2.6.2/lib/capybara/node/base.rb:84:in `synchronize' from /app/vendor/bundle/ruby/2.3.0/gems/capybara-2.6.2/lib/capybara/node/element.rb:57:in `text' from scraper.rb:145:in `block in scrape_people' from scraper.rb:144:in `scrape_people' from scraper.rb:289:in `<main>'

Data

Downloaded 1673 times by everypolitician tmtmtmtm octopusinvitro struan MikeRalphson lpszBar chrismytton polmeth

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3.58 MB) Use the API

rows 10 / 5108

id name term iddiputado url
0
XII Legislatura
11
167
0
XI Legislatura
10
407
0
VII Legislatura
6
106
0
VI Legislatura
5
397
0
XII Legislatura
11
170
0
XI Legislatura
10
110
0
X Legislatura
9
6
0
IX Legislatura
8
129
0
VIII Legislatura
7
117
0
VI Legislatura
5
320

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3.58 MB) Use the API

rows 10 / 13

id name start_date end_date source
12
XII Legislatura
2016
11
XI Legislatura
2016
2016
10
X Legislatura
2011
2016
9
IX Legislatura
2008
2011
3
III Legislatura
1986
1989
2
II Legislatura
1982
1986
8
VIII Legislatura
2004
2008
7
VII Legislatura
2000
2004
6
VI Legislatura
1996
2000
5
V Legislatura
1993
1996

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3.58 MB) Use the API

rows 10 / 4971

iddiputado name sort_name given_name family_name gender party faction_id faction source dob term start_date end_date email twitter facebook phone fax constituency photo id
44
José Manuel García-Margallo y Marfil
García-Margallo y Marfil, José Manuel
José Manuel
García-Margallo y Marfil
male
PP
GP
G.P. Popular
Nacido el 13 de agosto de 1944 .
11
2015-12-30
Alicante
11_44
111
Ángel Martínez Sanjuán
Martínez Sanjuán, Ángel
Ángel
Martínez Sanjuán
male
PSOE
GS
G.P. Socialista
Nacido el 8 de abril de 1951 en San Sebastián .
8
2004-03-25
2008-01-15
La Rioja
2_159
407
Alejandro de la Mata Menéndez
Mata Menéndez, Alejandro de la
Alejandro de la
Mata Menéndez
male
PP
GP
G.P. Popular
Nacido el 28 de julio de 1945 en Blimes (Asturias) .
9
2011-06-21
2011-09-27
Zaragoza
9_407
41
Carmen Matador de Matos
Matador de Matos, Carmen
Carmen
Matador de Matos
female
PP
GP
G.P. Popular
Nacida el 27 de febrero de 1969 en Badajoz .
8
2004-03-23
2008-01-15
Badajoz
6_364
380
Jorge Juan Mataix Hidalgo
Mataix Hidalgo, Jorge Juan
Jorge Juan
Mataix Hidalgo
male
CDS
GCDS
Grupo Parlamentario CDS
Diputado de la III legislatura.
3
1988-06-14
1989-09-02
Barcelona
3_380
33
José Ramón Mateos Martín
Mateos Martín, José Ramón
José Ramón
Mateos Martín
male
PSOE
GS
G.P. Socialista
Nacido el 30 de mayo de 1975 en Madrid .
8
2004-03-23
2008-01-15
Illes Balears
8_33
370
Rafael Mateos Yuste
Mateos Yuste, Rafael
Rafael
Mateos Yuste
male
PP
GP
G.P. Popular
Nacido el 22 de junio de 1947 en Navalmoral de la Mata (Cáceres) .
7
2000-07-25
2004-01-20
Cáceres
7_370
118
Ana Mato Adrover
Mato Adrover, Ana
Ana
Mato Adrover
female
PP
GP
G.P. Popular
Nacida el 24 de septiembre de 1959 en Madrid .
10
2011-11-30
2016-01-13
Madrid
5_115
51
Gabriel Mato Adrover
Mato Adrover, Gabriel
Gabriel
Mato Adrover
male
PP
GP
G.P. Popular
Nacido el 29 de abril de 1961 en Madrid .
9
2008-03-24
2009-07-13
Santa Cruz de Tenerife
7_272
238
José Antonio Maturana Plaza
Maturana Plaza, José Antonio
José Antonio
Maturana Plaza
male
PSE-PSOE
GSV
Grupo Parlamentario Socialista Vasco
Diputado de la Constituyente y I legislaturas.
1
1979-03-15
1980-03-26
Guipúzcoa
0_161

Statistics

Average successful run time: about 2 hours

Total run time: about 2 months

Total cpu time used: 1 day

Total disk space used: 109 MB

History

  • Auto ran revision 0c9b0942 and failed .
    nothing changed in the database
    16 pages scraped
  • Auto ran revision 0c9b0942 and failed .
    nothing changed in the database
    14 pages scraped
  • Auto ran revision 0c9b0942 and failed .
    nothing changed in the database
    16 pages scraped
  • Auto ran revision 0c9b0942 and failed .
    nothing changed in the database
    16 pages scraped
  • Auto ran revision 0c9b0942 and failed .
    nothing changed in the database
    16 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

spain_congreso_es / scraper.rb