everypolitician-scrapers / denmark-folketing

Scrapes github.com and www.thedanishparliament.dk

Build software better, together.


Contributors tmtmtmtm ondenman

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.1 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/..........  Fetching version metadata from https://rubygems.org/..  Fetching https://github.com/everypolitician/field_serializer.git  Fetching https://github.com/everypolitician/scraped_page_archive.git  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Installing ast 2.3.0  Installing public_suffix 2.0.4  Installing coderay 1.1.0  Using field_serializer 0.1.0 from https://github.com/everypolitician/field_serializer.git (at master@16d5d90)  Installing git 1.3.0  Installing colorize 0.7.7  Installing safe_yaml 1.0.4  Installing hashdiff 0.3.1  Installing method_source 0.8.2  Installing mini_portile 0.6.2  Installing httpclient 2.6.0.1  Installing open-uri-cached 0.0.5  Installing powerpack 0.1.1  Installing slop 3.6.0  Installing ruby-progressbar 1.8.1  Installing rainbow 2.1.0  Installing unicode-display_width 1.1.1  Using bundler 1.11.2  Installing sqlite3 1.3.10 with native extensions  Installing vcr 3.0.3  Installing parser 2.3.1.4  Installing addressable 2.5.0  Installing crack 0.4.3  Installing pry 0.10.1  Installing rubocop 0.43.0  Installing nokogiri 1.6.6.2 with native extensions  Installing webmock 2.0.3  Installing vcr-archive 0.3.0  Using scraped_page_archive 0.5.0 from https://github.com/everypolitician/scraped_page_archive.git (at master@28f93d7)  Installing sqlite_magic 0.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Bundle complete! 8 Gemfile dependencies, 31 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from webmock:  WebMock 2.0 has some breaking changes. Please check the CHANGELOG: https://goo.gl/piDGLu  Bundle completed (18.78s)  Cleaning up the bundler cache.  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 29 times by ondenman Protesuiq

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (203 KB) Use the API

rows 1 / 1

id name start_date
2015
2015–
2015

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (203 KB) Use the API

rows 10 / 189

id name given_name family_name party party_id constituency email homepage image memberships term source
sjebr
Jeppe Bruus
Jeppe
Bruus
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
Greater Copenhagen
jeppe.bruus@ft.dk
Member of Parliament for the Social Democratic Party in Greater Copenhagen greater constituency from January 1st 2014 to June 18th 2015.+++Temporary Member of Parliament for the Social Democratic Party in Greater Copenhagen greater constituency from July 4th 2015 to September 30th 2016 (substitute for Mogens Lykketoft).+++Temporary Member of Parliament for the Social Democratic Party in Greater Copenhagen greater constituency from November 5th 2013 to December 3rd 2013 (substitute for Sophie Hæstorp Andersen).+++Candidate for the Social Democratic Party in Gladsaxe nomination district from 2008.
2015
danny-malkowski
Danny K. Malkowski
Danny K.
Malkowski
Liberal Alliance
ACA585B8-A1C1-4440-A657-A3A00C2859E6
Copenhagen
danny.malkowski@ft.dk
Temporary Member of Parliament for Liberal Alliance in Copenhagen greater constituency from October 1st 2016 (substitute for Laura Lindahl).
2015
elheof
Helle Osther
Helle
Osther
The Red-Green Alliance
782DE1D5-0990-43C4-94E7-43FC6C381D8F
North Zealand
helle.friedrichsen@ft.dk
Temporary Member of Parliament for The Red-Green Alliance in North Zealand greater constituency from October 26th 2016 (substitute for Maria Reumert Gjerding).
2015
kfanjo
Anders Johansson
Anders
Johansson
The Conservative Party
AF65F1D8-0AFE-47DB-ADB1-7CA3C4550EC5
Funen
anders.johansson@ft.dk
Temporary Member of Parliament for the Conservative Party in Funen greater constituency from April 26th 2016 (substitute for Mai Mercado).+++Temporary Member of Parliament for the Conservative Party in Funen greater constituency from December 4th 2015 to January 19th 2016 (substitute for Mai Mercado).
2015
syiak
Yildiz Akdogan
Yildiz
Akdogan
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
Copenhagen
yildiz.akdogan@ft.dk
Member of Parliament for the Social Democratic Party in Copenhagen greater constituency from October 21st 2014.+++Member of Parliament for the Social Democratic Party in Copenhagen greater constituency from November 13th 2007 to September 15th 2011.+++Candidate for the Social Democratic Party in Brønshøj nomination district from 2007.+++Candidate for the Social Democratic Party in Husum nomination district from 2004 to 2006.
2015
schan
Christine Antorini
Christine
Antorini
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
North Zealand
christine.antorini@ft.dk
Member of Parliament for the Social Democratic Party in North Zealand greater constituency from September 15th 2011.+++Member of Parliament for the Social Democratic Party in Copenhagen greater constituency from November 13th 2007 to September 15th 2011.+++Member of Parliament for the Social Democratic Party in Southern Copenhagen constituency from February 8th 2005 to November 13th 2007.+++Member of Parliament for the Socialist People’s Party in Eastern Copenhagen constituency from March 18th 1998 to November 30th 1999.+++Candidate for the Social Democratic Party in Hillerød nomination district from 2010.+++Candidate for the Social Democratic Party in Inner city nomination district from 2007 to 2009.+++Candidate for the Social Democratic Party in Rådhus nomination district from 2004 to 2006.+++Candidate for the Socialist People’s Party in Brønshøj nomination district from 1996 to 1999.+++Candidate for the Socialist People’s Party in Vesterbro nomination district from 1992 to 1995.
2015
strbr
Trine Bramsen
Trine
Bramsen
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
Funen
trine.bramsen@ft.dk
Member of Parliament for the Social Democratic Party in Funen greater constituency from September 15th 2011.+++Candidate for the Social Democratic Party in Odense West nomination district from 2009.
2015
skibr
Kirsten Brosbøl
Kirsten
Brosbøl
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
East Jutland
kirsten.brosbol@ft.dk
http://www.kirstenbrosbøl.dk
Member of Parliament for the Social Democratic Party in East Jutland greater constituency from November 13th 2007.+++Member of Parliament for the Social Democratic Party in Århus County constituency from February 8th 2005 to November 13th 2007.+++Candidate for the Social Democratic Party in Skanderborg nomination district from 2004.
2015
smorb
Morten Bødskov
Morten
Bødskov
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
Greater Copenhagen
morten.bodskov@ft.dk
Member of Parliament for the Social Democratic Party in Greater Copenhagen greater constituency from November 13th 2007.+++Member of Parliament for the Social Democratic Party in Copenhagen County constituency from November 20th 2001 to November 13th 2007.+++Candidate for the Social Democratic Party in Rødovre nomination district from 1999.
2015
serch
Erik Christensen
Erik
Christensen
The Social Democratic Party
F9E65D32-6F2F-4762-A7D5-56DCD0FC9BAF
Funen
erik.christensen@ft.dk
Member of Parliament for the Social Democratic Party in Funen greater constituency from June 18th 2015.+++Candidate for the Social Democratic Party in Odense East nomination district from 2014.
2015

Statistics

Average successful run time: about 2 hours

Total run time: 6 days

Total cpu time used: 3 days

Total disk space used: 25.8 MB

History

  • Auto ran revision e7af0786 and completed successfully .
    185 records added, 185 records removed in the database
    802 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    185 records added, 185 records removed in the database
    800 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    185 records added, 185 records removed in the database
    800 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    185 records added, 185 records removed in the database
    800 pages scraped
  • Auto ran revision e7af0786 and failed .
    nothing changed in the database
  • Auto ran revision e7af0786 and completed successfully .
    185 records added, 185 records removed in the database
  • Auto ran revision e7af0786 and completed successfully .
    1 record added, 1 record removed in the database
    17 pages scraped
  • Auto ran revision e7af0786 and failed .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    185 records added, 185 records removed in the database
  • Auto ran revision e7af0786 and completed successfully .
    186 records added, 186 records removed in the database
    804 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    nothing changed in the database
    804 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    186 records added, 186 records removed in the database
    804 pages scraped
  • Auto ran revision e7af0786 and completed successfully .
    186 records added, 186 records removed in the database
    804 pages scraped
  • Auto ran revision 8d703c68 and failed .
    85 records added, 85 records removed in the database
    1082 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1204 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1225 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1204 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1202 pages scraped
  • Auto ran revision 8d703c68 and failed .
    88 records added, 88 records removed in the database
    566 pages scraped
  • Auto ran revision 8d703c68 and failed .
    109 records added, 109 records removed in the database
    684 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1204 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1202 pages scraped
  • Auto ran revision 8d703c68 and failed .
    nothing changed in the database
    556 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    2198 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    124 records added, 124 records removed in the database
    782 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    796 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    793 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    796 pages scraped
  • Auto ran revision 8d703c68 and failed .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    794 pages scraped
  • Auto ran revision 8d703c68 and failed .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 8d703c68 and failed .
    127 records added, 127 records removed in the database
    2002 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    187 records added, 186 records removed in the database
    1213 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1202 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 186 records removed in the database
    1202 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    nothing changed in the database
    1209 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    186 records added, 185 records removed in the database
    1202 pages scraped
  • Auto ran revision 8d703c68 and failed .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1198 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1200 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    2392 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    nothing changed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 185 records removed in the database
    1196 pages scraped
  • Auto ran revision 8d703c68 and failed .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    185 records added, 184 records removed in the database
    1214 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    184 records added, 183 records removed in the database
    1192 pages scraped
  • Auto ran revision 8d703c68 and completed successfully .
    184 records added, 183 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and failed .
    142 records added, 142 records removed in the database
    890 pages scraped
  • Auto ran revision 30780196 and failed .
    142 records added, 142 records removed in the database
    890 pages scraped
  • Auto ran revision 30780196 and failed .
    142 records added, 142 records removed in the database
    890 pages scraped
  • Auto ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and failed .
    39 records added, 39 records removed in the database
    242 pages scraped
  • Auto ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision 30780196 and failed .
    30 records added, 30 records removed in the database
    188 pages scraped
  • Manually ran revision 30780196 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision ca2370f4 and completed successfully .
    185 records added, 185 records removed in the database
    1190 pages scraped
  • Auto ran revision ca2370f4 and failed .
    11 records added, 11 records removed in the database
    1269 pages scraped
  • Auto ran revision ca2370f4 and completed successfully .
    185 records added, 116 records removed in the database
    1190 pages scraped
  • Auto ran revision ca2370f4 and failed .
    116 records added, 1 record removed in the database
    722 pages scraped
  • Auto ran revision ca2370f4 and completed successfully .
    nothing changed in the database
    1190 pages scraped
  • Manually ran revision ca2370f4 and failed .
    1 record added, 1 record removed in the database
    5 pages scraped
  • Manually ran revision ca2370f4 and failed .
    1 record added in the database
    5 pages scraped
  • Manually ran revision ca2370f4 and failed .
    nothing changed in the database
    6 pages scraped
  • Manually ran revision ca2370f4 and failed .
    nothing changed in the database
    7 pages scraped
  • Manually ran revision ca2370f4 and failed .
    nothing changed in the database
    60 pages scraped
  • Created on morph.io

Scraper code

Ruby

denmark-folketing / scraper.rb