tmtmtmtm / syria-peoples-council

Scrapes github.com and parliament.gov.sy

Build software better, together.


This is a scraper that runs on Morph. To get started see the documentation

Contributors tmtmtmtm octopusinvitro

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Step 1/6 : FROM sha256:9532c9b47c81922f0a59f468c2b6095d1ae14bc576aed239aab85a6a54d5675b Step 2/6 : ADD app /app Step 3/6 : ENV CURL_TIMEOUT 180 Step 4/6 : ENV NPM_CONFIG_CAFILE /etc/ssl/certs/ca-certificates.crt Step 5/6 : ENV NODE_TLS_REJECT_UNAUTHORIZED 0 Step 6/6 : RUN /build/builder Injecting scraper and running... http://parliament.gov.sy/arabic/index.php?node=210&RID=1 http://parliament.gov.sy/arabic/index.php?node=210&RID=1 http://parliament.gov.sy/arabic/index.php?node=210&RID=1 http://parliament.gov.sy/arabic/index.php?node=210&RID=1 /app/vendor/bundle/ruby/2.3.0/gems/vcr-archive-0.3.0/lib/vcr/archive.rb:57:in `binwrite': No space left on device @ fptr_finalize - /tmp/d20170124-16-1wzv0po/parliament.gov.sy/3952fee51401205e99b0e0f1de8474449ccda6a9.yml (Errno::ENOSPC) from /app/vendor/bundle/ruby/2.3.0/gems/vcr-archive-0.3.0/lib/vcr/archive.rb:57:in `block in []=' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-archive-0.3.0/lib/vcr/archive.rb:51:in `each' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-archive-0.3.0/lib/vcr/archive.rb:51:in `[]=' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-3.0.3/lib/vcr/cassette.rb:266:in `write_recorded_interactions_to_disk' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-3.0.3/lib/vcr/cassette.rb:65:in `eject' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-3.0.3/lib/vcr.rb:156:in `eject_cassette' from /app/vendor/bundle/ruby/2.3.0/gems/vcr-3.0.3/lib/vcr.rb:191:in `use_cassette' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-28f93d74b1c1/lib/scraped_page_archive.rb:36:in `record' from /app/vendor/bundle/ruby/2.3.0/bundler/gems/scraped_page_archive-28f93d74b1c1/lib/scraped_page_archive/open-uri.rb:9:in `open_uri' from /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/open-uri.rb:717:in `open' from /app/vendor/ruby-2.3.3/lib/ruby/2.3.0/open-uri.rb:31:in `open' from scraper.rb:19:in `noko_for' from scraper.rb:45:in `scrape_person' from scraper.rb:39:in `block in scrape_list' from /app/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.7.0.1/lib/nokogiri/xml/node_set.rb:187:in `block in each' from /app/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.7.0.1/lib/nokogiri/xml/node_set.rb:186:in `upto' from /app/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.7.0.1/lib/nokogiri/xml/node_set.rb:186:in `each' from scraper.rb:25:in `scrape_list' from scraper.rb:58:in `<main>'

Data

Downloaded 38 times by everypolitician ondenman MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (186 KB) Use the API

rows 10 / 13

id name area category party term deceased source birth_date email image
144
آل حسن بري
حلب
(ب) قطاع باقي فئات الشعب
مستقل
2012
1972-01-01
255
أحمد ابراهيم
مناطق محافظة حلب
(أ) قطاع عمال وفلاحين
حزب البعث العربي الاشتراكي
2012
1960-01-01
109
أحمد الحميدي
الرقة
(ب) قطاع باقي فئات الشعب
حزب البعث العربي الاشتراكي
2012
1964-01-01
256
أحمد العلي الحمرة
مناطق محافظة حلب
(أ) قطاع عمال وفلاحين
مستقل
2012
1977-01-01
78
أحمد الفرج
إدلب
(أ) قطاع عمال وفلاحين
حزب البعث العربي الاشتراكي
2012
1967-01-01
79
أحمد المبارك
إدلب
(أ) قطاع عمال وفلاحين
حزب البعث العربي الاشتراكي
2012
1959-01-01
143
أحمد جريخ
حلب
(أ) قطاع عمال وفلاحين
مستقل
2012
1948-01-01
229
أحمد كزبري
دمشق
(ب) قطاع باقي فئات الشعب
مستقل
2012
1971-01-01
ahmad@alkezbari.com
77
أحمد الحسن الهلال
إدلب
(أ) قطاع عمال وفلاحين
حزب البعث العربي الاشتراكي
2012
1951-01-01
80
أحمد ديب قباني
إدلب
(أ) قطاع عمال وفلاحين
حزب البعث العربي الاشتراكي
2012
1962-01-01

Statistics

Average successful run time: about 5 hours

Total run time: 1 day

Total cpu time used: about 21 hours

Total disk space used: 38.1 MB

History

  • Manually ran revision 9ce08c1b and failed .
    426 records removed in the database
    59 pages scraped
  • Auto ran and failed .
  • Auto ran revision 9ce08c1b and failed .
    nothing changed in the database
  • Auto ran revision 9ce08c1b and failed .
    437 records added in the database
    1767 pages scraped
  • Auto ran revision 9ce08c1b and failed .
    47 records removed in the database
    15 pages scraped
  • Auto ran revision 9ce08c1b and failed .
    nothing changed in the database
    142 pages scraped
  • Auto ran revision b0028b6c and failed .
    10 records added in the database
    203 pages scraped
  • Auto ran revision b0028b6c and failed .
    5 records removed in the database
    163 pages scraped
  • Auto ran revision b0028b6c and failed .
    11 records added in the database
    183 pages scraped
  • Auto ran revision b0028b6c and failed .
    476 records removed in the database
    139 pages scraped
  • Auto ran revision b0028b6c and completed successfully .
    nothing changed in the database
    2046 pages scraped
  • Auto ran revision b0028b6c and completed successfully .
    197 records added in the database
    2046 pages scraped
  • Auto ran revision b0028b6c and failed .
    38 records added, 274 records updated in the database
    1259 pages scraped
  • Auto ran revision b0028b6c and failed .
    nothing changed in the database
  • Manually ran revision ff851862 and failed .
    nothing changed in the database
  • Manually ran revision ff851862 and failed .
    nothing changed in the database
  • Manually ran and failed .
  • Manually ran revision 8c9d1df1 and completed successfully .
    274 records added in the database
    277 pages scraped
  • Created on morph.io

Scraper code

Ruby

syria-peoples-council / scraper.rb