loren / mca_data

MCA trade leads from dgMarket

Scrapes www.dgmarket.com

dgMarket - Tenders worldwide - dgMarket is the largest portal for tenders and consulting opportunities worldwide, with solicitations from national governments and international donor agencies


This is a scraper of MCA Trade Lead data from dgMarket that runs on Morph. It extracts country name, country code, deadline, and type from the multi-value attribute.

Contributors loren

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.2.5 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/...........Retrying dependency api due to error (2/4): Bundler::HTTPError Net::HTTPTooManyRequests: Retrying dependency api due to error (3/4): Bundler::HTTPError Net::HTTPTooManyRequests: .  Fetching version metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.11.2  Installing sqlite3 1.3.10 with native extensions  Installing httpclient 2.6.0.1  Installing sqlite_magic 0.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Bundle complete! 1 Gemfile dependency, 5 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (8.13s)  Cleaning up the bundler cache.  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 169 times by loren

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 KB) Use the API

rows 10 / 28

title link description pubDate guid categories country_code country_name
Sanitation Connection Action Plan (SCAP) Formative Research
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=6944664">MILLENNIUM CHALLENGE ACCOUNT ZAMBIA</a>
2016-11-22T21:03:11-05:00
type/spn,CPV/90900000,CPV/90000000,deadline/2016/Dec/09
ZM
Zambia
Carry out Work Targeting Water and Wastewater Sector in Zarqa Governorate
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=7080808">MILLENNIUM CHALLENGE ACCOUNT (MCA)</a>
2016-10-03T21:05:25-04:00
type/spn,CPV/45232100,CPV/45000000,deadline/2016/Dec/13
JO
Jordan
Asset Management Technical Assistance to Lusaka Water Sewerage Company Phase II - Implementation
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=6989172">MILLENNIUM CHALLENGE ACCOUNT - ZAMBIA LIMITED</a>
2016-11-05T13:11:56-04:00
type/spn,CPV/71356200,CPV/71800000,CPV/71000000,deadline/2016/Dec/05
ZM
Zambia
Construction management services
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=1277964">MCA-Benin II</a>
2016-11-24T12:05:50-05:00
type/spn,CPV/71540000,CPV/71700000,CPV/90700000,CPV/90500000,CPV/71530000,CPV/09310000,deadline/2017/Jan/31
BJ
Benin
General Procurement Notice N° 2 (Amendment 1)
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=1277964">MCA-Benin II</a>
2016-06-09T06:02:24-04:00
type/pp,CPV/09310000,CPV/51110000,CPV/48219300,CPV/48810000,CPV/48000000,CPV/48931000,CPV/79212200,CPV/79314000
BJ
Benin
General Procurement Notice N°2 (Amendment 3
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=1277964">MCA-Benin II</a>
2016-08-24T07:48:18-04:00
type/pp,CPV/09310000,CPV/51110000,CPV/48219300,CPV/48810000,CPV/48000000,CPV/48931000,CPV/79212200,CPV/79314000
BJ
Benin
Selection of a Program, Environmental and Social Management Consultant (PESMC)
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=1277964">MCA-Benin II</a>
2016-11-24T12:06:21-05:00
type/spn,CPV/71540000,CPV/71700000,CPV/90700000,CPV/90500000,CPV/71530000,CPV/09310000,deadline/2017/Jan/31
BJ
Benin
General Procurement Notice N°2 (Amendment 2)
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=1277964">MCA-Benin II</a>
2016-07-22T04:01:17-04:00
type/pp,CPV/09310000,CPV/51110000,CPV/48219300,CPV/48810000,CPV/48000000,CPV/48931000,CPV/79212200,CPV/79314000
BJ
Benin
Sanitation Connection Action Plan (SCAP) Formative Research
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=6614069">MCA-Zambia</a>
2016-11-17T09:31:33-05:00
type/spn,CPV/73200000,deadline/2016/Dec/09
ZM
Zambia
ASSET MANAGEMENT TECHNICAL ASSISTANCE TO LUSAKA WATER SEWERAGE COMPANY PHASE II - IMPLEMENTATION
Buyer: <a href="http://www.dgmarket.com/tenders/adminShowBuyer.do?buyerId=6614069">MCA-Zambia</a>
2016-10-25T04:28:34-04:00
type/spn,CPV/73200000,deadline/2016/Dec/05
ZM
Zambia

Statistics

Average successful run time: half a minute

Total run time: about 1 hour

Total cpu time used: 3 minutes

Total disk space used: 60.8 KB

History

  • Auto ran revision 639d14e0 and completed successfully .
    16 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 15 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records added, 24 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 24 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 19 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    12 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records removed, 25 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    5 records removed, 28 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    8 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    33 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 14 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 30 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 1 record updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records removed, 32 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 26 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 13 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 22 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    5 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records added, 20 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 23 records updated in the database
    2 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    9389 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 31 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    5 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    8 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 33 records updated in the database
    102 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 14 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    5 records added, 27 records updated in the database
    3 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    5 records added, 24 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    80 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 22 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 10 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 21 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 23 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 9 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    12 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 25 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    3 records removed, 22 records updated in the database
    3 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    12 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 14 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 3 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 5 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    24 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    29 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 30 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 28 records updated in the database
    7694 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 10 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 20 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    9 records removed, 25 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    14 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    3 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 17 records updated in the database
    2 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 28 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    6 records updated in the database
    37 pages scraped
  • Auto ran revision 639d14e0 and completed successfully .
    33 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    5 records updated in the database
    1 page scraped
  • Auto ran revision 639d14e0 and completed successfully .
    4 records added, 23 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 11 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 8 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 15 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 34 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 6 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    1 record added, 18 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    5 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 21 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    4 records removed, 30 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Manually ran revision 639d14e0 and completed successfully .
    37 records added in the database
  • Auto ran revision 639d14e0 and failed .
    37 records removed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    1 record removed, 21 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    15 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    18 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    22 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    7 records added, 30 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 13 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 25 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    27 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    3 records added in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    5 records removed, 24 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 18 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    5 records added, 26 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records added, 24 records updated in the database
  • Auto ran revision 639d14e0 and completed successfully .
    nothing changed in the database
  • Auto ran revision 639d14e0 and completed successfully .
    2 records removed, 20 records updated in the database
  • Manually ran revision 639d14e0 and completed successfully .
    26 records added in the database
  • Manually ran revision 26f62640 and failed .
    nothing changed in the database
  • Created on morph.io

Scraper code

Ruby

mca_data / scraper.rb