ar73m / weather

Ukrainian Hydrometeorological Center - http://meteo.gov.ua/

Scrapes meteo.gov.ua and www.parliament.go.ug

Погода в Украине и в мире. Точный прогноз погоды для Украины.Погодные информеры. Погодные карты, спутниковые снимки, температура воды. Новостные ленты о погоде в Украине и мире. Погода и здоровье


This is a scraper that runs on Morph. To get started see the documentation

Contributors ar73m

The scraper is running. It was queued manually .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-2.7.9  $ pip install -r requirements.txt  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki  Collecting lxml==3.4.4 (from -r requirements.txt (line 8))  Downloading lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1 (from -r requirements.txt (line 9))  Downloading cssselect-0.9.1.tar.gz  Collecting grab==0.6.21 (from -r requirements.txt (line 10))  Downloading grab-0.6.21.tar.gz (89kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 6))  Downloading dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r requirements.txt (line 6))  Downloading requests-2.12.4-py2.py3-none-any.whl (576kB)  Collecting pycurl (from grab==0.6.21->-r requirements.txt (line 10))  Downloading pycurl-7.43.0.tar.gz (182kB)  Collecting selection (from grab==0.6.21->-r requirements.txt (line 10))  Downloading selection-0.0.11.tar.gz  Collecting weblib>=0.1.10 (from grab==0.6.21->-r requirements.txt (line 10))  Downloading weblib-0.1.20.tar.gz  Collecting six (from grab==0.6.21->-r requirements.txt (line 10))  Downloading six-1.10.0-py2.py3-none-any.whl  Collecting pytils (from weblib>=0.1.10->grab==0.6.21->-r requirements.txt (line 10))  Downloading pytils-0.3.tar.gz (89kB)  Collecting user_agent (from weblib>=0.1.10->grab==0.6.21->-r requirements.txt (line 10))  Downloading user_agent-0.1.5.tar.gz  Installing collected packages: dumptruck, requests, scraperwiki, lxml, cssselect, pycurl, pytils, six, user-agent, weblib, selection, grab  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for cssselect: started  Running setup.py install for cssselect: finished with status 'done'  Running setup.py install for pycurl: started  Running setup.py install for pycurl: finished with status 'done'  Running setup.py install for pytils: started  Running setup.py install for pytils: finished with status 'done'  Running setup.py install for user-agent: started  Running setup.py install for user-agent: finished with status 'done'  Running setup.py install for weblib: started  Running setup.py install for weblib: finished with status 'done'  Running setup.py install for selection: started  Running setup.py install for selection: finished with status 'done'  Running setup.py install for grab: started  Running setup.py install for grab: finished with status 'done'  Successfully installed cssselect-0.9.1 dumptruck-0.1.6 grab-0.6.21 lxml-3.4.4 pycurl-7.43.0 pytils-0.3 requests-2.12.4 scraperwiki selection-0.0.11 six-1.10.0 user-agent-0.1.5 weblib-0.1.20   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Start 2016/12/26 16:13:20 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Start 2016/12/26 16:13:20 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Start 2016/12/26 16:13:20 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Start 2016/12/26 16:13:20 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Injecting configuration and compiling... Injecting scraper and running... Start 2017/01/15 12:23:35 Start 2017/01/15 12:23:35 Start 2017/01/15 12:23:35 Start 2017/01/15 12:23:35 Injecting configuration and compiling...  -----> Python app detected -----> Installing python-2.7.9  $ pip install -r requirements.txt  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki  Collecting lxml==3.4.4 (from -r requirements.txt (line 8))  Downloading lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1 (from -r requirements.txt (line 9))  Downloading cssselect-0.9.1.tar.gz  Collecting grab==0.6.21 (from -r requirements.txt (line 10))  Downloading grab-0.6.21.tar.gz (89kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 6))  Downloading dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r requirements.txt (line 6))  Downloading requests-2.12.4-py2.py3-none-any.whl (576kB)  Collecting pycurl (from grab==0.6.21->-r requirements.txt (line 10))  Downloading pycurl-7.43.0.tar.gz (182kB)  Collecting selection (from grab==0.6.21->-r requirements.txt (line 10))  Downloading selection-0.0.12.tar.gz  Collecting weblib>=0.1.10 (from grab==0.6.21->-r requirements.txt (line 10))  Downloading weblib-0.1.23.tar.gz  Collecting six (from grab==0.6.21->-r requirements.txt (line 10))  Downloading six-1.10.0-py2.py3-none-any.whl  Collecting pytils (from weblib>=0.1.10->grab==0.6.21->-r requirements.txt (line 10))  Downloading pytils-0.3.tar.gz (89kB)  Collecting user_agent (from weblib>=0.1.10->grab==0.6.21->-r requirements.txt (line 10))  Downloading user_agent-0.1.5.tar.gz  Installing collected packages: dumptruck, requests, scraperwiki, lxml, cssselect, pycurl, pytils, six, user-agent, weblib, selection, grab  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for cssselect: started  Running setup.py install for cssselect: finished with status 'done'  Running setup.py install for pycurl: started  Running setup.py install for pycurl: finished with status 'done'  Running setup.py install for pytils: started  Running setup.py install for pytils: finished with status 'done'  Running setup.py install for user-agent: started  Running setup.py install for user-agent: finished with status 'done'  Running setup.py install for weblib: started  Running setup.py install for weblib: finished with status 'done'  Running setup.py install for selection: started  Running setup.py install for selection: finished with status 'done'  Running setup.py install for grab: started  Running setup.py install for grab: finished with status 'done'  Successfully installed cssselect-0.9.1 dumptruck-0.1.6 grab-0.6.21 lxml-3.4.4 pycurl-7.43.0 pytils-0.3 requests-2.12.4 scraperwiki selection-0.0.12 six-1.10.0 user-agent-0.1.5 weblib-0.1.23   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Start 2017/01/16 00:37:03 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Start 2017/01/16 00:37:03 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Start 2017/01/16 00:37:03 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622 Start 2017/01/16 00:37:03 Error: [Errno 28] Operation timed out after 10001 milliseconds with 0 bytes received, URL: http://meteo.gov.ua/ua/34622

Statistics

Average successful run time: 5 minutes

Total run time: about 2 hours

Total cpu time used: 1 minute

Total disk space used: 290 KB

History

  • Manually ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    162 records added, 162 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    166 records added, 166 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    166 records added, 166 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    165 records added, 165 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    166 records added, 166 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    167 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    336 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    171 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    165 records added, 165 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    167 records added, 167 records removed in the database
    168 pages scraped
  • Auto ran revision a12f250a and completed successfully .
    165 records added, 165 records removed in the database
    168 pages scraped
  • Manually ran revision a12f250a and completed successfully .
    167 records added, 166 records removed in the database
    168 pages scraped
  • Manually ran revision a12f250a and completed successfully .
    166 records added, 166 records removed in the database
    168 pages scraped
  • Manually ran revision a12f250a and completed successfully .
    166 records added in the database
    168 pages scraped
  • Created on morph.io

Scraper code

Python

weather / scraper.py