TMMV / weatherforecastscraper

A scrapper that collects min and max temperatures of major european cities and their 7 day forecasts every day from the Met Office.


A scrapper that collects min and max temperatures of major european cities and their 7 day forecasts every day from the Met Office.

Contributors TMMV

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.10, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.10 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support  Collecting BeautifulSoup4==4.5.1  Downloading beautifulsoup4-4.5.1-py2-none-any.whl (83 kB)  Collecting requests>=2.20.0  Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)  Collecting wheel==0.26.  Downloading wheel-0.26.0-py2.py3-none-any.whl (63 kB)  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 4))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1  Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)  Collecting chardet<4,>=3.0.2  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)  Collecting idna<3,>=2.5  Downloading idna-2.9-py2.py3-none-any.whl (58 kB)  Collecting dumptruck>=0.1.2  Downloading dumptruck-0.1.6.tar.gz (15 kB)  Building wheels for collected packages: dumptruck  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Created wheel for dumptruck: filename=dumptruck-0.1.6-py2-none-any.whl size=11842 sha256=6ce13681539b9664a23295db9ec612fafcb9fd81e86d96e9f8731bf0de55bcc1  Stored in directory: /tmp/pip-ephem-wheel-cache-F1yGTL/wheels/dc/75/e9/1e61c4080c73e7bda99614549591f83b53bcc2d682f26fce62  Successfully built dumptruck  Installing collected packages: BeautifulSoup4, urllib3, certifi, chardet, idna, requests, wheel, dumptruck, scraperwiki  Attempting uninstall: wheel  Found existing installation: wheel 0.34.2  Uninstalling wheel-0.34.2:  Successfully uninstalled wheel-0.34.2  Running setup.py develop for scraperwiki  Successfully installed BeautifulSoup4-4.5.1 certifi-2020.4.5.1 chardet-3.0.4 dumptruck-0.1.6 idna-2.9 requests-2.23.0 scraperwiki urllib3-1.25.9 wheel-0.26.0 DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support    -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Bern Minsk Pristina Warsaw Bratislava Nicosia Oslo Stockholm Saint Helier Edinburgh Vaduz Monaco Sofia Athens Helsinki Chisinau Bucharest Vilnius Vienna Dublin London Luxembourg Zagreb Valletta Gibraltar Belgrade Tallinn Madrid Riga Ljubljana Tirana Copenhagen Skopje St. Peter Port Prague Reykjavik Rome Amsterdam Berlin Brussels Budapest Podgorica Moscow Lisbon Douglas Paris Sarajevo Andorra la Vella

Data

Downloaded 13 times by TMMV mathematixy

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.14 MB) Use the API

rows 0 / 0

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.14 MB) Use the API

rows 10 / 15984

day3_mintemp city day0_mintemp day6_mintemp day4_maxtemp day5_maxtemp day4_mintemp day3_maxtemp day1_maxtemp day1_mintemp day6_maxtemp day5_mintemp day2_mintemp day2_maxtemp date day0_maxtemp
6.01
Vaduz
2.61
8.48
19.15
19.12
6.88
16.17
14.72
4.27
19.21
8.18
8.21
15.72
2019-04-14
6.37
13.39
Gibraltar
12.49
13.23
19.56
19.01
11.65
17.35
23.1
10.79
19.72
12.26
14.93
21.95
2019-04-14
23.81
8.97
Dublin
6.13
8.36
13.09
15.97
7.83
11.76
9.44
7.61
17.01
7.63
6.24
10.84
2019-04-14
8.48
7.86
Douglas
5.4
8.14
13.29
13.43
8.61
12.03
10.29
7.41
12.87
8.63
6.6
9.06
2019-04-14
8.16
9.82
Madrid
12.7
8.7
14.7
15.2
8.22
24.82
23.7
7.53
17.39
8.67
9.06
20.1
2019-04-14
21.91
6.1
Zagreb
6.86
8.25
17.11
18.35
6.2
15.9
14.25
4.1
18.51
6.99
3.98
14.52
2019-04-14
9.25
3.49
Oslo
1.98
4.53
12.7
13.38
3.43
10.37
9.54
3.84
14.45
3.61
4.44
9.8
2019-04-14
9.63
8.63
Belgrade
8.66
9.36
18.67
18.9
9.4
15.96
15.88
7.18
19.58
8.6
6.15
16.16
2019-04-14
14.58
2.87
Riga
1.6
3.94
13.06
12.24
3
13.28
9.7
3.72
10.27
3.73
3.68
13.24
2019-04-14
7.51
5.34
Warsaw
1.74
6.31
15.19
16.66
4.88
14.82
9.28
0.2
16.77
5.81
3.57
9.69
2019-04-14
11.39

Statistics

Average successful run time: 2 minutes

Total run time: about 15 hours

Total cpu time used: about 2 hours

Total disk space used: 2.18 MB

History

  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

weatherforecastscraper / scraper.py