TMMV / weatherforecastscraper

A scrapper that collects min and max temperatures of major european cities and their 7 day forecasts every day from the Met Office.


A scrapper that collects min and max temperatures of major european cities and their 7 day forecasts every day from the Met Office.

Contributors TMMV

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.10, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.10 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.  Collecting BeautifulSoup4==4.5.1 (from -r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/dc/27/8919b7a505b00f410c1c113c6c93b3253784375797976272e4198f4e455a/beautifulsoup4-4.5.1-py2-none-any.whl (83kB)  Collecting requests>=2.20.0 (from -r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)  Collecting wheel==0.26. (from -r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/d9/6f/cfb45d489703e4843b30c28e01ab39c87b0648bfa11a6c4a959f426fe217/wheel-0.26.0-py2.py3-none-any.whl (63kB)  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 4))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting certifi>=2017.4.17 (from requests>=2.20.0->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl (157kB)  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests>=2.20.0->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl (150kB)  Collecting idna<2.9,>=2.5 (from requests>=2.20.0->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.20.0->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Building wheels for collected packages: dumptruck  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Stored in directory: /tmp/pip-ephem-wheel-cache-9H1RNb/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e  Successfully built dumptruck  Installing collected packages: BeautifulSoup4, certifi, urllib3, idna, chardet, requests, wheel, dumptruck, scraperwiki  Found existing installation: wheel 0.33.4  Uninstalling wheel-0.33.4:  Successfully uninstalled wheel-0.33.4  Running setup.py develop for scraperwiki  Successfully installed BeautifulSoup4-4.5.1 certifi-2019.6.16 chardet-3.0.4 dumptruck-0.1.6 idna-2.8 requests-2.22.0 scraperwiki urllib3-1.25.3 wheel-0.26.0 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Warsaw Madrid Monaco Belgrade Moscow Stockholm Sarajevo Lisbon Budapest Rome Helsinki Minsk Valletta Tirana Bratislava Skopje Brussels Reykjavik Dublin Vilnius Athens Douglas Vienna Saint Helier Ljubljana Sofia Chisinau Amsterdam Edinburgh Riga Prague Tallinn Berlin Gibraltar Paris Oslo Luxembourg Pristina Copenhagen Nicosia Andorra la Vella Bucharest Zagreb Bern Podgorica Vaduz London St. Peter Port

Data

Downloaded 9 times by TMMV

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (390 KB) Use the API

rows 0 / 0

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (390 KB) Use the API

rows 10 / 2736

day3_mintemp city day0_mintemp day6_mintemp day4_maxtemp day5_maxtemp day4_mintemp day3_maxtemp day1_maxtemp day1_mintemp day6_maxtemp day5_mintemp day2_mintemp day2_maxtemp date day0_maxtemp
6.01
Vaduz
2.61
8.48
19.15
19.12
6.88
16.17
14.72
4.27
19.21
8.18
8.21
15.72
2019-04-14
6.37
13.39
Gibraltar
12.49
13.23
19.56
19.01
11.65
17.35
23.1
10.79
19.72
12.26
14.93
21.95
2019-04-14
23.81
8.97
Dublin
6.13
8.36
13.09
15.97
7.83
11.76
9.44
7.61
17.01
7.63
6.24
10.84
2019-04-14
8.48
7.86
Douglas
5.4
8.14
13.29
13.43
8.61
12.03
10.29
7.41
12.87
8.63
6.6
9.06
2019-04-14
8.16
9.82
Madrid
12.7
8.7
14.7
15.2
8.22
24.82
23.7
7.53
17.39
8.67
9.06
20.1
2019-04-14
21.91
6.1
Zagreb
6.86
8.25
17.11
18.35
6.2
15.9
14.25
4.1
18.51
6.99
3.98
14.52
2019-04-14
9.25
3.49
Oslo
1.98
4.53
12.7
13.38
3.43
10.37
9.54
3.84
14.45
3.61
4.44
9.8
2019-04-14
9.63
8.63
Belgrade
8.66
9.36
18.67
18.9
9.4
15.96
15.88
7.18
19.58
8.6
6.15
16.16
2019-04-14
14.58
2.87
Riga
1.6
3.94
13.06
12.24
3
13.28
9.7
3.72
10.27
3.73
3.68
13.24
2019-04-14
7.51
5.34
Warsaw
1.74
6.31
15.19
16.66
4.88
14.82
9.28
0.2
16.77
5.81
3.57
9.69
2019-04-14
11.39

Statistics

Average successful run time: 3 minutes

Total run time: about 3 hours

Total cpu time used: 21 minutes

Total disk space used: 432 KB

History

  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and failed .
    nothing changed in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • Auto ran revision f0695c94 and completed successfully .
    48 records added in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

weatherforecastscraper / scraper.py