This is a scraper that runs on Morph. To get started see the documentation
Contributors
Last run failed
with status code 1.
Console output of last run
Injecting configuration and compiling...
[1G [1G-----> Python app detected
[1G ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).
[1G ! We recommend upgrading by specifying the latest version (python-2.7.14).
[1G Learn More: https://devcenter.heroku.com/articles/python-runtimes
[1G-----> Installing python-2.7.9
[1G-----> Installing pip
[1G-----> Installing requirements with pip
[1G DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
[1G Collecting beautifulsoup4==4.2.0
[1G Downloading beautifulsoup4-4.2.0.tar.gz (63 kB)
[1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 4))
[1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki
[1G Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki
[1G Running command git checkout -b morph_defaults --track origin/morph_defaults
[1G Switched to a new branch 'morph_defaults'
[1G Branch morph_defaults set up to track remote branch morph_defaults from origin.
[1G Collecting dumptruck>=0.1.2
[1G Downloading dumptruck-0.1.6.tar.gz (15 kB)
[1G Collecting requests
[1G Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
[1G Collecting chardet<5,>=3.0.2
[1G Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
[1G Collecting certifi>=2017.4.17
[1G Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
[1G Collecting urllib3<1.27,>=1.21.1
[1G Downloading urllib3-1.26.4-py2.py3-none-any.whl (153 kB)
[1G Collecting idna<3,>=2.5
[1G Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
[1G Building wheels for collected packages: beautifulsoup4, dumptruck
[1G Building wheel for beautifulsoup4 (setup.py): started
[1G Building wheel for beautifulsoup4 (setup.py): finished with status 'done'
[1G Created wheel for beautifulsoup4: filename=beautifulsoup4-4.2.0-py2-none-any.whl size=71804 sha256=dae9eee95ff50a71662126f7334984967e37fe56aae93027a2ae3b50c2e6f341
[1G Stored in directory: /tmp/pip-ephem-wheel-cache-kI4uie/wheels/dc/69/36/2edde9ec10080447fc56d1a3d1235ddba3dd4b0dc9fff29134
[1G Building wheel for dumptruck (setup.py): started
[1G Building wheel for dumptruck (setup.py): finished with status 'done'
[1G Created wheel for dumptruck: filename=dumptruck-0.1.6-py2-none-any.whl size=11844 sha256=d0414c803ff4c206e34992068c41b2ffbf74a0979da8cfff944996fcf864a499
[1G Stored in directory: /tmp/pip-ephem-wheel-cache-kI4uie/wheels/dc/75/e9/1e61c4080c73e7bda99614549591f83b53bcc2d682f26fce62
[1G Successfully built beautifulsoup4 dumptruck
[1G Installing collected packages: beautifulsoup4, dumptruck, chardet, certifi, urllib3, idna, requests, scraperwiki
[1G Running setup.py develop for scraperwiki
[1G Successfully installed beautifulsoup4-4.2.0 certifi-2020.12.5 chardet-4.0.0 dumptruck-0.1.6 idna-2.10 requests-2.25.1 scraperwiki urllib3-1.26.4
[91mDEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
[0m
[1G
[1G [1G-----> Discovering process types
[1G Procfile declares types -> scraper
Injecting scraper and running...
Traceback (most recent call last):
File "scraper.py", line 107, in <module>
url = link['href']
File "/app/.heroku/python/lib/python2.7/site-packages/bs4/element.py", line 892, in __getitem__
return self.attrs[key]
KeyError: 'href'
Data
Downloaded
821 times
by
To download data sign in with GitHub
rows 10 / 122
| f | d | l |
|---|---|---|
|
E3520_SCC_gov_2015_08
|
2015-11-05 07:30:44.393847
|
|
|
E3520_SCC_gov_2015_08
|
2015-11-05 07:30:51.944453
|
|
|
E3520_SCC_gov_2016_01
|
2016-04-04 01:26:46.860070
|
|
|
E3520_SCC_gov_2015_12
|
2016-04-04 01:26:57.732484
|
|
|
E3520_SCC_gov_2015_11
|
2016-04-04 01:27:04.470437
|
|
|
E3520_SCC_gov_2015_10
|
2016-04-04 01:27:09.283733
|
|
|
E3520_SCC_gov_2015_09
|
2016-04-04 01:27:18.096753
|
|
|
E3520_SCC_gov_2015_08
|
2016-04-04 01:27:22.096502
|
|
|
E3520_SCC_gov_2016_07
|
2016-12-04 15:33:29.190942
|
|
|
E3520_SCC_gov_2013_12
|
2017-05-22 05:01:11.189042
|
Statistics
Average successful run time: 3 minutes
Total run time: 4 days
Total cpu time used: 21 minutes
Total disk space used: 86.2 KB
History
-
Auto
ran
revision
75dd0b31
and
failed
.
nothing changed in the database
-
Auto
ran
revision
75dd0b31
and
failed
.
nothing changed in the database
-
Auto
ran
revision
75dd0b31
and
failed
.
nothing changed in the database
-
Auto
ran
revision
75dd0b31
and
failed
.
nothing changed in the database
-
Auto
ran
revision
75dd0b31
and
failed
.
nothing changed in the database
- ...
- Created on morph.io