This is a scraper that runs on Morph. To get started see the documentation
Contributors
Last run completed successfully
.
Console output of last run
Injecting configuration and compiling...
[1G [1G-----> Python app detected
[1G ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).
[1G ! We recommend upgrading by specifying the latest version (python-2.7.14).
[1G Learn More: https://devcenter.heroku.com/articles/python-runtimes
[1G-----> Installing python-2.7.9
[1G-----> Installing pip
[1G-----> Installing requirements with pip
[1G DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
[1G Collecting beautifulsoup4==4.2.0
[1G Downloading https://files.pythonhosted.org/packages/b3/fe/6888f755c0b9f66d4cc1f9ee0077a613a09e211acca3242a7335ffe7ed06/beautifulsoup4-4.2.0.tar.gz (63kB)
[1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 10))
[1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki
[1G Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki
[1G Running command git checkout -b morph_defaults --track origin/morph_defaults
[1G Switched to a new branch 'morph_defaults'
[1G Branch morph_defaults set up to track remote branch morph_defaults from origin.
[1G Collecting lxml==3.4.4
[1G Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)
[1G Collecting cssselect==0.9.1
[1G Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz
[1G Collecting dumptruck>=0.1.2
[1G Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz
[1G Collecting requests
[1G Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)
[1G Collecting certifi>=2017.4.17
[1G Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
[1G Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
[1G Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB)
[1G Collecting idna<2.9,>=2.5
[1G Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
[1G Collecting chardet<3.1.0,>=3.0.2
[1G Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
[1G Building wheels for collected packages: beautifulsoup4, lxml, cssselect, dumptruck
[1G Building wheel for beautifulsoup4 (setup.py): started
[1G Building wheel for beautifulsoup4 (setup.py): finished with status 'done'
[1G Created wheel for beautifulsoup4: filename=beautifulsoup4-4.2.0-cp27-none-any.whl size=71806 sha256=03f3b7288b614d8234ba7b67b3565be5ee5cf3e05e4f7a1d560f58368d288528
[1G Stored in directory: /tmp/pip-ephem-wheel-cache-KZBSwA/wheels/3b/fb/48/5009f96b53a4d41ebb53c3e5598204dd3032b8db0110526e88
[1G Building wheel for lxml (setup.py): started
[1G Building wheel for lxml (setup.py): still running...
[1G Building wheel for lxml (setup.py): finished with status 'done'
[1G Created wheel for lxml: filename=lxml-3.4.4-cp27-cp27m-linux_x86_64.whl size=2989869 sha256=727866c2707a47346ebf38aab206d6d87f1f461be683a98634bc401dd686527e
[1G Stored in directory: /tmp/pip-ephem-wheel-cache-KZBSwA/wheels/f6/df/7b/af9cace9baf95a6e4a2b5790e30da55fc780ddee598314d1ed
[1G Building wheel for cssselect (setup.py): started
[1G Building wheel for cssselect (setup.py): finished with status 'done'
[1G Created wheel for cssselect: filename=cssselect-0.9.1-cp27-none-any.whl size=26994 sha256=0ae773d7f2b2a727fe9ebd65234d336fd3e3abcf5b1c2a6d90c4594f6f274a3e
[1G Stored in directory: /tmp/pip-ephem-wheel-cache-KZBSwA/wheels/45/25/d7/5a3b06d22b1ffb616f868a74729a5a002bcc04d45109b4f223
[1G Building wheel for dumptruck (setup.py): started
[1G Building wheel for dumptruck (setup.py): finished with status 'done'
[1G Created wheel for dumptruck: filename=dumptruck-0.1.6-cp27-none-any.whl size=11845 sha256=d4cc98eefa3b9180216a456bb53a48ab7ebd63de31a6b6589852cde67275f8b8
[1G Stored in directory: /tmp/pip-ephem-wheel-cache-KZBSwA/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e
[1G Successfully built beautifulsoup4 lxml cssselect dumptruck
[1G Installing collected packages: beautifulsoup4, dumptruck, certifi, urllib3, idna, chardet, requests, scraperwiki, lxml, cssselect
[1G Running setup.py develop for scraperwiki
[1G Successfully installed beautifulsoup4-4.2.0 certifi-2019.11.28 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 idna-2.8 lxml-3.4.4 requests-2.22.0 scraperwiki urllib3-1.25.7
[91mDEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
[0m
[1G
[1G [1G-----> Discovering process types
[1G Procfile declares types -> scraper
Injecting scraper and running...
E1821_WCC_gov_2019_11
E1821_WCC_gov_2019_10
E1821_WCC_gov_2019_09
E1821_WCC_gov_2019_08
E1821_WCC_gov_2019_07
E1821_WCC_gov_2019_06
E1821_WCC_gov_2019_05
E1821_WCC_gov_2019_04
E1821_WCC_gov_2019_03
E1821_WCC_gov_2019_02
E1821_WCC_gov_2019_01
E1821_WCC_gov_2018_12
E1821_WCC_gov_2018_11
E1821_WCC_gov_2018_10
E1821_WCC_gov_2018_09
E1821_WCC_gov_2018_08
E1821_WCC_gov_2018_07
E1821_WCC_gov_2018_06
E1821_WCC_gov_2018_05
E1821_WCC_gov_2018_04
E1821_WCC_gov_2018_03
E1821_WCC_gov_2018_02
E1821_WCC_gov_2018_01
E1821_WCC_gov_2017_12
E1821_WCC_gov_2017_11
E1821_WCC_gov_2017_10
E1821_WCC_gov_2017_09
E1821_WCC_gov_2017_08
E1821_WCC_gov_2017_07
E1821_WCC_gov_2017_06
E1821_WCC_gov_2017_05
E1821_WCC_gov_2017_03
E1821_WCC_gov_2017_04
E1821_WCC_gov_2017_02
E1821_WCC_gov_2017_01
E1821_WCC_gov_2016_12
E1821_WCC_gov_2016_11
E1821_WCC_gov_2016_10
E1821_WCC_gov_2016_09
E1821_WCC_gov_2016_08
E1821_WCC_gov_2016_07
E1821_WCC_gov_2016_06
E1821_WCC_gov_2016_05
E1821_WCC_gov_2016_04
E1821_WCC_gov_2016_03
E1821_WCC_gov_2016_02
E1821_WCC_gov_2016_01
E1821_WCC_gov_2015_12
E1821_WCC_gov_2015_11
E1821_WCC_gov_2015_10
E1821_WCC_gov_2015_09
E1821_WCC_gov_2015_08
E1821_WCC_gov_2015_07
E1821_WCC_gov_2015_06
E1821_WCC_gov_2015_05
E1821_WCC_gov_2015_04
E1821_WCC_gov_2015_03
E1821_WCC_gov_2015_02
E1821_WCC_gov_2015_01
E1821_WCC_gov_2014_12
E1821_WCC_gov_2014_11
E1821_WCC_gov_2014_10
E1821_WCC_gov_2014_09
E1821_WCC_gov_2014_08
E1821_WCC_gov_2014_07
E1821_WCC_gov_2014_06
E1821_WCC_gov_2014_05
E1821_WCC_gov_2014_04
E1821_WCC_gov_2014_03
E1821_WCC_gov_2014_02
E1821_WCC_gov_2014_01
E1821_WCC_gov_2013_12
E1821_WCC_gov_2013_11
E1821_WCC_gov_2013_10
E1821_WCC_gov_2013_09
E1821_WCC_gov_2013_08
E1821_WCC_gov_2013_07
E1821_WCC_gov_2013_06
E1821_WCC_gov_2013_05
E1821_WCC_gov_2013_04
E1821_WCC_gov_2013_03
E1821_WCC_gov_2013_02
E1821_WCC_gov_2013_01
E1821_WCC_gov_2012_12
E1821_WCC_gov_2012_11
E1821_WCC_gov_2012_10
E1821_WCC_gov_2012_09
E1821_WCC_gov_2012_08
E1821_WCC_gov_2012_07
E1821_WCC_gov_2012_06
E1821_WCC_gov_2012_05
E1821_WCC_gov_2012_04
E1821_WCC_gov_2012_03
E1821_WCC_gov_2012_02
E1821_WCC_gov_2012_01
E1821_WCC_gov_2011_12
E1821_WCC_gov_2011_11
E1821_WCC_gov_2011_10
E1821_WCC_gov_2011_09
E1821_WCC_gov_2011_08
E1821_WCC_gov_2011_07
E1821_WCC_gov_2011_06
E1821_WCC_gov_2011_05
E1821_WCC_gov_2011_04
E1821_WCC_gov_2011_03
E1821_WCC_gov_2011_02
E1821_WCC_gov_2011_01
E1821_WCC_gov_2010_12
E1821_WCC_gov_2010_11
E1821_WCC_gov_2010_10
E1821_WCC_gov_2010_09
E1821_WCC_gov_2010_08
E1821_WCC_gov_2010_07
E1821_WCC_gov_2010_06
E1821_WCC_gov_2010_05
E1821_WCC_gov_2010_04
Data
Downloaded
863 times
by
To download data sign in with GitHub
rows 10 / 135
| l | f | d |
|---|---|---|
|
E1821_WCC_gov_2017_12
|
2018-05-31 14:04:05.627855
|
|
|
E1821_WCC_gov_2017_11
|
2018-05-31 14:04:08.300110
|
|
|
E1821_WCC_gov_2017_10
|
2018-05-31 14:04:11.068998
|
|
|
E1821_WCC_gov_2017_09
|
2018-05-31 14:04:14.409051
|
|
|
E1821_WCC_gov_2017_08
|
2018-05-31 14:04:16.837873
|
|
|
E1821_WCC_gov_2017_07
|
2018-05-31 14:04:20.042820
|
|
|
E1821_WCC_gov_2017_06
|
2018-05-31 14:04:21.530343
|
|
|
E1821_WCC_gov_2017_05
|
2018-05-31 14:04:22.837482
|
|
|
E1821_WCC_gov_2017_04
|
2018-05-31 14:04:24.110445
|
|
|
E1821_WCC_gov_2017_03
|
2018-05-31 14:04:25.040471
|
Statistics
Average successful run time: 9 minutes
Total run time: 24 days
Total cpu time used: about 1 hour
Total disk space used: 94 KB
History
-
Auto
ran
revision
50cff8fa
and
completed successfully
.
116 records added, 116 records removed in the database
-
Auto
ran
revision
50cff8fa
and
completed successfully
.
116 records added, 116 records removed in the database
-
Auto
ran
revision
50cff8fa
and
completed successfully
.
116 records added, 116 records removed in the database
-
Auto
ran
revision
50cff8fa
and
completed successfully
.
116 records added, 116 records removed in the database
-
Auto
ran
revision
50cff8fa
and
completed successfully
.
116 records added, 116 records removed in the database
- ...
- Created on morph.io