morph-test-scrapers / test-python

Simple test scraper for Python - PLEASE DO NOT DELETE

Contributors mlandauer henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Python app detected [1G ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.15, which is unsupported). [1G ! We recommend upgrading by specifying the latest version (python-2.7.14). [1G Learn More: https://devcenter.heroku.com/articles/python-runtimes [1G-----> Installing python-2.7.15 [1G-----> Installing pip [1G-----> Installing requirements with pip [1G DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support [1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 2)) [1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki [1G Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki [1G Running command git checkout -b morph_defaults --track origin/morph_defaults [1G Switched to a new branch 'morph_defaults' [1G Branch morph_defaults set up to track remote branch morph_defaults from origin. [1G Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 2)) [1G Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz [1G Collecting requests (from scraperwiki->-r /tmp/build/requirements.txt (line 2)) [1G Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) [1G Collecting certifi>=2017.4.17 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2)) [1G Downloading https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl (157kB) [1G Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2)) [1G Downloading https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl (150kB) [1G Collecting idna<2.9,>=2.5 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2)) [1G Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) [1G Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 2)) [1G Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) [1G Building wheels for collected packages: dumptruck [1G Building wheel for dumptruck (setup.py): started [1G Building wheel for dumptruck (setup.py): finished with status 'done' [1G Created wheel for dumptruck: filename=dumptruck-0.1.6-cp27-none-any.whl size=11845 sha256=8e743afe368bead377617ca3b8b0023d9ae6b1670710d6e3d3642d9adcadb469 [1G Stored in directory: /tmp/pip-ephem-wheel-cache-Y8nOVI/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e [1G Successfully built dumptruck [1G Installing collected packages: dumptruck, certifi, urllib3, idna, chardet, requests, scraperwiki [1G Running setup.py develop for scraperwiki [1G Successfully installed certifi-2019.6.16 chardet-3.0.4 dumptruck-0.1.6 idna-2.8 requests-2.22.0 scraperwiki urllib3-1.25.3 [91mDEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support [0m [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... 1... 2... 3... 4... 5...

Data

Downloaded 3 times by MikeRalphson howawong

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3 KB) Use the API

rows 1 / 1

occupation name
software developer
susan

Statistics

Average successful run time: 3 minutes

Total run time: about 2 hours

Total cpu time used: half a minute

Total disk space used: 32.8 KB

History

  • Manually ran revision 7cb85cdd and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision b6159004 and failed .
    nothing changed in the database
  • Manually ran revision 28907c40 and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision 643effee and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision 91271203 and completed successfully .
    1 record added, 1 record removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

test-python