CharlotteMaher / chapter18scraper

free school meals scotland

Contributors CharlotteMaher

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 8))  Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)  Collecting cssselect==0.9.1 (from -r /tmp/build/requirements.txt (line 9))  Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl (157kB)  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl (150kB)  Collecting idna<2.9,>=2.5 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki->-r /tmp/build/requirements.txt (line 6))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Building wheels for collected packages: lxml, cssselect, dumptruck  Building wheel for lxml (setup.py): started  Building wheel for lxml (setup.py): still running...  Building wheel for lxml (setup.py): finished with status 'done'  Created wheel for lxml: filename=lxml-3.4.4-cp27-cp27m-linux_x86_64.whl size=2989851 sha256=a4fb573ad54a996c3450ce0108f30177cc163d00fbe36128616366e9ebfe6d9c  Stored in directory: /tmp/pip-ephem-wheel-cache-uCxY6_/wheels/f6/df/7b/af9cace9baf95a6e4a2b5790e30da55fc780ddee598314d1ed  Building wheel for cssselect (setup.py): started  Building wheel for cssselect (setup.py): finished with status 'done'  Created wheel for cssselect: filename=cssselect-0.9.1-cp27-none-any.whl size=26992 sha256=05ad04eccb2c964286b041c3a821023d606b83d5900044656a09cf3e22d74e60  Stored in directory: /tmp/pip-ephem-wheel-cache-uCxY6_/wheels/45/25/d7/5a3b06d22b1ffb616f868a74729a5a002bcc04d45109b4f223  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Created wheel for dumptruck: filename=dumptruck-0.1.6-cp27-none-any.whl size=11845 sha256=47961fc6a28339e45dfed201497a11388319a3438b4bc18838adb892b6ff7221  Stored in directory: /tmp/pip-ephem-wheel-cache-uCxY6_/wheels/57/df/83/32654ae89119876c7a7db66829bbdb646caa151589dbaf226e  Successfully built lxml cssselect dumptruck  Installing collected packages: dumptruck, certifi, urllib3, idna, chardet, requests, scraperwiki, lxml, cssselect  Running setup.py develop for scraperwiki  Successfully installed certifi-2019.6.16 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 idna-2.8 lxml-3.4.4 requests-2.22.0 scraperwiki urllib3-1.25.3 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support    -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... 5237521 <!DOCTYPE html > <html dir="ltr" lang="en-GB"> <head><title> Abbotswell School </title> </head> <body> <h1> Abbotswell School </h1> <table class="destinations"> <tr> <th> Free school meals </th> </tr> <tr> <td> 24.10% </td> </tr> </table> </body> </html> {'ID': '5237521', 'FSM': '\n24.10%\n'} ------------ 5244439 <!DOCTYPE html > <html dir="ltr" lang="en-GB"> <head><title> Aberdeen Grammar School </title> </head> <body> <h1> Aberdeen Grammar School </h1> <table class="destinations"> <tr> <th> Free school meals </th> </tr> <tr> <td> 4.10% </td> </tr> </table> </body> </html> {'ID': '5244439', 'FSM': '\n4.10%\n'} ------------ 5237629 <!DOCTYPE html > <html dir="ltr" lang="en-GB"> <head><title> Airyhall School </title> </head> <body> <h1> Airyhall School </h1> <table class="destinations"> <tr> <th> Free school meals </th> </tr> <tr> <td> 1.60% </td> </tr> </table> </body> </html> {'ID': '5237629', 'FSM': '\n1.60%\n'} ------------ 5237823 <!DOCTYPE html > <html dir="ltr" lang="en-GB"> <head><title> Ashley Road School </title> </head> <body> <h1> Ashley Road School </h1> <table class="destinations"> <tr> <th> Free school meals </th> </tr> <tr> <td> 3.50% </td> </tr> </table> </body> </html> {'ID': '5237823', 'FSM': '\n3.50%\n'} ------------ 5234026 <!DOCTYPE html > <html dir="ltr" lang="en-GB"> <head><title> Braehead School and Nursery Class </title> </head> <body> <h1> Braehead School and Nursery Class </h1> <table class="destinations"> <tr> <th> Free school meals </th> </tr> <tr> <td> 6.60% </td> </tr> </table> </body> </html> {'ID': '5234026', 'FSM': '\n6.60%\n'} ------------

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3 KB) Use the API

rows 5 / 5

FSM ID
24.10%
5237521
4.10%
5244439
1.60%
5237629
3.50%
5237823
6.60%
5234026

Statistics

Average successful run time: less than a minute

Total run time: 2 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 27.7 KB

History

  • Manually ran revision 68b7476c and completed successfully .
    5 records added, 5 records removed in the database
  • Manually ran revision 68b7476c and completed successfully .
    5 records added in the database
  • Created on morph.io

Scraper code

chapter18scraper