g0rd / cet_lebanon

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G-----> Python app detected [1G-----> Stack changed, re-installing runtime [1G-----> Installing runtime (python-2.7.9) [1G-----> Installing dependencies with pip [1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 6)) [1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki [1G Collecting lxml==3.4.4 (from -r requirements.txt (line 8)) [1G Downloading lxml-3.4.4.tar.gz (3.5MB) [1G Building lxml version 3.4.4. [1G Building without Cython. [1G Using build configuration of libxslt 1.1.28 [1G /app/.heroku/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url' [1G warnings.warn(msg) [1G Collecting cssselect==0.9.1 (from -r requirements.txt (line 9)) [1G Downloading cssselect-0.9.1.tar.gz [1G Collecting splinter>=0.7.3 (from -r requirements.txt (line 10)) [1G Downloading splinter-0.7.3.tar.gz [1G Collecting scrapy==0.24 (from -r requirements.txt (line 11)) [1G Downloading Scrapy-0.24.0-py2-none-any.whl (501kB) [1G Collecting service-identity==14.0.0 (from -r requirements.txt (line 12)) [1G Downloading service_identity-14.0.0-py2.py3-none-any.whl [1G Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 6)) [1G Downloading dumptruck-0.1.6.tar.gz [1G Collecting requests (from scraperwiki->-r requirements.txt (line 6)) [1G Downloading requests-2.7.0-py2.py3-none-any.whl (470kB) [1G Collecting selenium>=2.47.1 (from splinter>=0.7.3->-r requirements.txt (line 10)) [1G Downloading selenium-2.47.1-py2-none-any.whl (3.0MB) [1G Collecting queuelib (from scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading queuelib-1.3.0-py2.py3-none-any.whl [1G Collecting pyOpenSSL (from scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading pyOpenSSL-0.15.1-py2.py3-none-any.whl (102kB) [1G Collecting Twisted>=10.0.0 (from scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading Twisted-15.4.0.tar.bz2 (3.1MB) [1G Collecting six>=1.5.2 (from scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading six-1.9.0-py2.py3-none-any.whl [1G Collecting w3lib>=1.2 (from scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading w3lib-1.12.0-py2.py3-none-any.whl [1G Collecting characteristic>=14.0.0 (from service-identity==14.0.0->-r requirements.txt (line 12)) [1G Downloading characteristic-14.3.0-py2.py3-none-any.whl [1G Collecting pyasn1-modules (from service-identity==14.0.0->-r requirements.txt (line 12)) [1G Downloading pyasn1-modules-0.0.7.tar.gz [1G Collecting pyasn1 (from service-identity==14.0.0->-r requirements.txt (line 12)) [1G Downloading pyasn1-0.1.8.tar.gz (75kB) [1G Collecting cryptography>=0.7 (from pyOpenSSL->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading cryptography-1.0.tar.gz (331kB) [1G Collecting zope.interface>=3.6.0 (from Twisted>=10.0.0->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading zope.interface-4.1.2.tar.gz (919kB) [1G Collecting idna>=2.0 (from cryptography>=0.7->pyOpenSSL->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading idna-2.0-py2.py3-none-any.whl (61kB) [1G Collecting enum34 (from cryptography>=0.7->pyOpenSSL->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading enum34-1.0.4.tar.gz [1G Collecting ipaddress (from cryptography>=0.7->pyOpenSSL->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading ipaddress-1.0.14-py27-none-any.whl [1G Collecting cffi>=1.1.0 (from cryptography>=0.7->pyOpenSSL->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading cffi-1.2.1.tar.gz (335kB) [1G Collecting pycparser (from cffi>=1.1.0->cryptography>=0.7->pyOpenSSL->scrapy==0.24->-r requirements.txt (line 11)) [1G Downloading pycparser-2.14.tar.gz (223kB) [1G Installing collected packages: pycparser, cffi, ipaddress, enum34, idna, zope.interface, cryptography, pyasn1, pyasn1-modules, characteristic, w3lib, six, Twisted, pyOpenSSL, queuelib, selenium, requests, dumptruck, service-identity, scrapy, splinter, cssselect, lxml, scraperwiki [1G Running setup.py install for pycparser [1G Build the lexing/parsing tables [1G Running setup.py install for cffi [1G building '_cffi_backend' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -I/app/.heroku/python/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/c/_cffi_backend.o -lffi -o build/lib.linux-x86_64-2.7/_cffi_backend.so [1G [1G Running setup.py install for enum34 [1G [1G Running setup.py install for zope.interface [1G building 'zope.interface._zope_interface_coptimizations' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-2.7/src/zope/interface/_zope_interface_coptimizations.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/src/zope/interface/_zope_interface_coptimizations.o -o build/lib.linux-x86_64-2.7/zope/interface/_zope_interface_coptimizations.so [1G Skipping installation of /app/.heroku/python/lib/python2.7/site-packages/zope/__init__.py (namespace package) [1G Installing /app/.heroku/python/lib/python2.7/site-packages/zope.interface-4.1.2-py2.7-nspkg.pth [1G Running setup.py install for cryptography [1G generating cffi module 'build/temp.linux-x86_64-2.7/_padding.c' [1G generating cffi module 'build/temp.linux-x86_64-2.7/_constant_time.c' [1G generating cffi module 'build/temp.linux-x86_64-2.7/_openssl.c' [1G building '_openssl' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c build/temp.linux-x86_64-2.7/_openssl.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o -lssl -lcrypto -o build/lib.linux-x86_64-2.7/cryptography/hazmat/bindings/_openssl.so [1G building '_constant_time' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c build/temp.linux-x86_64-2.7/_constant_time.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_constant_time.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_constant_time.o -o build/lib.linux-x86_64-2.7/cryptography/hazmat/bindings/_constant_time.so [1G building '_padding' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c build/temp.linux-x86_64-2.7/_padding.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_padding.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_padding.o -o build/lib.linux-x86_64-2.7/cryptography/hazmat/bindings/_padding.so [1G Running setup.py install for pyasn1 [1G Running setup.py install for pyasn1-modules [1G [1G [1G [1G Running setup.py install for Twisted [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c conftest.c -o conftest.o [1G building 'twisted.test.raiser' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c twisted/test/raiser.c -o build/temp.linux-x86_64-2.7/twisted/test/raiser.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/twisted/test/raiser.o -o build/lib.linux-x86_64-2.7/twisted/test/raiser.so [1G building 'twisted.python._sendmsg' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c twisted/python/_sendmsg.c -o build/temp.linux-x86_64-2.7/twisted/python/_sendmsg.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/twisted/python/_sendmsg.o -o build/lib.linux-x86_64-2.7/twisted/python/_sendmsg.so [1G building 'twisted.runner.portmap' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/app/.heroku/python/include/python2.7 -c twisted/runner/portmap.c -o build/temp.linux-x86_64-2.7/twisted/runner/portmap.o [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/twisted/runner/portmap.o -o build/lib.linux-x86_64-2.7/twisted/runner/portmap.so [1G changing mode of build/scripts-2.7/tkconch from 644 to 755 [1G changing mode of build/scripts-2.7/conch from 644 to 755 [1G changing mode of build/scripts-2.7/pyhtmlizer from 644 to 755 [1G changing mode of build/scripts-2.7/tap2rpm from 644 to 755 [1G changing mode of build/scripts-2.7/manhole from 644 to 755 [1G changing mode of build/scripts-2.7/ckeygen from 644 to 755 [1G changing mode of build/scripts-2.7/tap2deb from 644 to 755 [1G changing mode of build/scripts-2.7/mailmail from 644 to 755 [1G changing mode of build/scripts-2.7/trial from 644 to 755 [1G changing mode of build/scripts-2.7/twistd from 644 to 755 [1G changing mode of build/scripts-2.7/cftp from 644 to 755 [1G changing mode of /app/.heroku/python/bin/tkconch to 755 [1G changing mode of /app/.heroku/python/bin/conch to 755 [1G changing mode of /app/.heroku/python/bin/pyhtmlizer to 755 [1G changing mode of /app/.heroku/python/bin/tap2rpm to 755 [1G changing mode of /app/.heroku/python/bin/manhole to 755 [1G changing mode of /app/.heroku/python/bin/ckeygen to 755 [1G changing mode of /app/.heroku/python/bin/tap2deb to 755 [1G changing mode of /app/.heroku/python/bin/mailmail to 755 [1G changing mode of /app/.heroku/python/bin/trial to 755 [1G changing mode of /app/.heroku/python/bin/twistd to 755 [1G changing mode of /app/.heroku/python/bin/cftp to 755 [1G [1G [1G Compiling /tmp/pip-build-cf1_q8/selenium/selenium/test/selenium/webdriver/browser_specific_template.py [1G [1G [1G [1G Running setup.py install for dumptruck [1G [1G [1G Running setup.py install for splinter [1G Running setup.py install for cssselect [1G Running setup.py install for lxml [1G Building lxml version 3.4.4. [1G Building without Cython. [1G Using build configuration of libxslt 1.1.28 [1G /app/.heroku/python/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url' [1G warnings.warn(msg) [1G building 'lxml.etree' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-cf1_q8/lxml/src/lxml/includes -I/app/.heroku/python/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so [1G building 'lxml.objectify' extension [1G gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-cf1_q8/lxml/src/lxml/includes -I/app/.heroku/python/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -w [1G gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/objectify.so [1G Running setup.py develop for scraperwiki [1G Creating /app/.heroku/python/lib/python2.7/site-packages/scraperwiki.egg-link (link to .) [1G Adding scraperwiki 0.3.7 to easy-install.pth file [1G Installed /app/.heroku/src/scraperwiki [1G Successfully installed Twisted-15.4.0 cffi-1.2.1 characteristic-14.3.0 cryptography-1.0 cssselect-0.9.1 dumptruck-0.1.6 enum34-1.0.4 idna-2.0 ipaddress-1.0.14 lxml-3.4.4 pyOpenSSL-0.15.1 pyasn1-0.1.8 pyasn1-modules-0.0.7 pycparser-2.14 queuelib-1.3.0 requests-2.7.0 scraperwiki scrapy-0.24.0 selenium-2.47.1 service-identity-14.0.0 six-1.9.0 splinter-0.7.3 w3lib-1.12.0 zope.interface-4.1.2 [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Scrapy 0.24.0 - no active project Unknown command: crawl Use "scrapy" to see available commands

Statistics

Average successful run time: 1 minute

Total run time: about 2 hours

Total cpu time used: 5 minutes

Total disk space used: 47.1 KB

History

  • Manually ran revision e3ae7399 and completed successfully .
    nothing changed in the database
  • Manually ran revision 1be7ae1d and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision 3a86dd76 and failed .
    nothing changed in the database
  • Manually ran revision d161339d and failed .
    nothing changed in the database
    22 pages scraped
  • Manually ran revision d93dcbd0 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

cet_lebanon