This is a scraper that runs on Morph. To get started see the documentation

Contributors woodbine blablupcom

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-2.7.6 -----> Noticed cffi. Bootstrapping libffi.  $ pip install -r requirements.txt  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 2))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki  Collecting BeautifulSoup==3.2.0 (from -r requirements.txt (line 9))  /app/.heroku/python/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.  SNIMissingWarning  /app/.heroku/python/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.  InsecurePlatformWarning  Downloading BeautifulSoup-3.2.0.tar.gz  Collecting Creoleparser==0.7.4 (from -r requirements.txt (line 10))  Downloading Creoleparser-0.7.4.zip  Collecting Genshi==0.6 (from -r requirements.txt (line 11))  Downloading Genshi-0.6.tar.gz (433kB)  Collecting Jinja2==2.6 (from -r requirements.txt (line 12))  Downloading Jinja2-2.6.tar.gz (389kB)  Collecting Markdown==2.2.0 (from -r requirements.txt (line 13))  Downloading Markdown-2.2.0.tar.gz (236kB)  Collecting Pygments==1.4 (from -r requirements.txt (line 14))  Downloading Pygments-1.4.tar.gz (3.5MB)  Collecting SQLAlchemy==0.6.6 (from -r requirements.txt (line 15))  Downloading SQLAlchemy-0.6.6.tar.gz (2.1MB)  Collecting Twisted==11.1.0 (from -r requirements.txt (line 16))  Downloading Twisted-11.1.0.tar.bz2 (2.8MB)  Collecting Unidecode==0.04.9 (from -r requirements.txt (line 17))  Downloading Unidecode-0.04.9.tar.gz (196kB)  Collecting anyjson==0.3.3 (from -r requirements.txt (line 18))  Downloading anyjson-0.3.3.tar.gz  Collecting argparse==1.2.1 (from -r requirements.txt (line 19))  Downloading argparse-1.2.1.tar.gz (69kB)  Collecting beautifulsoup4==4.1.3 (from -r requirements.txt (line 20))  Downloading beautifulsoup4-4.1.3.tar.gz (58kB)  Collecting bitlyapi==0.1.1 (from -r requirements.txt (line 21))  Downloading bitlyapi-0.1.1.tar.gz  Collecting blinker==1.2 (from -r requirements.txt (line 22))  Downloading blinker-1.2.tar.gz (66kB)  Collecting cartodb==0.6 (from -r requirements.txt (line 23))  Downloading cartodb-0.6.tar.gz  Collecting certifi==0.0.8 (from -r requirements.txt (line 24))  Downloading certifi-0.0.8.tar.gz (118kB)  Collecting chardet==2.1.1 (from -r requirements.txt (line 25))  Downloading chardet-2.1.1.tar.gz (178kB)  Collecting ckanclient==0.10 (from -r requirements.txt (line 26))  Downloading ckanclient-0.10.tar.gz  Collecting colormath==1.0.8 (from -r requirements.txt (line 27))  Downloading colormath-1.0.8.tar.gz  Collecting csvkit==0.3.0 (from -r requirements.txt (line 28))  Downloading csvkit-0.3.0.tar.gz  Collecting dataset==0.5.2 (from -r requirements.txt (line 29))  Downloading dataset-0.5.2.tar.gz  Collecting demjson==1.6 (from -r requirements.txt (line 30))  Downloading demjson-1.6.tar.gz (64kB)  Collecting dropbox==1.4 (from -r requirements.txt (line 31))  Downloading dropbox-1.4.tar.gz  Collecting errorhandler==1.1.1 (from -r requirements.txt (line 32))  Downloading errorhandler-1.1.1.tar.gz  Collecting feedparser==5.0.1 (from -r requirements.txt (line 33))  Downloading feedparser-5.0.1.zip (2.0MB)  Collecting fluidinfo.py==1.1.2 (from -r requirements.txt (line 34))  Downloading fluidinfo.py-1.1.2.tar.gz  Collecting gdata==2.0.15 (from -r requirements.txt (line 35))  Downloading gdata-2.0.15.tar.gz (2.0MB)  Collecting geopy==0.94.1 (from -r requirements.txt (line 36))  Downloading geopy-0.94.1.tar.gz  Collecting gevent==0.13.6 (from -r requirements.txt (line 37))  Downloading gevent-0.13.6.tar.gz (289kB)  Collecting google-api-python-client==1.0beta8 (from -r requirements.txt (line 38))  Downloading google-api-python-client-1.0beta8.zip (631kB)  Collecting googlemaps==1.0.2 (from -r requirements.txt (line 39))  Downloading googlemaps-1.0.2.tar.gz (60kB)  Collecting greenlet==0.3.2 (from -r requirements.txt (line 40))  Downloading greenlet-0.3.2.zip (50kB)  Collecting html5lib==0.90 (from -r requirements.txt (line 41))  Downloading html5lib-0.90.tar.gz (86kB)  Collecting httplib2==0.7.4 (from -r requirements.txt (line 42))  Downloading httplib2-0.7.4.tar.gz (106kB)  Collecting imposm.parser==1.0.3 (from -r requirements.txt (line 43))  Downloading imposm.parser-1.0.3.tar.gz  Collecting jellyfish==0.2.0 (from -r requirements.txt (line 44))  Downloading jellyfish-0.2.0.tar.gz  Collecting mechanize==0.2.5 (from -r requirements.txt (line 45))  Downloading mechanize-0.2.5.tar.gz (383kB)  Collecting mock==0.7.2 (from -r requirements.txt (line 46))  Downloading mock-0.7.2.zip (938kB)  Collecting networkx==1.6 (from -r requirements.txt (line 47))  Downloading networkx-1.6.zip (1.0MB)  Collecting ngram==3.3.0 (from -r requirements.txt (line 48))  Downloading ngram-3.3.0.tar.gz  Collecting nose==1.1.2 (from -r requirements.txt (line 49))  Downloading nose-1.1.2.tar.gz (729kB)  Collecting oauth2==1.5.170 (from -r requirements.txt (line 50))  Downloading oauth2-1.5.170.tar.gz  Collecting oauth==1.0.1 (from -r requirements.txt (line 51))  Downloading oauth-1.0.1.tar.gz  Collecting oauthlib==0.1.2 (from -r requirements.txt (line 52))  Downloading oauthlib-0.1.2.tar.gz  Collecting openpyxl==1.5.7 (from -r requirements.txt (line 53))  Downloading openpyxl-1.5.7.tar.gz (67kB)  Collecting ordereddict==1.1 (from -r requirements.txt (line 54))  Downloading ordereddict-1.1.tar.gz  Collecting pbkdf2==1.3 (from -r requirements.txt (line 55))  Downloading pbkdf2-1.3.tar.gz  Collecting pdfminer==20110515 (from -r requirements.txt (line 56))  Downloading pdfminer-20110515.tar.gz (4.1MB)  Collecting pexpect==2.4 (from -r requirements.txt (line 57))  Downloading pexpect-2.4.tar.gz (113kB)  Collecting pipe2py==0.9.2 (from -r requirements.txt (line 58))  Downloading pipe2py-0.9.2.tar.gz (57kB)  Collecting pyOpenSSL==0.13 (from -r requirements.txt (line 59))  Downloading pyOpenSSL-0.13.tar.gz (250kB)  Collecting pycrypto==2.5 (from -r requirements.txt (line 60))  Downloading pycrypto-2.5.tar.gz (426kB)  Collecting pycurl==7.19.0 (from -r requirements.txt (line 61))  Downloading pycurl-7.19.0.tar.gz (70kB)  Collecting pyephem==3.7.5.1 (from -r requirements.txt (line 62))  Downloading pyephem-3.7.5.1.tar.gz (703kB)  Collecting pyparsing==1.5.6 (from -r requirements.txt (line 63))  Downloading pyparsing-1.5.6.zip (1.7MB)  Collecting pyth==0.5.6 (from -r requirements.txt (line 64))  Downloading pyth-0.5.6.tar.gz  Collecting python-Levenshtein==0.10.2 (from -r requirements.txt (line 65))  Downloading python-Levenshtein-0.10.2.tar.gz (45kB)  Collecting python-dateutil==1.5 (from -r requirements.txt (line 66))  Downloading python-dateutil-1.5.tar.gz (233kB)  Collecting python-gflags==2.0 (from -r requirements.txt (line 67))  Downloading python-gflags-2.0.tar.gz (65kB)  Collecting python-modargs==1.2 (from -r requirements.txt (line 68))  Downloading python-modargs-1.2.tar.gz  Collecting python-stdnum==0.7 (from -r requirements.txt (line 69))  Downloading python-stdnum-0.7.tar.gz (113kB)  Collecting pytz==2011k (from -r requirements.txt (line 70))  Downloading pytz-2011k.zip (517kB)  Collecting rdflib==3.1.0 (from -r requirements.txt (line 71))  Downloading rdflib-3.1.0.tar.gz (249kB)  Collecting requests-foauth==0.1.1 (from -r requirements.txt (line 72))  Downloading requests-foauth-0.1.1.tar.gz  Collecting requests==1.0.4 (from -r requirements.txt (line 73))  Downloading requests-1.0.4.tar.gz (336kB)  Collecting selenium==2.5.0 (from -r requirements.txt (line 74))  Downloading selenium-2.5.0.tar.gz (2.4MB)  Collecting simplejson==2.2.1 (from -r requirements.txt (line 75))  Downloading simplejson-2.2.1.tar.gz (49kB)  Collecting suds==0.4 (from -r requirements.txt (line 76))  Downloading suds-0.4.tar.gz (104kB)  Collecting tweepy==1.7.1 (from -r requirements.txt (line 77))  Downloading tweepy-1.7.1.tar.gz  Collecting tweetstream==1.1.1 (from -r requirements.txt (line 78))  Downloading tweetstream-1.1.1.tar.gz  Collecting w3lib==1.0 (from -r requirements.txt (line 79))  Downloading w3lib-1.0.tar.gz  Collecting xlrd==0.7.1 (from -r requirements.txt (line 81))  Downloading xlrd-0.7.1.zip (125kB)  Collecting xlutils==1.4.1 (from -r requirements.txt (line 82))  Downloading xlutils-1.4.1.tar.gz (40kB)  Collecting xlwt==0.7.2 (from -r requirements.txt (line 83))  Downloading xlwt-0.7.2.zip (131kB)  Collecting xmltodict==0.4 (from -r requirements.txt (line 84))  Downloading xmltodict-0.4.tar.gz  Collecting zope.interface==3.8.0 (from -r requirements.txt (line 88))  Downloading zope.interface-3.8.0.tar.gz (111kB)  Collecting lxml==2.3.3 (from -r requirements.txt (line 89))  Downloading lxml-2.3.3.tar.gz (3.1MB)  Collecting chromium-compact-language-detector==0.031415 (from -r requirements.txt (line 90))  Downloading chromium_compact_language_detector-0.031415.tar.gz (2.2MB)  Collecting icalendar==3.0.1b1 (from -r requirements.txt (line 93))  Downloading icalendar-3.0.1b1.tar.gz  Collecting pyquery==1.0 (from -r requirements.txt (line 96))  Downloading pyquery-1.0.tar.gz  Collecting scrapely==0.9 (from -r requirements.txt (line 98))  Downloading scrapely-0.9.tar.gz  Collecting Fom==0.9.8 (from -r requirements.txt (line 103))  Downloading Fom-0.9.8.zip (79kB)  Collecting PyYAML==3.10 (from -r requirements.txt (line 105))  Downloading PyYAML-3.10.tar.gz (241kB)  Collecting Scrapy==0.14.1 (from -r requirements.txt (line 107))  Downloading Scrapy-0.14.1.tar.gz (719kB)  Collecting adspygoogle.adwords==15.6.2 (from -r requirements.txt (line 109))  Downloading adspygoogle.adwords-15.6.2.tar.gz (166kB)  Collecting nltk==3.0.2 (from -r requirements.txt (line 111))  Downloading nltk-3.0.2.tar.gz (991kB)  Collecting pydot==1.0.2 (from -r requirements.txt (line 113))  Downloading pydot-1.0.2.tar.gz  Collecting M2Crypto==0.22.3 (from -r requirements.txt (line 115))  Downloading M2Crypto-0.22.3.tar.gz (74kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 2))  Downloading dumptruck-0.1.6.tar.gz  Collecting alembic>=0.6.2 (from dataset==0.5.2->-r requirements.txt (line 29))  Downloading alembic-0.8.8.tar.gz (970kB)  Collecting python-slugify>=0.0.6 (from dataset==0.5.2->-r requirements.txt (line 29))  Downloading python-slugify-1.2.1.tar.gz  Collecting distribute (from python-stdnum==0.7->-r requirements.txt (line 69))  Downloading distribute-0.7.3.zip (145kB)  Collecting numpy (from scrapely==0.9->-r requirements.txt (line 98))  Downloading numpy-1.11.2-cp27-cp27m-manylinux1_x86_64.whl (15.3MB)  Collecting Mako (from alembic>=0.6.2->dataset==0.5.2->-r requirements.txt (line 29))  Downloading Mako-1.0.4.tar.gz (574kB)  Collecting python-editor>=0.3 (from alembic>=0.6.2->dataset==0.5.2->-r requirements.txt (line 29))  Downloading python-editor-1.0.1.tar.gz  Collecting MarkupSafe>=0.9.2 (from Mako->alembic>=0.6.2->dataset==0.5.2->-r requirements.txt (line 29))  Downloading MarkupSafe-0.23.tar.gz  Installing collected packages: dumptruck, requests, scraperwiki, BeautifulSoup, Genshi, Creoleparser, Jinja2, Markdown, Pygments, SQLAlchemy, zope.interface, Twisted, Unidecode, anyjson, argparse, beautifulsoup4, bitlyapi, blinker, httplib2, oauth2, cartodb, certifi, chardet, ckanclient, colormath, xlrd, python-dateutil, csvkit, MarkupSafe, Mako, python-editor, alembic, python-slugify, PyYAML, dataset, demjson, oauth, simplejson, dropbox, errorhandler, feedparser, fluidinfo.py, gdata, geopy, greenlet, gevent, python-gflags, google-api-python-client, googlemaps, html5lib, imposm.parser, jellyfish, mechanize, mock, networkx, ngram, nose, pycrypto, oauthlib, openpyxl, ordereddict, pbkdf2, pdfminer, pexpect, pipe2py, pyOpenSSL, pycurl, pyephem, pyparsing, pyth, python-Levenshtein, python-modargs, distribute, python-stdnum, pytz, rdflib, requests-foauth, selenium, suds, tweepy, tweetstream, w3lib, xlwt, xlutils, xmltodict, lxml, chromium-compact-language-detector, icalendar, pyquery, numpy, scrapely, Fom, Scrapy, adspygoogle.adwords, nltk, pydot, M2Crypto  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py install for requests: started  Running setup.py install for requests: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for BeautifulSoup: started  Running setup.py install for BeautifulSoup: finished with status 'done'  Running setup.py install for Genshi: started  Running setup.py install for Genshi: finished with status 'done'  Running setup.py install for Creoleparser: started  Running setup.py install for Creoleparser: finished with status 'done'  Running setup.py install for Jinja2: started  Running setup.py install for Jinja2: finished with status 'done'  Running setup.py install for Markdown: started  Running setup.py install for Markdown: finished with status 'done'  Running setup.py install for Pygments: started  Running setup.py install for Pygments: finished with status 'done'  Running setup.py install for SQLAlchemy: started  Running setup.py install for SQLAlchemy: finished with status 'done'  Running setup.py install for zope.interface: started  Running setup.py install for zope.interface: finished with status 'done'  Running setup.py install for Twisted: started  Running setup.py install for Twisted: finished with status 'done'  Running setup.py install for Unidecode: started  Running setup.py install for Unidecode: finished with status 'done'  Running setup.py install for anyjson: started  Running setup.py install for anyjson: finished with status 'done'  Running setup.py install for argparse: started  Running setup.py install for argparse: finished with status 'done'  Running setup.py install for beautifulsoup4: started  Running setup.py install for beautifulsoup4: finished with status 'done'  Running setup.py install for bitlyapi: started  Running setup.py install for bitlyapi: finished with status 'done'  Running setup.py install for blinker: started  Running setup.py install for blinker: finished with status 'done'  Running setup.py install for httplib2: started  Running setup.py install for httplib2: finished with status 'done'  Running setup.py install for oauth2: started  Running setup.py install for oauth2: finished with status 'done'  Running setup.py install for cartodb: started  Running setup.py install for cartodb: finished with status 'done'  Running setup.py install for certifi: started  Running setup.py install for certifi: finished with status 'done'  Running setup.py install for chardet: started  Running setup.py install for chardet: finished with status 'done'  Running setup.py install for ckanclient: started  Running setup.py install for ckanclient: finished with status 'done'  Running setup.py install for colormath: started  Running setup.py install for colormath: finished with status 'done'  Running setup.py install for xlrd: started  Running setup.py install for xlrd: finished with status 'done'  Running setup.py install for python-dateutil: started  Running setup.py install for python-dateutil: finished with status 'done'  Running setup.py install for csvkit: started  Running setup.py install for csvkit: finished with status 'done'  Running setup.py install for MarkupSafe: started  Running setup.py install for MarkupSafe: finished with status 'done'  Running setup.py install for Mako: started  Running setup.py install for Mako: finished with status 'done'  Running setup.py install for python-editor: started  Running setup.py install for python-editor: finished with status 'done'  Running setup.py install for alembic: started  Running setup.py install for alembic: finished with status 'done'  Running setup.py install for python-slugify: started  Running setup.py install for python-slugify: finished with status 'done'  Running setup.py install for PyYAML: started  Running setup.py install for PyYAML: finished with status 'done'  Running setup.py install for dataset: started  Running setup.py install for dataset: finished with status 'done'  Running setup.py install for demjson: started  Running setup.py install for demjson: finished with status 'done'  Running setup.py install for oauth: started  Running setup.py install for oauth: finished with status 'done'  Running setup.py install for simplejson: started  Running setup.py install for simplejson: finished with status 'done'  Running setup.py install for dropbox: started  Running setup.py install for dropbox: finished with status 'done'  Running setup.py install for errorhandler: started  Running setup.py install for errorhandler: finished with status 'done'  Running setup.py install for feedparser: started  Running setup.py install for feedparser: finished with status 'done'  Running setup.py install for fluidinfo.py: started  Running setup.py install for fluidinfo.py: finished with status 'done'  Running setup.py install for gdata: started  Running setup.py install for gdata: finished with status 'done'  Running setup.py install for geopy: started  Running setup.py install for geopy: finished with status 'done'  Running setup.py install for greenlet: started  Running setup.py install for greenlet: finished with status 'done'  Running setup.py install for gevent: started  Running setup.py install for gevent: finished with status 'done'  Running setup.py install for python-gflags: started  Running setup.py install for python-gflags: finished with status 'done'  Running setup.py install for google-api-python-client: started  Running setup.py install for google-api-python-client: finished with status 'done'  Running setup.py install for googlemaps: started  Running setup.py install for googlemaps: finished with status 'done'  Running setup.py install for html5lib: started  Running setup.py install for html5lib: finished with status 'done'  Running setup.py install for imposm.parser: started  Running setup.py install for imposm.parser: finished with status 'done'  Running setup.py install for jellyfish: started  Running setup.py install for jellyfish: finished with status 'done'  Running setup.py install for mechanize: started  Running setup.py install for mechanize: finished with status 'done'  Running setup.py install for mock: started  Running setup.py install for mock: finished with status 'done'  Running setup.py install for networkx: started  Running setup.py install for networkx: finished with status 'done'  Running setup.py install for ngram: started  Running setup.py install for ngram: finished with status 'done'  Running setup.py install for nose: started  Running setup.py install for nose: finished with status 'done'  Running setup.py install for pycrypto: started  Running setup.py install for pycrypto: finished with status 'done'  Running setup.py install for oauthlib: started  Running setup.py install for oauthlib: finished with status 'done'  Running setup.py install for openpyxl: started  Running setup.py install for openpyxl: finished with status 'done'  Running setup.py install for ordereddict: started  Running setup.py install for ordereddict: finished with status 'done'  Running setup.py install for pbkdf2: started  Running setup.py install for pbkdf2: finished with status 'done'  Running setup.py install for pdfminer: started  Running setup.py install for pdfminer: finished with status 'done'  Running setup.py install for pexpect: started  Running setup.py install for pexpect: finished with status 'done'  Running setup.py install for pipe2py: started  Running setup.py install for pipe2py: finished with status 'done'  Running setup.py install for pyOpenSSL: started  Running setup.py install for pyOpenSSL: finished with status 'done'  Running setup.py install for pycurl: started  Running setup.py install for pycurl: finished with status 'done'  Running setup.py install for pyephem: started  Running setup.py install for pyephem: finished with status 'done'  Running setup.py install for pyparsing: started  Running setup.py install for pyparsing: finished with status 'done'  Running setup.py install for pyth: started  Running setup.py install for pyth: finished with status 'done'  Running setup.py install for python-Levenshtein: started  Running setup.py install for python-Levenshtein: finished with status 'done'  Running setup.py install for python-modargs: started  Running setup.py install for python-modargs: finished with status 'done'  Running setup.py install for distribute: started  Running setup.py install for distribute: finished with status 'done'  Running setup.py install for python-stdnum: started  Running setup.py install for python-stdnum: finished with status 'done'  Running setup.py install for pytz: started  Running setup.py install for pytz: finished with status 'done'  Running setup.py install for rdflib: started  Running setup.py install for rdflib: finished with status 'done'  Running setup.py install for requests-foauth: started  Running setup.py install for requests-foauth: finished with status 'done'  Running setup.py install for selenium: started  Running setup.py install for selenium: finished with status 'done'  Running setup.py install for suds: started  Running setup.py install for suds: finished with status 'done'  Running setup.py install for tweepy: started  Running setup.py install for tweepy: finished with status 'done'  Running setup.py install for tweetstream: started  Running setup.py install for tweetstream: finished with status 'done'  Running setup.py install for w3lib: started  Running setup.py install for w3lib: finished with status 'done'  Running setup.py install for xlwt: started  Running setup.py install for xlwt: finished with status 'done'  Running setup.py install for xlutils: started  Running setup.py install for xlutils: finished with status 'done'  Running setup.py install for xmltodict: started  Running setup.py install for xmltodict: finished with status 'done'  Running setup.py install for lxml: started  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for chromium-compact-language-detector: started  Running setup.py install for chromium-compact-language-detector: finished with status 'done'  Running setup.py install for icalendar: started  Running setup.py install for icalendar: finished with status 'done'  Running setup.py install for pyquery: started  Running setup.py install for pyquery: finished with status 'done'  Running setup.py install for scrapely: started  Running setup.py install for scrapely: finished with status 'done'  Running setup.py install for Fom: started  Running setup.py install for Fom: finished with status 'done'  Running setup.py install for Scrapy: started  Running setup.py install for Scrapy: finished with status 'done'  Running setup.py install for adspygoogle.adwords: started  Running setup.py install for adspygoogle.adwords: finished with status 'done'  Running setup.py install for nltk: started  Running setup.py install for nltk: finished with status 'done'  Running setup.py install for pydot: started  Running setup.py install for pydot: finished with status 'done'  Running setup.py install for M2Crypto: started  Running setup.py install for M2Crypto: finished with status 'done'  Successfully installed BeautifulSoup-3.2.0 Creoleparser-0.7.4 Fom-0.9.8 Genshi-0.6 Jinja2-2.6 M2Crypto-0.22.3 Mako-1.0.4 Markdown-2.2.0 MarkupSafe-0.23 PyYAML-3.10 Pygments-1.4 SQLAlchemy-0.6.6 Scrapy-0.14.1 Twisted-11.1.0 Unidecode-0.4.9 adspygoogle.adwords-15.6.2 alembic-0.8.8 anyjson-0.3.3 argparse-1.2.1 beautifulsoup4-4.1.3 bitlyapi-0.1.1 blinker-1.2 cartodb-0.6 certifi-0.0.8 chardet-2.1.1 chromium-compact-language-detector-0.31415 ckanclient-0.10 colormath-1.0.8 csvkit-0.3.0 dataset-0.5.2 demjson-1.6 distribute-0.7.3 dropbox-1.4 dumptruck-0.1.6 errorhandler-1.1.1 feedparser-5.0.1 fluidinfo.py-1.1.2 gdata-2.0.15 geopy-0.94.1 gevent-0.13.6 google-api-python-client-1.0b8 googlemaps-1.0.2 greenlet-0.3.2 html5lib-0.90 httplib2-0.7.4 icalendar-3.0.1b1 imposm.parser-1.0.3 jellyfish-0.2.0 lxml-2.3.3 mechanize-0.2.5 mock-0.7.2 networkx-1.6 ngram-3.3.0 nltk-3.0.2 nose-1.1.2 numpy-1.11.2 oauth-1.0.1 oauth2-1.5.170 oauthlib-0.1.2 openpyxl-1.5.7 ordereddict-1.1 pbkdf2-1.3 pdfminer-20110515 pexpect-2.4 pipe2py-0.9.2 pyOpenSSL-0.13 pycrypto-2.5 pycurl-7.19.0 pydot-1.0.2 pyephem-3.7.5.1 pyparsing-1.5.6 pyquery-1.0 pyth-0.5.6 python-Levenshtein-0.10.2 python-dateutil-1.5 python-editor-1.0.1 python-gflags-2.0 python-modargs-1.2 python-slugify-1.2.1 python-stdnum-0.7 pytz-2011k rdflib-3.1.0 requests-1.0.4 requests-foauth-0.1.1 scrapely-0.9 scraperwiki selenium-2.5.0 simplejson-2.2.1 suds-0.4 tweepy-1.7.1 tweetstream-1.1.1 w3lib-1.0 xlrd-0.7.1 xlutils-1.4.1 xlwt-0.7.2 xmltodict-0.4 zope.interface-3.8.0   ! Hello! It looks like your application is using an outdated version of Python.  ! This caused the security warning you saw above during the 'pip install' step.  ! We recommend 'python-2.7.12', which you can specify in a 'runtime.txt' file.  ! -- Much Love, Heroku.   ! Hello! Your requirements.txt file contains the distribute package.  ! This library is automatically installed by Heroku and shouldn't be in  ! Your requirements.txt file. This can cause unexpected behavior.  ! -- Much Love, Heroku.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... E5040_HLBC_gov_2016_08 E5040_HLBC_gov_2016_07 E5040_HLBC_gov_2016_06 E5040_HLBC_gov_2016_05 E5040_HLBC_gov_2016_04 E5040_HLBC_gov_2016_03 E5040_HLBC_gov_2016_02 E5040_HLBC_gov_2016_01 E5040_HLBC_gov_2015_12 E5040_HLBC_gov_2015_11 E5040_HLBC_gov_2015_10 E5040_HLBC_gov_2015_09 E5040_HLBC_gov_2015_08 E5040_HLBC_gov_2015_07 E5040_HLBC_gov_2015_06 E5040_HLBC_gov_2015_05 E5040_HLBC_gov_2015_04 E5040_HLBC_gov_2015_03 E5040_HLBC_gov_2015_02 E5040_HLBC_gov_2015_01 E5040_HLBC_gov_2014_12 E5040_HLBC_gov_2014_11 E5040_HLBC_gov_2014_10 E5040_HLBC_gov_2014_09 E5040_HLBC_gov_2014_08 E5040_HLBC_gov_2014_07 E5040_HLBC_gov_2014_06 E5040_HLBC_gov_2014_05 E5040_HLBC_gov_2014_04 E5040_HLBC_gov_2014_03 E5040_HLBC_gov_2014_02 E5040_HLBC_gov_2014_01 E5040_HLBC_gov_2013_12 E5040_HLBC_gov_2013_11 E5040_HLBC_gov_2013_10 E5040_HLBC_gov_2013_09 E5040_HLBC_gov_2013_08 E5040_HLBC_gov_2013_07 E5040_HLBC_gov_2013_06 E5040_HLBC_gov_2013_05 E5040_HLBC_gov_2013_04 E5040_HLBC_gov_2013_03 E5040_HLBC_gov_2013_02 E5040_HLBC_gov_2013_01 E5040_HLBC_gov_2012_12 E5040_HLBC_gov_2012_11 E5040_HLBC_gov_2012_10 E5040_HLBC_gov_2012_09 E5040_HLBC_gov_2012_08 E5040_HLBC_gov_2012_07 E5040_HLBC_gov_2012_06 E5040_HLBC_gov_2012_05 E5040_HLBC_gov_2012_04 E5040_HLBC_gov_2012_03 E5040_HLBC_gov_2012_02 E5040_HLBC_gov_2012_01 E5040_HLBC_gov_2011_12 E5040_HLBC_gov_2011_11 E5040_HLBC_gov_2011_10 E5040_HLBC_gov_2011_09 E5040_HLBC_gov_2011_08 E5040_HLBC_gov_2011_07 E5040_HLBC_gov_2011_06 E5040_HLBC_gov_2011_05 E5040_HLBC_gov_2011_04 E5040_HLBC_gov_2011_03 E5040_HLBC_gov_2011_02 E5040_HLBC_gov_2011_01 E5040_HLBC_gov_2010_12

Data

Downloaded 1 time by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (48 KB) Use the API

rows 10 / 117

l f d
E5040_HLBC_gov_2014_12
2016-04-04 06:32:16.009806
E5040_HLBC_gov_2014_11
2016-04-04 06:32:21.208965
E5040_HLBC_gov_2014_10
2016-04-04 06:32:25.621117
E5040_HLBC_gov_2014_09
2016-04-04 06:32:30.992623
E5040_HLBC_gov_2014_08
2016-04-04 06:32:35.569912
E5040_HLBC_gov_2014_06
2016-04-04 06:32:45.732051
E5040_HLBC_gov_2014_05
2016-04-04 06:32:51.382260
E5040_HLBC_gov_2014_04
2016-04-04 06:32:56.141856
E5040_HLBC_gov_2014_03
2016-04-04 06:33:00.297328
E5040_HLBC_gov_2014_02
2016-04-04 06:33:05.119679

Statistics

Average successful run time: 7 minutes

Total run time: 17 minutes

Total cpu time used: half a minute

Total disk space used: 85.8 KB

History

  • Manually ran revision 824b8c8a and completed successfully .
    69 records added, 13 records removed in the database
  • Manually ran revision 297a18b0 and failed .
    nothing changed in the database
    62 pages scraped
  • Manually ran revision 297a18b0 and completed successfully .
    61 records added in the database
    123 pages scraped
  • Created on morph.io

Scraper code

Python

sp_E5040_HLBC_gov / scraper.py