This is a scraper that runs on Morph. To get started see the documentation

Contributors SimKennedy woodbine blablupcom

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.13, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.13 -----> Installing pip -----> Noticed cffi. Bootstrapping libffi. -----> Installing requirements with pip  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 2))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to /app/.heroku/src/scraperwiki  Collecting BeautifulSoup==3.2.0 (from -r /tmp/build/requirements.txt (line 9))  Downloading BeautifulSoup-3.2.0.tar.gz  Collecting Creoleparser==0.7.4 (from -r /tmp/build/requirements.txt (line 10))  Downloading Creoleparser-0.7.4.zip  Collecting Genshi==0.6 (from -r /tmp/build/requirements.txt (line 11))  Downloading Genshi-0.6.tar.gz (433kB)  Collecting Jinja2==2.6 (from -r /tmp/build/requirements.txt (line 12))  Downloading Jinja2-2.6.tar.gz (389kB)  Collecting Markdown==2.2.0 (from -r /tmp/build/requirements.txt (line 13))  Downloading Markdown-2.2.0.tar.gz (236kB)  Collecting Pygments==1.4 (from -r /tmp/build/requirements.txt (line 14))  Downloading Pygments-1.4.tar.gz (3.5MB)  Collecting SQLAlchemy==0.6.6 (from -r /tmp/build/requirements.txt (line 15))  Downloading SQLAlchemy-0.6.6.tar.gz (2.1MB)  Collecting Twisted==11.1.0 (from -r /tmp/build/requirements.txt (line 16))  Downloading Twisted-11.1.0.tar.bz2 (2.8MB)  Collecting Unidecode==0.04.9 (from -r /tmp/build/requirements.txt (line 17))  Downloading Unidecode-0.04.9.tar.gz (196kB)  Collecting anyjson==0.3.3 (from -r /tmp/build/requirements.txt (line 18))  Downloading anyjson-0.3.3.tar.gz  Collecting argparse==1.2.1 (from -r /tmp/build/requirements.txt (line 19))  Downloading argparse-1.2.1.tar.gz (69kB)  Collecting beautifulsoup4==4.1.3 (from -r /tmp/build/requirements.txt (line 20))  Downloading beautifulsoup4-4.1.3.tar.gz (58kB)  Collecting bitlyapi==0.1.1 (from -r /tmp/build/requirements.txt (line 21))  Downloading bitlyapi-0.1.1.tar.gz  Collecting blinker==1.2 (from -r /tmp/build/requirements.txt (line 22))  Downloading blinker-1.2.tar.gz (66kB)  Collecting cartodb==0.6 (from -r /tmp/build/requirements.txt (line 23))  Downloading cartodb-0.6.tar.gz  Collecting certifi==0.0.8 (from -r /tmp/build/requirements.txt (line 24))  Downloading certifi-0.0.8.tar.gz (118kB)  Collecting chardet==2.1.1 (from -r /tmp/build/requirements.txt (line 25))  Downloading chardet-2.1.1.tar.gz (178kB)  Collecting ckanclient==0.10 (from -r /tmp/build/requirements.txt (line 26))  Downloading ckanclient-0.10.tar.gz  Collecting colormath==1.0.8 (from -r /tmp/build/requirements.txt (line 27))  Downloading colormath-1.0.8.tar.gz  Collecting csvkit==0.3.0 (from -r /tmp/build/requirements.txt (line 28))  Downloading csvkit-0.3.0.tar.gz  Collecting dataset==0.5.2 (from -r /tmp/build/requirements.txt (line 29))  Downloading dataset-0.5.2.tar.gz  Collecting demjson==1.6 (from -r /tmp/build/requirements.txt (line 30))  Downloading demjson-1.6.tar.gz (64kB)  Collecting dropbox==1.4 (from -r /tmp/build/requirements.txt (line 31))  Downloading dropbox-1.4.tar.gz  Collecting errorhandler==1.1.1 (from -r /tmp/build/requirements.txt (line 32))  Downloading errorhandler-1.1.1.tar.gz  Collecting feedparser==5.0.1 (from -r /tmp/build/requirements.txt (line 33))  Downloading feedparser-5.0.1.zip (2.0MB)  Collecting fluidinfo.py==1.1.2 (from -r /tmp/build/requirements.txt (line 34))  Downloading fluidinfo.py-1.1.2.tar.gz  Collecting gdata==2.0.15 (from -r /tmp/build/requirements.txt (line 35))  Downloading gdata-2.0.15.tar.gz (2.0MB)  Collecting geopy==0.94.1 (from -r /tmp/build/requirements.txt (line 36))  Downloading geopy-0.94.1.tar.gz  Collecting gevent==0.13.6 (from -r /tmp/build/requirements.txt (line 37))  Downloading gevent-0.13.6.tar.gz (289kB)  Collecting google-api-python-client==1.0beta8 (from -r /tmp/build/requirements.txt (line 38))  Downloading google-api-python-client-1.0beta8.zip (631kB)  Collecting googlemaps==1.0.2 (from -r /tmp/build/requirements.txt (line 39))  Downloading googlemaps-1.0.2.tar.gz (60kB)  Collecting greenlet==0.3.2 (from -r /tmp/build/requirements.txt (line 40))  Downloading greenlet-0.3.2.zip (50kB)  Collecting html5lib==0.90 (from -r /tmp/build/requirements.txt (line 41))  Downloading html5lib-0.90.tar.gz (86kB)  Collecting httplib2==0.7.4 (from -r /tmp/build/requirements.txt (line 42))  Downloading httplib2-0.7.4.tar.gz (106kB)  Collecting imposm.parser==1.0.3 (from -r /tmp/build/requirements.txt (line 43))  Downloading imposm.parser-1.0.3.tar.gz  Collecting jellyfish==0.2.0 (from -r /tmp/build/requirements.txt (line 44))  Downloading jellyfish-0.2.0.tar.gz  Collecting mechanize==0.2.5 (from -r /tmp/build/requirements.txt (line 45))  Downloading mechanize-0.2.5.tar.gz (383kB)  Collecting mock==0.7.2 (from -r /tmp/build/requirements.txt (line 46))  Downloading mock-0.7.2.zip (938kB)  Collecting networkx==1.6 (from -r /tmp/build/requirements.txt (line 47))  Downloading networkx-1.6.zip (1.0MB)  Collecting ngram==3.3.0 (from -r /tmp/build/requirements.txt (line 48))  Downloading ngram-3.3.0.tar.gz  Collecting nose==1.1.2 (from -r /tmp/build/requirements.txt (line 49))  Downloading nose-1.1.2.tar.gz (729kB)  Collecting oauth2==1.5.170 (from -r /tmp/build/requirements.txt (line 50))  Downloading oauth2-1.5.170.tar.gz  Collecting oauth==1.0.1 (from -r /tmp/build/requirements.txt (line 51))  Downloading oauth-1.0.1.tar.gz  Collecting oauthlib==0.1.2 (from -r /tmp/build/requirements.txt (line 52))  Downloading oauthlib-0.1.2.tar.gz  Collecting openpyxl==1.5.7 (from -r /tmp/build/requirements.txt (line 53))  Downloading openpyxl-1.5.7.tar.gz (67kB)  Collecting ordereddict==1.1 (from -r /tmp/build/requirements.txt (line 54))  Downloading ordereddict-1.1.tar.gz  Collecting pbkdf2==1.3 (from -r /tmp/build/requirements.txt (line 55))  Downloading pbkdf2-1.3.tar.gz  Collecting pdfminer==20110515 (from -r /tmp/build/requirements.txt (line 56))  Downloading pdfminer-20110515.tar.gz (4.1MB)  Collecting pexpect==2.4 (from -r /tmp/build/requirements.txt (line 57))  Downloading pexpect-2.4.tar.gz (113kB)  Collecting pipe2py==0.9.2 (from -r /tmp/build/requirements.txt (line 58))  Downloading pipe2py-0.9.2.tar.gz (57kB)  Collecting pyOpenSSL==0.13 (from -r /tmp/build/requirements.txt (line 59))  Downloading pyOpenSSL-0.13.tar.gz (250kB)  Collecting pycrypto==2.5 (from -r /tmp/build/requirements.txt (line 60))  Downloading pycrypto-2.5.tar.gz (426kB)  Collecting pycurl==7.19.0 (from -r /tmp/build/requirements.txt (line 61))  Downloading pycurl-7.19.0.tar.gz (70kB)  Collecting pyephem==3.7.5.1 (from -r /tmp/build/requirements.txt (line 62))  Downloading pyephem-3.7.5.1.tar.gz (703kB)  Collecting pyparsing==1.5.6 (from -r /tmp/build/requirements.txt (line 63))  Downloading pyparsing-1.5.6.zip (1.7MB)  Collecting pyth==0.5.6 (from -r /tmp/build/requirements.txt (line 64))  Downloading pyth-0.5.6.tar.gz  Collecting python-Levenshtein==0.10.2 (from -r /tmp/build/requirements.txt (line 65))  Downloading python-Levenshtein-0.10.2.tar.gz (45kB)  Collecting python-dateutil==1.5 (from -r /tmp/build/requirements.txt (line 66))  Downloading python-dateutil-1.5.tar.gz (233kB)  Collecting python-gflags==2.0 (from -r /tmp/build/requirements.txt (line 67))  Downloading python-gflags-2.0.tar.gz (65kB)  Collecting python-modargs==1.2 (from -r /tmp/build/requirements.txt (line 68))  Downloading python-modargs-1.2.tar.gz  Collecting python-stdnum==0.7 (from -r /tmp/build/requirements.txt (line 69))  Downloading python-stdnum-0.7.tar.gz (113kB)  Collecting pytz==2011k (from -r /tmp/build/requirements.txt (line 70))  Downloading pytz-2011k.zip (517kB)  Collecting rdflib==3.1.0 (from -r /tmp/build/requirements.txt (line 71))  Downloading rdflib-3.1.0.tar.gz (249kB)  Collecting requests-foauth==0.1.1 (from -r /tmp/build/requirements.txt (line 72))  Downloading requests-foauth-0.1.1.tar.gz  Collecting requests==1.0.4 (from -r /tmp/build/requirements.txt (line 73))  Downloading requests-1.0.4.tar.gz (336kB)  Collecting selenium==2.5.0 (from -r /tmp/build/requirements.txt (line 74))  Downloading selenium-2.5.0.tar.gz (2.4MB)  Collecting simplejson==2.2.1 (from -r /tmp/build/requirements.txt (line 75))  Downloading simplejson-2.2.1.tar.gz (49kB)  Collecting suds==0.4 (from -r /tmp/build/requirements.txt (line 76))  Downloading suds-0.4.tar.gz (104kB)  Collecting tweepy==1.7.1 (from -r /tmp/build/requirements.txt (line 77))  Downloading tweepy-1.7.1.tar.gz  Collecting tweetstream==1.1.1 (from -r /tmp/build/requirements.txt (line 78))  Downloading tweetstream-1.1.1.tar.gz  Collecting w3lib==1.0 (from -r /tmp/build/requirements.txt (line 79))  Downloading w3lib-1.0.tar.gz  Collecting xlrd==0.7.1 (from -r /tmp/build/requirements.txt (line 81))  Downloading xlrd-0.7.1.zip (125kB)  Collecting xlutils==1.4.1 (from -r /tmp/build/requirements.txt (line 82))  Downloading xlutils-1.4.1.tar.gz (40kB)  Collecting xlwt==0.7.2 (from -r /tmp/build/requirements.txt (line 83))  Downloading xlwt-0.7.2.zip (131kB)  Collecting xmltodict==0.4 (from -r /tmp/build/requirements.txt (line 84))  Downloading xmltodict-0.4.tar.gz  Collecting zope.interface==3.8.0 (from -r /tmp/build/requirements.txt (line 88))  Downloading zope.interface-3.8.0.tar.gz (111kB)  Collecting lxml==2.3.3 (from -r /tmp/build/requirements.txt (line 89))  Downloading lxml-2.3.3.tar.gz (3.1MB)  Collecting chromium-compact-language-detector==0.031415 (from -r /tmp/build/requirements.txt (line 90))  Downloading chromium_compact_language_detector-0.031415.tar.gz (2.2MB)  Collecting icalendar==3.0.1b1 (from -r /tmp/build/requirements.txt (line 93))  Downloading icalendar-3.0.1b1.tar.gz  Collecting pyquery==1.0 (from -r /tmp/build/requirements.txt (line 96))  Downloading pyquery-1.0.tar.gz  Collecting scrapely==0.9 (from -r /tmp/build/requirements.txt (line 98))  Downloading scrapely-0.9.tar.gz  Collecting Fom==0.9.8 (from -r /tmp/build/requirements.txt (line 103))  Downloading Fom-0.9.8.zip (79kB)  Collecting PyYAML==3.10 (from -r /tmp/build/requirements.txt (line 105))  Downloading PyYAML-3.10.tar.gz (241kB)  Collecting Scrapy==0.14.1 (from -r /tmp/build/requirements.txt (line 107))  Downloading Scrapy-0.14.1.tar.gz (719kB)  Collecting adspygoogle.adwords==15.6.2 (from -r /tmp/build/requirements.txt (line 109))  Downloading adspygoogle.adwords-15.6.2.tar.gz (166kB)  Collecting nltk==3.0.2 (from -r /tmp/build/requirements.txt (line 111))  Downloading nltk-3.0.2.tar.gz (991kB)  Collecting pydot==1.0.2 (from -r /tmp/build/requirements.txt (line 113))  Downloading pydot-1.0.2.tar.gz  Collecting M2Crypto==0.22.3 (from -r /tmp/build/requirements.txt (line 115))  Downloading M2Crypto-0.22.3.tar.gz (74kB)  Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 2))  Downloading dumptruck-0.1.6.tar.gz  Collecting alembic>=0.6.2 (from dataset==0.5.2->-r /tmp/build/requirements.txt (line 29))  Downloading alembic-0.9.6.tar.gz (998kB)  Collecting python-slugify>=0.0.6 (from dataset==0.5.2->-r /tmp/build/requirements.txt (line 29))  Downloading python_slugify-1.2.4-py2.py3-none-any.whl  Collecting distribute (from python-stdnum==0.7->-r /tmp/build/requirements.txt (line 69))  Downloading distribute-0.7.3.zip (145kB)  Collecting numpy (from scrapely==0.9->-r /tmp/build/requirements.txt (line 98))  Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)  Collecting Mako (from alembic>=0.6.2->dataset==0.5.2->-r /tmp/build/requirements.txt (line 29))  Downloading Mako-1.0.7.tar.gz (564kB)  Collecting python-editor>=0.3 (from alembic>=0.6.2->dataset==0.5.2->-r /tmp/build/requirements.txt (line 29))  Downloading python-editor-1.0.3.tar.gz  Collecting MarkupSafe>=0.9.2 (from Mako->alembic>=0.6.2->dataset==0.5.2->-r /tmp/build/requirements.txt (line 29))  Downloading MarkupSafe-1.0.tar.gz  Installing collected packages: dumptruck, requests, scraperwiki, BeautifulSoup, Genshi, Creoleparser, Jinja2, Markdown, Pygments, SQLAlchemy, zope.interface, Twisted, Unidecode, anyjson, argparse, beautifulsoup4, bitlyapi, blinker, httplib2, oauth2, cartodb, certifi, chardet, ckanclient, colormath, xlrd, python-dateutil, csvkit, MarkupSafe, Mako, python-editor, alembic, python-slugify, PyYAML, dataset, demjson, oauth, simplejson, dropbox, errorhandler, feedparser, fluidinfo.py, gdata, geopy, greenlet, gevent, python-gflags, google-api-python-client, googlemaps, html5lib, imposm.parser, jellyfish, mechanize, mock, networkx, ngram, nose, pycrypto, oauthlib, openpyxl, ordereddict, pbkdf2, pdfminer, pexpect, pipe2py, pyOpenSSL, pycurl, pyephem, pyparsing, pyth, python-Levenshtein, python-modargs, distribute, python-stdnum, pytz, rdflib, requests-foauth, selenium, suds, tweepy, tweetstream, w3lib, xlwt, xlutils, xmltodict, lxml, chromium-compact-language-detector, icalendar, pyquery, numpy, scrapely, Fom, Scrapy, adspygoogle.adwords, nltk, pydot, M2Crypto  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py install for requests: started  Running setup.py install for requests: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for BeautifulSoup: started  Running setup.py install for BeautifulSoup: finished with status 'done'  Running setup.py install for Genshi: started  Running setup.py install for Genshi: finished with status 'done'  Running setup.py install for Creoleparser: started  Running setup.py install for Creoleparser: finished with status 'done'  Running setup.py install for Jinja2: started  Running setup.py install for Jinja2: finished with status 'done'  Running setup.py install for Markdown: started  Running setup.py install for Markdown: finished with status 'done'  Running setup.py install for Pygments: started  Running setup.py install for Pygments: finished with status 'done'  Running setup.py install for SQLAlchemy: started  Running setup.py install for SQLAlchemy: finished with status 'done'  Running setup.py install for zope.interface: started  Running setup.py install for zope.interface: finished with status 'done'  Running setup.py install for Twisted: started  Running setup.py install for Twisted: finished with status 'done'  Running setup.py install for Unidecode: started  Running setup.py install for Unidecode: finished with status 'done'  Running setup.py install for anyjson: started  Running setup.py install for anyjson: finished with status 'done'  Running setup.py install for argparse: started  Running setup.py install for argparse: finished with status 'done'  Running setup.py install for beautifulsoup4: started  Running setup.py install for beautifulsoup4: finished with status 'done'  Running setup.py install for bitlyapi: started  Running setup.py install for bitlyapi: finished with status 'done'  Running setup.py install for blinker: started  Running setup.py install for blinker: finished with status 'done'  Running setup.py install for httplib2: started  Running setup.py install for httplib2: finished with status 'done'  Running setup.py install for oauth2: started  Running setup.py install for oauth2: finished with status 'done'  Running setup.py install for cartodb: started  Running setup.py install for cartodb: finished with status 'done'  Running setup.py install for certifi: started  Running setup.py install for certifi: finished with status 'done'  Running setup.py install for chardet: started  Running setup.py install for chardet: finished with status 'done'  Running setup.py install for ckanclient: started  Running setup.py install for ckanclient: finished with status 'done'  Running setup.py install for colormath: started  Running setup.py install for colormath: finished with status 'done'  Running setup.py install for xlrd: started  Running setup.py install for xlrd: finished with status 'done'  Running setup.py install for python-dateutil: started  Running setup.py install for python-dateutil: finished with status 'done'  Running setup.py install for csvkit: started  Running setup.py install for csvkit: finished with status 'done'  Running setup.py install for MarkupSafe: started  Running setup.py install for MarkupSafe: finished with status 'done'  Running setup.py install for Mako: started  Running setup.py install for Mako: finished with status 'done'  Running setup.py install for python-editor: started  Running setup.py install for python-editor: finished with status 'done'  Running setup.py install for alembic: started  Running setup.py install for alembic: finished with status 'done'  Running setup.py install for PyYAML: started  Running setup.py install for PyYAML: finished with status 'done'  Running setup.py install for dataset: started  Running setup.py install for dataset: finished with status 'done'  Running setup.py install for demjson: started  Running setup.py install for demjson: finished with status 'done'  Running setup.py install for oauth: started  Running setup.py install for oauth: finished with status 'done'  Running setup.py install for simplejson: started  Running setup.py install for simplejson: finished with status 'done'  Running setup.py install for dropbox: started  Running setup.py install for dropbox: finished with status 'done'  Running setup.py install for errorhandler: started  Running setup.py install for errorhandler: finished with status 'done'  Running setup.py install for feedparser: started  Running setup.py install for feedparser: finished with status 'done'  Running setup.py install for fluidinfo.py: started  Running setup.py install for fluidinfo.py: finished with status 'done'  Running setup.py install for gdata: started  Running setup.py install for gdata: finished with status 'done'  Running setup.py install for geopy: started  Running setup.py install for geopy: finished with status 'done'  Running setup.py install for greenlet: started  Running setup.py install for greenlet: finished with status 'done'  Running setup.py install for gevent: started  Running setup.py install for gevent: finished with status 'done'  Running setup.py install for python-gflags: started  Running setup.py install for python-gflags: finished with status 'done'  Running setup.py install for google-api-python-client: started  Running setup.py install for google-api-python-client: finished with status 'done'  Running setup.py install for googlemaps: started  Running setup.py install for googlemaps: finished with status 'done'  Running setup.py install for html5lib: started  Running setup.py install for html5lib: finished with status 'done'  Running setup.py install for imposm.parser: started  Running setup.py install for imposm.parser: finished with status 'done'  Running setup.py install for jellyfish: started  Running setup.py install for jellyfish: finished with status 'done'  Running setup.py install for mechanize: started  Running setup.py install for mechanize: finished with status 'done'  Running setup.py install for mock: started  Running setup.py install for mock: finished with status 'done'  Running setup.py install for networkx: started  Running setup.py install for networkx: finished with status 'done'  Running setup.py install for ngram: started  Running setup.py install for ngram: finished with status 'done'  Running setup.py install for nose: started  Running setup.py install for nose: finished with status 'done'  Running setup.py install for pycrypto: started  Running setup.py install for pycrypto: finished with status 'done'  Running setup.py install for oauthlib: started  Running setup.py install for oauthlib: finished with status 'done'  Running setup.py install for openpyxl: started  Running setup.py install for openpyxl: finished with status 'done'  Running setup.py install for ordereddict: started  Running setup.py install for ordereddict: finished with status 'done'  Running setup.py install for pbkdf2: started  Running setup.py install for pbkdf2: finished with status 'done'  Running setup.py install for pdfminer: started  Running setup.py install for pdfminer: finished with status 'done'  Running setup.py install for pexpect: started  Running setup.py install for pexpect: finished with status 'done'  Running setup.py install for pipe2py: started  Running setup.py install for pipe2py: finished with status 'done'  Running setup.py install for pyOpenSSL: started  Running setup.py install for pyOpenSSL: finished with status 'done'  Running setup.py install for pycurl: started  Running setup.py install for pycurl: finished with status 'done'  Running setup.py install for pyephem: started  Running setup.py install for pyephem: finished with status 'done'  Running setup.py install for pyparsing: started  Running setup.py install for pyparsing: finished with status 'done'  Running setup.py install for pyth: started  Running setup.py install for pyth: finished with status 'done'  Running setup.py install for python-Levenshtein: started  Running setup.py install for python-Levenshtein: finished with status 'done'  Running setup.py install for python-modargs: started  Running setup.py install for python-modargs: finished with status 'done'  Running setup.py install for distribute: started  Running setup.py install for distribute: finished with status 'done'  Running setup.py install for python-stdnum: started  Running setup.py install for python-stdnum: finished with status 'done'  Running setup.py install for pytz: started  Running setup.py install for pytz: finished with status 'done'  Running setup.py install for rdflib: started  Running setup.py install for rdflib: finished with status 'done'  Running setup.py install for requests-foauth: started  Running setup.py install for requests-foauth: finished with status 'done'  Running setup.py install for selenium: started  Running setup.py install for selenium: finished with status 'done'  Running setup.py install for suds: started  Running setup.py install for suds: finished with status 'done'  Running setup.py install for tweepy: started  Running setup.py install for tweepy: finished with status 'done'  Running setup.py install for tweetstream: started  Running setup.py install for tweetstream: finished with status 'done'  Running setup.py install for w3lib: started  Running setup.py install for w3lib: finished with status 'done'  Running setup.py install for xlwt: started  Running setup.py install for xlwt: finished with status 'done'  Running setup.py install for xlutils: started  Running setup.py install for xlutils: finished with status 'done'  Running setup.py install for xmltodict: started  Running setup.py install for xmltodict: finished with status 'done'  Running setup.py install for lxml: started  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for chromium-compact-language-detector: started  Running setup.py install for chromium-compact-language-detector: finished with status 'done'  Running setup.py install for icalendar: started  Running setup.py install for icalendar: finished with status 'done'  Running setup.py install for pyquery: started  Running setup.py install for pyquery: finished with status 'done'  Running setup.py install for scrapely: started  Running setup.py install for scrapely: finished with status 'done'  Running setup.py install for Fom: started  Running setup.py install for Fom: finished with status 'done'  Running setup.py install for Scrapy: started  Running setup.py install for Scrapy: finished with status 'done'  Running setup.py install for adspygoogle.adwords: started  Running setup.py install for adspygoogle.adwords: finished with status 'done'  Running setup.py install for nltk: started  Running setup.py install for nltk: finished with status 'done'  Running setup.py install for pydot: started  Running setup.py install for pydot: finished with status 'done'  Running setup.py install for M2Crypto: started  Running setup.py install for M2Crypto: finished with status 'done'  Successfully installed BeautifulSoup-3.2.0 Creoleparser-0.7.4 Fom-0.9.8 Genshi-0.6 Jinja2-2.6 M2Crypto-0.22.3 Mako-1.0.7 Markdown-2.2.0 MarkupSafe-1.0 PyYAML-3.10 Pygments-1.4 SQLAlchemy-0.6.6 Scrapy-0.14.1 Twisted-11.1.0 Unidecode-0.4.9 adspygoogle.adwords-15.6.2 alembic-0.9.6 anyjson-0.3.3 argparse-1.2.1 beautifulsoup4-4.1.3 bitlyapi-0.1.1 blinker-1.2 cartodb-0.6 certifi-0.0.8 chardet-2.1.1 chromium-compact-language-detector-0.31415 ckanclient-0.10 colormath-1.0.8 csvkit-0.3.0 dataset-0.5.2 demjson-1.6 distribute-0.7.3 dropbox-1.4 dumptruck-0.1.6 errorhandler-1.1.1 feedparser-5.0.1 fluidinfo.py-1.1.2 gdata-2.0.15 geopy-0.94.1 gevent-0.13.6 google-api-python-client-1.0b8 googlemaps-1.0.2 greenlet-0.3.2 html5lib-0.90 httplib2-0.7.4 icalendar-3.0.1b1 imposm.parser-1.0.3 jellyfish-0.2.0 lxml-2.3.3 mechanize-0.2.5 mock-0.7.2 networkx-1.6 ngram-3.3.0 nltk-3.0.2 nose-1.1.2 numpy-1.13.3 oauth-1.0.1 oauth2-1.5.170 oauthlib-0.1.2 openpyxl-1.5.7 ordereddict-1.1 pbkdf2-1.3 pdfminer-20110515 pexpect-2.4 pipe2py-0.9.2 pyOpenSSL-0.13 pycrypto-2.5 pycurl-7.19.0 pydot-1.0.2 pyephem-3.7.5.1 pyparsing-1.5.6 pyquery-1.0 pyth-0.5.6 python-Levenshtein-0.10.2 python-dateutil-1.5 python-editor-1.0.3 python-gflags-2.0 python-modargs-1.2 python-slugify-1.2.4 python-stdnum-0.7 pytz-2011k rdflib-3.1.0 requests-1.0.4 requests-foauth-0.1.1 scrapely-0.9 scraperwiki selenium-2.5.0 simplejson-2.2.1 suds-0.4 tweepy-1.7.1 tweetstream-1.1.1 w3lib-1.0 xlrd-0.7.1 xlutils-1.4.1 xlwt-0.7.2 xmltodict-0.4 zope.interface-3.8.0   ! Hello! Your requirements.txt file contains the distribute package.  ! This library is automatically installed by Heroku and shouldn't be in  ! Your requirements.txt file. This can cause unexpected behavior.  ! -- Much Love, Heroku.  -----> Downloading NLTK corpora…  ! 'nltk.txt' not found, not downloading any corpora  ! Learn more: https://devcenter.heroku.com/articles/python-nltk  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... E5043_KUTCRBO_gov_2017_04 E5043_KUTCRBO_gov_2017_05 E5043_KUTCRBO_gov_2017_06 E5043_KUTCRBO_gov_2017_07 E5043_KUTCRBO_gov_2017_08 E5043_KUTCRBO_gov_2017_09 E5043_KUTCRBO_gov_2017_10 E5043_KUTCRBO_gov_2016_04 E5043_KUTCRBO_gov_2016_05 E5043_KUTCRBO_gov_2016_06 E5043_KUTCRBO_gov_2016_07 E5043_KUTCRBO_gov_2016_08 E5043_KUTCRBO_gov_2016_09 E5043_KUTCRBO_gov_2016_10 E5043_KUTCRBO_gov_2016_11 E5043_KUTCRBO_gov_2016_12 E5043_KUTCRBO_gov_2017_01 E5043_KUTCRBO_gov_2017_02 E5043_KUTCRBO_gov_2017_03 E5043_KUTCRBO_gov_2015_04 E5043_KUTCRBO_gov_2015_05 E5043_KUTCRBO_gov_2015_06 E5043_KUTCRBO_gov_2015_07 E5043_KUTCRBO_gov_2015_08 E5043_KUTCRBO_gov_2015_09 E5043_KUTCRBO_gov_2015_10 E5043_KUTCRBO_gov_2015_11 E5043_KUTCRBO_gov_2015_12 E5043_KUTCRBO_gov_2016_01 E5043_KUTCRBO_gov_2016_02 E5043_KUTCRBO_gov_2016_03 E5043_KUTCRBO_gov_2014_04 E5043_KUTCRBO_gov_2014_05 E5043_KUTCRBO_gov_2014_06 E5043_KUTCRBO_gov_2014_07 E5043_KUTCRBO_gov_2014_08 E5043_KUTCRBO_gov_2014_09 E5043_KUTCRBO_gov_2014_10 E5043_KUTCRBO_gov_2014_11 E5043_KUTCRBO_gov_2014_12 E5043_KUTCRBO_gov_2015_01 E5043_KUTCRBO_gov_2015_02 E5043_KUTCRBO_gov_2015_03 E5043_KUTCRBO_gov_2013_04 E5043_KUTCRBO_gov_2013_05 E5043_KUTCRBO_gov_2013_06 E5043_KUTCRBO_gov_2013_07 E5043_KUTCRBO_gov_2013_08 E5043_KUTCRBO_gov_2013_09 E5043_KUTCRBO_gov_2013_10 E5043_KUTCRBO_gov_2013_11 E5043_KUTCRBO_gov_2013_12 E5043_KUTCRBO_gov_2014_01 E5043_KUTCRBO_gov_2014_02 E5043_KUTCRBO_gov_2014_03

Data

Downloaded 3 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (37 KB) Use the API

rows 10 / 101

Statistics

Average successful run time: 2 minutes

Total run time: 16 minutes

Total cpu time used: half a minute

Total disk space used: 82.6 KB

History

  • Manually ran revision b1315252 and completed successfully .
    55 records added, 55 records removed in the database
  • Manually ran revision b1315252 and completed successfully .
    55 records added in the database
  • Manually ran revision 8256ee8e and completed successfully .
    nothing changed in the database
  • Manually ran revision ad2ce173 and completed successfully .
    nothing changed in the database
  • Manually ran revision ad2ce173 and completed successfully .
    nothing changed in the database
  • Manually ran revision ad2ce173 and completed successfully .
    nothing changed in the database
  • Manually ran revision 6bd38bf1 and failed .
    nothing changed in the database
  • Manually ran revision 09409ec8 and failed .
    nothing changed in the database
  • Manually ran revision 09409ec8 and completed successfully .
    46 records added in the database
  • Manually ran revision acacbc02 and failed .
    34 records added, 30 records removed in the database
    108 pages scraped
  • Manually ran revision acacbc02 and completed successfully .
    31 records added in the database
    95 pages scraped
  • Created on morph.io

Scraper code

Python

sp_E5043_KUTCRBO_gov / scraper.py