planningalerts-scrapers / orange_city_council

Orange City Council Development Applications


Development applications for PlanningAlerts.

Contributors otherchirps

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.6, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.6 -----> Installing pip -----> Installing requirements with pip  Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 1))  /app/.heroku/python/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings  SNIMissingWarning  /app/.heroku/python/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings  InsecurePlatformWarning  /app/.heroku/python/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings  InsecurePlatformWarning  Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB)  Collecting scraperwiki==0.4.1 (from -r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/80/e5/4a50dc3c382a8f9ad21e72d143f8c0a2a7fd24d0e5a43f425e3d31edd1fe/scraperwiki-0.4.1.tar.gz  Collecting Scrapy==0.24.6 (from -r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/9c/b4/a84869ab40c33162827e26f72daa3cb04f54cbd3822d901f744f48447db5/Scrapy-0.24.6-py2-none-any.whl (444kB)  Collecting service-identity==14.0.0 (from -r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/98/1a/2bd789e1623a18a2e54ff57934d169e5f1599d2a719954ad67bbb8cb0f72/service_identity-14.0.0-py2.py3-none-any.whl  Collecting requests (from scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl (57kB)  Collecting sqlalchemy (from scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/e2/0a/05b7d13618ad41c108a6c2b886af83bf9bb7e35f8951227abb18b1330745/SQLAlchemy-1.2.14.tar.gz (5.7MB)  Collecting alembic (from scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/1c/65/b8e4f5b2f345bb13b5e0a3fddd892b0b3f0e8ad4880e954fdc6a50d00d84/alembic-1.0.5.tar.gz (1.0MB)  Collecting pyOpenSSL (from Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/96/af/9d29e6bd40823061aea2e0574ccb2fcf72bfd6130ce53d32773ec375458c/pyOpenSSL-18.0.0-py2.py3-none-any.whl (53kB)  Collecting six>=1.5.2 (from Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl  Collecting w3lib>=1.8.0 (from Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/37/94/40c93ad0cadac0f8cb729e1668823c71532fd4a7361b141aec535acb68e3/w3lib-1.19.0-py2.py3-none-any.whl  Collecting Twisted>=10.0.0 (from Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/5d/0e/a72d85a55761c2c3ff1cb968143a2fd5f360220779ed90e0fadf4106d4f2/Twisted-18.9.0.tar.bz2 (3.1MB)  Collecting queuelib (from Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/4c/85/ae64e9145f39dd6d14f8af3fa809a270ef3729f3b90b3c0cf5aa242ab0d4/queuelib-1.5.0-py2.py3-none-any.whl  Collecting cssselect>=0.9 (from Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl  Collecting pyasn1-modules (from service-identity==14.0.0->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/19/02/fa63f7ba30a0d7b925ca29d034510fc1ffde53264b71b4155022ddf3ab5d/pyasn1_modules-0.2.2-py2.py3-none-any.whl (62kB)  Collecting characteristic>=14.0.0 (from service-identity==14.0.0->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/fa/0d/7fa43a50feaa8896e9acbb91d2256dc1341e8f9a0be629138490ee1f849e/characteristic-14.3.0-py2.py3-none-any.whl  Collecting pyasn1 (from service-identity==14.0.0->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)  Collecting chardet<3.1.0,>=3.0.2 (from requests->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Collecting certifi>=2017.4.17 (from requests->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/9f/e0/accfc1b56b57e9750eba272e24c4dddeac86852c2bebd1236674d7887e8a/certifi-2018.11.29-py2.py3-none-any.whl (154kB)  Collecting urllib3<1.25,>=1.21.1 (from requests->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)  Collecting idna<2.8,>=2.5 (from requests->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)  Collecting Mako (from alembic->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz (564kB)  Collecting python-editor>=0.3 (from alembic->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/65/1e/adf6e000ea5dc909aa420352d6ba37f16434c8a3c2fa030445411a1ed545/python-editor-1.0.3.tar.gz  Collecting python-dateutil (from alembic->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/74/68/d87d9b36af36f44254a8d512cbfc48369103a3b9e474be9bdfe536abfc45/python_dateutil-2.7.5-py2.py3-none-any.whl (225kB)  Collecting cryptography>=2.2.1 (from pyOpenSSL->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/1a/06/93bc0c01c61d7845b45e6751640d1ab83132964bfb3e71187acf94dc7193/cryptography-2.4.2-cp27-cp27m-manylinux1_x86_64.whl (2.1MB)  Collecting zope.interface>=4.4.2 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/b0/74/084cc7f111480c4ad60eecd02ea7830bd39259e29e40dc6a69fbed4b6f09/zope.interface-4.6.0-cp27-cp27m-manylinux1_x86_64.whl (164kB)  Collecting constantly>=15.1 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl  Collecting incremental>=16.10.1 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl  Collecting Automat>=0.3.0 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl  Collecting hyperlink>=17.1.1 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl  Collecting PyHamcrest>=1.9.0 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl (52kB)  Collecting attrs>=17.4.0 (from Twisted>=10.0.0->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl  Collecting MarkupSafe>=0.9.2 (from Mako->alembic->scraperwiki==0.4.1->-r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/94/7a/34f53c66e0f9070b273c083d674581158426f2670cfd03f07fec375f0325/MarkupSafe-1.1.0-cp27-cp27m-manylinux1_x86_64.whl  Collecting enum34; python_version < "3" (from cryptography>=2.2.1->pyOpenSSL->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl  Collecting cffi!=1.11.3,>=1.7 (from cryptography>=2.2.1->pyOpenSSL->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/5d/a7/348bf05f004e7534012dc533ee29650d88fb25bf013988518e0acf6961fa/cffi-1.11.5-cp27-cp27m-manylinux1_x86_64.whl (407kB)  Collecting ipaddress; python_version < "3" (from cryptography>=2.2.1->pyOpenSSL->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl  Collecting asn1crypto>=0.21.0 (from cryptography>=2.2.1->pyOpenSSL->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)  Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography>=2.2.1->pyOpenSSL->Scrapy==0.24.6->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)  Installing collected packages: lxml, chardet, certifi, urllib3, idna, requests, sqlalchemy, MarkupSafe, Mako, python-editor, six, python-dateutil, alembic, scraperwiki, enum34, pycparser, cffi, ipaddress, asn1crypto, cryptography, pyOpenSSL, w3lib, zope.interface, constantly, incremental, attrs, Automat, hyperlink, PyHamcrest, Twisted, queuelib, cssselect, Scrapy, pyasn1, pyasn1-modules, characteristic, service-identity  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for sqlalchemy: started  Running setup.py install for sqlalchemy: finished with status 'done'  Running setup.py install for Mako: started  Running setup.py install for Mako: finished with status 'done'  Running setup.py install for python-editor: started  Running setup.py install for python-editor: finished with status 'done'  Running setup.py install for alembic: started  Running setup.py install for alembic: finished with status 'done'  Running setup.py install for scraperwiki: started  Running setup.py install for scraperwiki: finished with status 'done'  Running setup.py install for pycparser: started  Running setup.py install for pycparser: finished with status 'done'  Running setup.py install for Twisted: started  Running setup.py install for Twisted: finished with status 'done'  Successfully installed Automat-0.7.0 Mako-1.0.7 MarkupSafe-1.1.0 PyHamcrest-1.9.0 Scrapy-0.24.6 Twisted-18.9.0 alembic-1.0.5 asn1crypto-0.24.0 attrs-18.2.0 certifi-2018.11.29 cffi-1.11.5 characteristic-14.3.0 chardet-3.0.4 constantly-15.1.0 cryptography-2.4.2 cssselect-1.0.3 enum34-1.1.6 hyperlink-18.0.0 idna-2.7 incremental-17.5.0 ipaddress-1.0.22 lxml-3.4.4 pyOpenSSL-18.0.0 pyasn1-0.4.4 pyasn1-modules-0.2.2 pycparser-2.19 python-dateutil-2.7.5 python-editor-1.0.3 queuelib-1.5.0 requests-2.20.1 scraperwiki-0.4.1 service-identity-14.0.0 six-1.11.0 sqlalchemy-1.2.14 urllib3-1.24.1 w3lib-1.19.0 zope.interface-4.6.0   ! Hello! It looks like your application is using an outdated version of Python.  ! This caused the security warning you saw above during the 'pip install' step.  ! We recommend 'python-3.6.2', which you can specify in a 'runtime.txt' file.  ! -- Much Love, Heroku. /app/.heroku/python/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible. utils.DeprecatedIn23,    -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... /app/.heroku/python/lib/python2.7/site-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible. utils.DeprecatedIn23, 2018-12-09 11:37:48+0000 [planningalerts] INFO: Searching for requests between 2018-12-01 and 2018-12-31 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 451/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 450/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 438/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 441/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 443/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 449/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 184/2018 2018-12-09 11:37:53+0000 [planningalerts] INFO: Skipping previously saved application: 178/2018 2018-12-09 11:37:54+0000 [planningalerts] INFO: Skipping previously saved application: 442/2018 2018-12-09 11:37:55+0000 [planningalerts] INFO: Skipping previously saved application: 440/2018 2018-12-09 11:37:55+0000 [planningalerts] INFO: Skipping previously saved application: 446/2018 2018-12-09 11:37:55+0000 [planningalerts] INFO: Skipping previously saved application: 447/2018 2018-12-09 11:37:56+0000 [planningalerts] INFO: Skipping previously saved application: 448/2018 2018-12-09 11:37:56+0000 [planningalerts] INFO: Skipping previously saved application: 445/2018 2018-12-09 11:37:56+0000 [planningalerts] INFO: Skipping previously saved application: 439/2018 2018-12-09 11:37:56+0000 [planningalerts] INFO: Skipping previously saved application: 444/2018 2018-12-09 11:37:56+0000 [planningalerts] INFO: Closing spider (finished) 2018-12-09 11:37:56+0000 [planningalerts] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 11262, 'downloader/request_count': 18, 'downloader/request_method_count/GET': 18, 'downloader/response_bytes': 357032, 'downloader/response_count': 18, 'downloader/response_status_count/200': 18, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2018, 12, 9, 11, 37, 56, 160323), 'item_scraped_count': 16, 'memdebug/gc_garbage_count': 0, 'memdebug/live_refs/PlanningalertsSpider': 1, 'memusage/max': 61980672, 'memusage/startup': 61980672, 'request_depth_max': 2, 'response_received_count': 18, 'scheduler/dequeued': 18, 'scheduler/dequeued/memory': 18, 'scheduler/enqueued': 18, 'scheduler/enqueued/memory': 18, 'start_time': datetime.datetime(2018, 12, 9, 11, 37, 46, 994137)} 2018-12-09 11:37:56+0000 [planningalerts] INFO: Spider closed (finished)

Data

Downloaded 2273 times by openaustralia LoveMyData jesse-a-reilly TheZepto MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (606 KB) Use the API

rows 10 / 1699

date_scraped description info_url date_received council_reference address comment_url external_reference
2015-06-15
Shed (with attached awning)
2015-06-12
194/2015
62 Valencia Drive ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20194/2015
PR26767
2015-06-15
Dwelling and Attached Garage
2015-06-09
188/2015
24 Nicholls Lane HUNTLEY NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20188/2015
PR19429
2015-06-15
Housing for Seniors or People With a Disability (including a community centre, an indoor pool and a bowling green)
2015-06-09
501/2007
109 Ploughmans Lane ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20501/2007
PR20497
2015-06-15
Garage
2015-06-12
193/2015
96 March Street ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20193/2015
PR7592
2015-06-15
Subdivision (two lot residential)
2015-06-10
192/2015
152 Moulder Street ORANGE NSW 280080 Wade Place ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20192/2015
PR8762
2015-06-15
Dwelling Alterations & Additions (carport)
2015-06-03
186/2015
106 Sieben Drive ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20186/2015
PR11066
2015-06-15
Shed
2015-06-04
187/2015
19 Wallace Lane CANOBOLAS NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20187/2015
PR12317
2015-06-15
Garages/Outbuildings
2015-06-15
196/2015
170 Kite Street ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20196/2015
PR17637
2015-06-15
Dwelling Alterations & Additions
2015-06-10
189/2015
40 Kite Street ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20189/2015
PR20715
2015-06-15
Dwelling and Attached Garage
2015-06-03
183/2015
11 Japonica Place ORANGE NSW 2800
mailto:council@orange.nsw.gov.au?subject=Development%20Application%20Enquiry%3A%20183/2015
PR26705

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 637 KB

History

  • Auto ran revision ba090094 and completed successfully .
    nothing changed in the database
  • Auto ran revision ba090094 and completed successfully .
    nothing changed in the database
  • Auto ran revision ba090094 and completed successfully .
    2 records added in the database
  • Auto ran revision ba090094 and completed successfully .
    4 records added in the database
  • Auto ran revision ba090094 and completed successfully .
    2 records added in the database
    11 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

orange_city_council / scraper.py