planningalerts-scrapers / cessnock

Cessnock City Council Development Applications

Scrapes datracker.cessnock.nsw.gov.au and aplikace.policie.cz

Cessnock City Council home page.


Cessnock City Council scraper

  • Server - .NET - ApplicationMaster
  • Accept Terms - Yes
  • Cookie tracking - Yes and managed by PGBrowser
  • Pagnation - No
  • Javascript - No
  • UserAgent - No
  • Request actual DA page for data - No
  • Clearly defined data within a row - Yes
  • JSON - Yes, in the payload

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default) * thismonth * lastmonth * custom = yyyy-mm (eg. 2017-01)

Enjoy

Contributors LoveMyData henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.json; requirements  from dependencies in composer.lock will be used for selection  - php (7.2.12)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - nginx (1.8.1)  - apache (2.4.37) -----> Installing dependencies...  Composer version 1.5.2 2017-09-11 16:59:25  Loading composer repositories with package information  Installing dependencies from lock file  Package operations: 4 installs, 0 updates, 0 removals  - Installing pguardiario/phpuri (1.0): Downloading (connecting...)Downloading (100%)   - Installing electrolinux/phpquery (0.9.6): Downloading (connecting...)Downloading (100%)   - Installing byjg/pgbrowser (1.0.0): Downloading (connecting...)Downloading (100%)   - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253  Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data between 29/11/2018 and 09/12/2018, changable via MORPH_PERIOD environment Skipping already saved record 8/2018/918/1 Skipping already saved record 8/2018/911/1 Skipping already saved record 44/2018/576/1 Skipping already saved record 8/2016/80/2 Skipping already saved record 15/2018/117/1 Skipping already saved record 8/2018/917/1 Skipping already saved record 10/2018/818/1 Skipping already saved record 10/2018/745/1 Skipping already saved record 15/2018/116/1 Skipping already saved record 8/2018/916/1 Skipping already saved record 44/2018/575/1 Skipping already saved record 12/2018/7/2 Skipping already saved record 8/2018/915/1 Skipping already saved record 12/2017/642/2 Skipping already saved record 8/2016/554/2 Skipping already saved record 10/2018/890/1 Skipping already saved record 8/2018/890/1 Skipping already saved record 10/2018/908/1 Skipping already saved record 8/2018/908/1 Skipping already saved record 8/2018/879/1 Skipping already saved record 8/2018/893/1 Skipping already saved record 10/2018/910/1 Skipping already saved record 8/2018/910/1 Skipping already saved record 10/2018/907/1 Skipping already saved record 8/2018/907/1 Skipping already saved record 10/2018/905/1 Skipping already saved record 8/2018/905/1 Skipping already saved record 8/2018/903/1 Skipping already saved record 9/2018/304/1 Skipping already saved record 8/2018/902/1 Skipping already saved record 12/2017/351/1 Skipping already saved record 12/2017/266/1 Skipping already saved record 44/2018/574/1 Skipping already saved record 12/2017/687/1 Skipping already saved record 8/2018/914/1 Skipping already saved record 44/2018/573/1 Skipping already saved record 8/2018/913/1 Skipping already saved record 12/2018/53/1 Skipping already saved record 8/2018/912/1 Skipping already saved record 9/2018/305/1 Skipping already saved record 10/2018/587/1 Skipping already saved record 8/2018/909/1 Skipping already saved record 10/2018/747/1 Skipping already saved record 9/2018/303/1 Skipping already saved record 10/2018/824/1 Skipping already saved record 12/2017/240/4 Skipping already saved record 12/2013/273/2 Skipping already saved record 10/2018/806/1 Skipping already saved record 44/2018/572/1 Skipping already saved record 8/2018/883/1 Skipping already saved record 10/2018/774/2 Skipping already saved record 8/2018/906/1 Skipping already saved record 8/2018/904/1 Skipping already saved record 8/2018/897/1 Skipping already saved record 8/2018/880/1 Skipping already saved record 8/2018/887/1 Skipping already saved record 12/2018/120/1 Skipping already saved record 8/2018/888/1 Skipping already saved record 12/2016/394/8 Skipping already saved record 9/2018/302/1 Skipping already saved record 15/2018/113/1 Skipping already saved record 8/2018/900/1 Skipping already saved record 44/2018/571/1 Skipping already saved record 12/2018/66/1 Skipping already saved record 14/2013/735/1 Skipping already saved record 12/2017/652/1 Skipping already saved record 10/2018/391/1 Skipping already saved record 48/2018/15/1 Skipping already saved record 10/2018/733/1 Skipping already saved record 12/2018/203/1 Skipping already saved record 15/2018/115/1 Skipping already saved record 8/2018/694/2 Skipping already saved record 8/2018/884/1 Skipping already saved record 10/2018/882/1 Skipping already saved record 8/2018/882/1 Skipping already saved record 10/2018/885/1 Skipping already saved record 8/2018/885/1 Skipping already saved record 8/2018/886/1 Skipping already saved record 10/2018/891/1 Skipping already saved record 8/2018/891/1 Skipping already saved record 10/2018/898/1 Skipping already saved record 8/2018/898/1 Skipping already saved record 8/2018/881/1 Skipping already saved record 10/2018/901/1 Skipping already saved record 8/2018/901/1 Skipping already saved record 10/2018/899/1 Skipping already saved record 8/2018/899/1 Skipping already saved record 10/2018/779/1 Skipping already saved record 44/2018/565/1 Skipping already saved record 44/2018/558/1 Skipping already saved record 44/2018/562/1 Skipping already saved record 44/2018/560/1 Skipping already saved record 44/2018/561/1 Skipping already saved record 44/2018/563/1 Skipping already saved record 44/2018/564/1 Skipping already saved record 44/2018/566/1 Skipping already saved record 44/2018/568/1 Skipping already saved record 44/2018/569/1 Skipping already saved record 12/2018/211/1 Skipping already saved record 15/2018/114/1 Skipping already saved record 8/2018/889/1 Skipping already saved record 12/2018/176/1 Skipping already saved record 44/2018/570/1 Skipping already saved record 9/2018/301/1 Skipping already saved record 14/2017/636/1 Skipping already saved record 15/2018/112/1 Skipping already saved record 44/2018/567/1 Skipping already saved record 8/2018/892/1 Skipping already saved record 110/2018/32/1 Skipping already saved record 12/2016/470/1 Skipping already saved record 12/2018/131/2 Skipping already saved record 12/2018/26/1 Skipping already saved record 10/2018/832/1 Skipping already saved record 10/2018/629/1 Skipping already saved record 12/2017/98/1 Skipping already saved record 12/2018/147/1 Skipping already saved record 47/2018/18/1 Skipping already saved record 48/2018/14/1 Skipping already saved record 12/2016/38/2 Skipping already saved record 10/2018/453/1 Skipping already saved record 12/2012/413/2 Skipping already saved record 12/2012/413/1 Skipping already saved record 110/2018/31/1

Data

Downloaded 1234 times by openaustralia jesse-a-reilly TheZepto MikeRalphson gdmhunter

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.22 MB) Use the API

rows 10 / 7493

id council_reference address description info_url comment_url date_scraped date_received
1
DA 2015/563
4 Sale ST GRETA, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000563/001&Body=
2015-11-30
2015-11-02
2
DA 2015/564
12 Elliott ST KURRI KURRI, NSW AUSTRALIA
Swimming Pool - Above Ground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000564/001&Body=
2015-11-30
2015-11-02
3
DA 2015/565
3 Quarrybylong ST CESSNOCK, NSW AUSTRALIA
Two (2) Lot Strata Subdivision
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000565/001&Body=
2015-11-30
2015-11-02
4
DA 2015/566
46 Peppertree DR ROTHBURY, NSW AUSTRALIA
Swimming Pool - Inground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000566/001&Body=
2015-11-30
2015-11-03
5
DA 2015/567
3 Valley View PL NULKABA, NSW AUSTRALIA
Swimming Pool - Inground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000567/001&Body=
2015-11-30
2015-11-03
6
DA 2015/568
5 Holmes ST ELLALONG, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000568/001&Body=
2015-11-30
2015-11-04
7
DA 2015/569
1 Tenth AV MILLFIELD, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000569/001&Body=
2015-11-30
2015-11-04
8
DA 2015/570
32 Rugby ST ELLALONG, NSW AUSTRALIA
Single Storey Dwelling with Attached Garage
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000570/001&Body=
2015-11-30
2015-11-04
9
DA 2015/571
3 Turpentine CL ROTHBURY, NSW AUSTRALIA
New Dwelling
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000571/001&Body=
2015-11-30
2015-11-04
10
DA 2015/572
1616 Broke RD POKOLBIN, NSW AUSTRALIA
Change of Use – Alteration of Gallery into Restaurant within Existing
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000572/001&Body=
2015-11-30
2015-11-04

Statistics

Average successful run time: 1 minute

Total run time: about 21 hours

Total cpu time used: 6 minutes

Total disk space used: 2.32 MB

History

  • Auto ran revision 098b9aa1 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    6 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    14 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    29 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    13 records added, 1 record updated in the database
    5 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

cessnock / scraper.php