planningalerts-scrapers / cessnock

Cessnock City Council Development Applications

Scrapes datracker.cessnock.nsw.gov.au

Cessnock City Council home page.


Cessnock City Council scraper

  • Server - .NET - ApplicationMaster
  • Accept Terms - Yes
  • Cookie tracking - Yes and managed by PGBrowser
  • Pagnation - No
  • Javascript - No
  • UserAgent - No
  • Request actual DA page for data - No
  • Clearly defined data within a row - Yes
  • JSON - Yes, in the payload

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default) * thismonth * lastmonth * custom = yyyy-mm (eg. 2017-01)

Enjoy

Contributors LoveMyData henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.json; requirements  from dependencies in composer.lock will be used for selection  - php (7.2.4)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - nginx (1.8.1)  - apache (2.4.33) -----> Installing dependencies...  Composer version 1.5.2 2017-09-11 16:59:25  Loading composer repositories with package information  Installing dependencies from lock file  Package operations: 4 installs, 0 updates, 0 removals  - Installing pguardiario/phpuri (1.0): Downloading (connecting...)Downloading (0%) Downloading (100%)  - Installing electrolinux/phpquery (0.9.6): Downloading (connecting...)Downloading (0%) Downloading (5%)Downloading (10%)Downloading (15%)Downloading (20%)Downloading (25%)Downloading (30%)Downloading (35%)Downloading (40%)Downloading (45%)Downloading (50%)Downloading (55%)Downloading (60%)Downloading (65%)Downloading (70%)Downloading (75%)Downloading (80%)Downloading (85%)Downloading (90%)Downloading (95%)Downloading (100%)  - Installing byjg/pgbrowser (1.0.0): Downloading (connecting...)Downloading (0%) Downloading (65%)Downloading (100%)  - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253  Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data between 13/04/2018 and 23/04/2018, changable via MORPH_PERIOD environment Skipping already saved record 44/2018/132/1 Skipping already saved record 44/2018/128/1 Saving record 10/2018/66/1 - 19 Armidale St, Abermain 2326 NSW Saving record 10/2018/107/1 - 1001 Leggetts Dr, Mount Vincent 2323 NSW Saving record 9/2018/89/1 - 14 Enright Dr, North Rothbury 2335 NSW Saving record 10/2014/315/13 - 42 Orient St, Greta 2334 NSW Saving record 10/2014/315/12 - 42 Orient St, Greta 2334 NSW Saving record 10/2014/315/11 - 42 Orient St, Greta 2334 NSW Saving record 10/2014/315/10 - 42 Orient St, Greta 2334 NSW Saving record 10/2014/315/9 - 42 Orient St, Greta 2334 NSW Saving record 10/2014/315/8 - 42 Orient St, Greta 2334 NSW Saving record 12/2017/87/3 - 30 Woodlands Dr, Weston 2326 NSW Saving record 34/2018/3/1 - 388 Wollombi Rd, Bellbird 2325 NSW Skipping already saved record 44/2018/127/1 Saving record 8/2018/235/1 - 50 Hall St, Cessnock 2325 NSW Saving record 12/2017/271/1 - 29 Mount Vincent Rd, Mount Vincent 2323 NSW Saving record 12/2018/33/1 - 25 Mavis St, Cessnock 2325 NSW Saving record 10/2017/537/1 - 19 Wollombi Rd, Cessnock 2325 NSW Skipping already saved record 9/2018/81/1 Saving record 10/2018/127/1 - 49 Bendeich Dr, North Rothbury 2335 NSW Saving record 8/2017/715/2 - 1843 Wine Country Dr, Branxton 2335 NSW Skipping already saved record 44/2018/126/1 Skipping already saved record 8/2018/234/1 Skipping already saved record 44/2018/134/1 Skipping already saved record 44/2018/125/1 Skipping already saved record 10/2017/34/1 Skipping already saved record 12/2016/594/2 Skipping already saved record 8/2018/233/1 Skipping already saved record 9/2018/87/1 Skipping already saved record 8/2018/232/1 Skipping already saved record 8/2018/231/1 Skipping already saved record 10/2018/231/1 Skipping already saved record 8/2006/897/3 Skipping already saved record 8/2018/230/1 Skipping already saved record 10/2018/229/1 Skipping already saved record 8/2018/229/1 Skipping already saved record 10/2018/230/1 Skipping already saved record 8/2018/227/1 Skipping already saved record 44/2018/133/1 Skipping already saved record 8/2018/228/1 Skipping already saved record 44/2018/131/1 Skipping already saved record 10/2018/91/1 Skipping already saved record 10/2018/225/1 Skipping already saved record 8/2018/225/1 Skipping already saved record 15/2018/30/1 Skipping already saved record 44/2018/124/1 Skipping already saved record 10/2018/226/1 Skipping already saved record 8/2018/226/1 Skipping already saved record 8/2018/224/1 Skipping already saved record 9/2018/88/1 Skipping already saved record 12/2014/201/1 Skipping already saved record 10/2018/223/1 Skipping already saved record 8/2018/223/1 Skipping already saved record 10/2017/543/2 Skipping already saved record 48/2018/1/1 Skipping already saved record 8/2017/543/2 Skipping already saved record 12/2016/12/2 Skipping already saved record 14/2014/114/1 Skipping already saved record 12/2016/727/1 Skipping already saved record 10/2018/221/1 Skipping already saved record 8/2018/221/1 Skipping already saved record 8/2018/222/1 Skipping already saved record 44/2018/129/1 Skipping already saved record 12/2010/43/1 Skipping already saved record 47/2018/7/1 Skipping already saved record 47/2018/6/1 Skipping already saved record 12/2018/37/1 Skipping already saved record 12/2017/237/1 Skipping already saved record 34/2018/2/1 Skipping already saved record 12/2014/460/1 Skipping already saved record 9/2018/84/1 Skipping already saved record 9/2018/86/1 Skipping already saved record 9/2018/85/1 Skipping already saved record 9/2018/82/1 Skipping already saved record 9/2018/83/1

Data

Downloaded 998 times by openaustralia TheZepto MikeRalphson gdmhunter

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.51 MB) Use the API

rows 10 / 4946

id council_reference address description info_url comment_url date_scraped date_received
1
DA 2015/563
4 Sale ST GRETA, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000563/001&Body=
2015-11-30
2015-11-02
2
DA 2015/564
12 Elliott ST KURRI KURRI, NSW AUSTRALIA
Swimming Pool - Above Ground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000564/001&Body=
2015-11-30
2015-11-02
3
DA 2015/565
3 Quarrybylong ST CESSNOCK, NSW AUSTRALIA
Two (2) Lot Strata Subdivision
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000565/001&Body=
2015-11-30
2015-11-02
4
DA 2015/566
46 Peppertree DR ROTHBURY, NSW AUSTRALIA
Swimming Pool - Inground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000566/001&Body=
2015-11-30
2015-11-03
5
DA 2015/567
3 Valley View PL NULKABA, NSW AUSTRALIA
Swimming Pool - Inground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000567/001&Body=
2015-11-30
2015-11-03
6
DA 2015/568
5 Holmes ST ELLALONG, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000568/001&Body=
2015-11-30
2015-11-04
7
DA 2015/569
1 Tenth AV MILLFIELD, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000569/001&Body=
2015-11-30
2015-11-04
8
DA 2015/570
32 Rugby ST ELLALONG, NSW AUSTRALIA
Single Storey Dwelling with Attached Garage
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000570/001&Body=
2015-11-30
2015-11-04
9
DA 2015/571
3 Turpentine CL ROTHBURY, NSW AUSTRALIA
New Dwelling
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000571/001&Body=
2015-11-30
2015-11-04
10
DA 2015/572
1616 Broke RD POKOLBIN, NSW AUSTRALIA
Change of Use – Alteration of Gallery into Restaurant within Existing
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000572/001&Body=
2015-11-30
2015-11-04

Statistics

Average successful run time: 1 minute

Total run time: about 18 hours

Total cpu time used: 5 minutes

Total disk space used: 1.62 MB

History

  • Auto ran revision 098b9aa1 and completed successfully .
    17 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    6 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    3 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    12 records added, 1 record updated in the database
    5 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

cessnock / scraper.php