planningalerts-scrapers / cessnock

Cessnock City Council Development Applications

Scrapes datracker.cessnock.nsw.gov.au

Cessnock City Council home page.


Cessnock City Council scraper

  • Server - .NET - ApplicationMaster
  • Accept Terms - Yes
  • Cookie tracking - Yes and managed by PGBrowser
  • Pagnation - No
  • Javascript - No
  • UserAgent - No
  • Request actual DA page for data - No
  • Clearly defined data within a row - Yes
  • JSON - Yes, in the payload

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default) * thismonth * lastmonth * custom = yyyy-mm (eg. 2017-01)

Enjoy

Contributors LoveMyData henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.json; requirements  from dependencies in composer.lock will be used for selection  - php (7.2.2)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - apache (2.4.29)  - nginx (1.8.1) -----> Installing dependencies...  Composer version 1.5.2 2017-09-11 16:59:25  Loading composer repositories with package information  Installing dependencies from lock file  Package operations: 4 installs, 0 updates, 0 removals  - Installing pguardiario/phpuri (1.0): Downloading (connecting...)Downloading (0%) Downloading (100%)  - Installing electrolinux/phpquery (0.9.6): Downloading (connecting...)Downloading (100%)   - Installing byjg/pgbrowser (1.0.0): Downloading (connecting...)Downloading (100%)   - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253  Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data between 14/02/2018 and 24/02/2018, changable via MORPH_PERIOD environment Saving record 12/2017/108/1 - 639 Buchanan Rd, Buchanan 2323 NSW Saving record 12/2017/165/1 - 19 Paperbark Dr, Pokolbin 2320 NSW Saving record 52/2018/1/1 - 1132 John Renshaw Dr, Black Hill 2322 NSW Saving record 8/2018/104/1 - 22 Rosehill Rd, Millfield 2325 NSW Saving record 10/2018/103/1 - 1 Jolley Rd, North Rothbury 2335 NSW Saving record 8/2018/103/1 - 1 Jolley Rd, North Rothbury 2335 NSW Skipping already saved record 10/2018/79/1 Saving record 15/2018/12/1 - 33 Baileys Lane, Abermain 2326 NSW Saving record 8/2018/102/1 - John Renshaw Dr, Black Hill 2322 NSW Saving record 8/2018/101/1 - John Renshaw Dr, Black Hill 2322 NSW Saving record 8/2018/100/1 - 15 Wethered Cr, North Rothbury 2335 NSW Skipping already saved record 8/2018/79/1 Saving record 9/2018/43/1 - 28 Mcgrane St, Cessnock 2325 NSW Saving record 9/2018/42/1 - 71 Elvin Cct, Millfield 2325 NSW Saving record 12/2017/441/1 - 15 Stanford St, Kitchener 2325 NSW Saving record 12/2017/257/2 - 62 Mount View Rd, Cessnock 2325 NSW Skipping already saved record 8/2017/86/2 Saving record 9/2018/41/1 - 18 Fletcher St, Nulkaba 2325 NSW Saving record 10/2018/13/1 - 63 Frame Dr, Abermain 2326 NSW Skipping already saved record 44/2018/45/1 Saving record 14/2016/718/1 - 18 Bendeich Dr, North Rothbury 2335 NSW Skipping already saved record 8/2017/307/2 Skipping already saved record 8/2014/527/4 Saving record 8/2018/94/1 - 1283 Leggetts Dr, Mount Vincent 2323 NSW Saving record 9/2018/39/1 - 69 Kesterton Rise, North Rothbury 2335 NSW Saving record 15/2018/11/1 - 98 Maitland Rd, Mulbring 2323 NSW Saving record 10/2018/84/1 - 176 Mathieson St, Bellbird Heights 2325 NSW Skipping already saved record 8/2018/84/1 Saving record 44/2018/57/1 - 33 Tooze Cct, North Rothbury 2335 NSW Skipping already saved record 9/2018/32/1 Saving record 9/2018/40/1 - 153A Aberdare St, Kurri Kurri 2327 NSW Saving record 8/2018/98/1 - 9 Edward St, Kurri Kurri 2327 NSW Saving record 8/2018/99/1 - 20 Callaghan St, Kurri Kurri 2327 NSW Skipping already saved record 8/2018/85/1 Saving record 10/2018/82/1 - 36 Weston St, Weston 2326 NSW Skipping already saved record 8/2018/82/1 Skipping already saved record 15/2018/10/1 Skipping already saved record 15/2018/8/1 Saving record 8/2018/97/1 - 12 Dunnett Av, North Rothbury 2335 NSW Skipping already saved record 110/2018/3/1 Saving record 10/2018/96/1 - 16 Aston Av, Cessnock 2325 NSW Saving record 8/2018/96/1 - 16 Aston Av, Cessnock 2325 NSW Saving record 10/2018/95/1 - 86 Alkira Av, Cessnock 2325 NSW Saving record 8/2018/95/1 - 86 Alkira Av, Cessnock 2325 NSW Skipping already saved record 9/2018/36/1 Saving record 8/2018/93/1 - 42 Maitland St, Kurri Kurri 2327 NSW Skipping already saved record 9/2018/35/1 Skipping already saved record 8/2018/81/1 Saving record 8/2018/92/1 - 119 Mcdonalds Rd, Pokolbin 2320 NSW Saving record 12/2017/405/1 - 18 Tallowood Dr, Nulkaba 2325 NSW Saving record 44/2018/56/1 - 22 Rosehill Rd, Millfield 2325 NSW Saving record 9/2018/38/1 - 11 Currawong Cl, Weston 2326 NSW Saving record 44/2018/55/1 - 86 Alkira Av, Cessnock 2325 NSW Saving record 44/2018/54/1 - 86 Alkira Av, Cessnock 2325 NSW Saving record 44/2018/53/1 - 16 Aston Av, Cessnock 2325 NSW Saving record 12/2017/153/1 - 53 Mitchell Av, Kurri Kurri 2327 NSW Saving record 12/2017/214/1 - 7 Busby St, Cliftleigh 2321 NSW Skipping already saved record 8/2014/528/4 Saving record 8/2018/91/1 - 60 Davis St, Millfield 2325 NSW Saving record 8/2018/90/1 - 8 Aberdare St, Kurri Kurri 2327 NSW Saving record 10/2012/447/2 - 191 Vermont Rd, Mulbring 2323 NSW Saving record 9/2018/37/1 - 149 Church St, Abermain 2326 NSW Saving record 10/2017/46/1 - 9E Quarrybylong St, Cessnock 2325 NSW Skipping already saved record 44/2018/49/1 Skipping already saved record 44/2018/50/1 Skipping already saved record 44/2018/51/1 Skipping already saved record 8/2016/556/2 Skipping already saved record 8/2018/83/1 Skipping already saved record 8/2018/78/1 Skipping already saved record 44/2018/47/1 Skipping already saved record 8/2017/223/2 Skipping already saved record 8/2018/89/1 Skipping already saved record 12/2016/779/2 Skipping already saved record 8/2018/88/1 Skipping already saved record 9/2018/34/1 Skipping already saved record 12/2016/458/1 Skipping already saved record 9/2018/33/1 Skipping already saved record 8/2018/86/1 Skipping already saved record 44/2018/48/1 Skipping already saved record 12/2016/335/1 Skipping already saved record 12/2017/274/1 Skipping already saved record 12/2017/478/1 Skipping already saved record 12/2017/161/2 Skipping already saved record 12/2017/154/1 Skipping already saved record 12/2017/53/1 Skipping already saved record 8/2018/80/1 Skipping already saved record 12/2017/337/1 Skipping already saved record 8/2018/74/1 Skipping already saved record 10/2018/73/1 Skipping already saved record 8/2018/73/1 Skipping already saved record 15/1979/130/2 Skipping already saved record 10/2018/75/1 Skipping already saved record 8/2018/75/1 Skipping already saved record 10/2018/72/1 Skipping already saved record 8/2018/72/1 Skipping already saved record 8/2018/76/1 Skipping already saved record 8/2017/233/2 Skipping already saved record 15/2018/9/1 Skipping already saved record 10/2017/602/1 Skipping already saved record 10/2018/77/1 Skipping already saved record 8/2017/649/2 Skipping already saved record 10/2016/556/6 Skipping already saved record 12/2017/207/1 Skipping already saved record 9/2018/31/1 Skipping already saved record 44/2018/46/1 Skipping already saved record 8/2017/698/2 Skipping already saved record 12/2016/493/1 Skipping already saved record 12/2017/27/1 Skipping already saved record 12/2015/2/1 Skipping already saved record 10/2017/767/1 Skipping already saved record 9/2018/30/1 Skipping already saved record 9/2018/29/1

Data

Downloaded 948 times by openaustralia TheZepto MikeRalphson gdmhunter

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.37 MB) Use the API

rows 10 / 4411

id council_reference address description info_url comment_url date_scraped date_received
1
DA 2015/563
4 Sale ST GRETA, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000563/001&Body=
2015-11-30
2015-11-02
2
DA 2015/564
12 Elliott ST KURRI KURRI, NSW AUSTRALIA
Swimming Pool - Above Ground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000564/001&Body=
2015-11-30
2015-11-02
3
DA 2015/565
3 Quarrybylong ST CESSNOCK, NSW AUSTRALIA
Two (2) Lot Strata Subdivision
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000565/001&Body=
2015-11-30
2015-11-02
4
DA 2015/566
46 Peppertree DR ROTHBURY, NSW AUSTRALIA
Swimming Pool - Inground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000566/001&Body=
2015-11-30
2015-11-03
5
DA 2015/567
3 Valley View PL NULKABA, NSW AUSTRALIA
Swimming Pool - Inground
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000567/001&Body=
2015-11-30
2015-11-03
6
DA 2015/568
5 Holmes ST ELLALONG, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000568/001&Body=
2015-11-30
2015-11-04
7
DA 2015/569
1 Tenth AV MILLFIELD, NSW AUSTRALIA
Shed
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000569/001&Body=
2015-11-30
2015-11-04
8
DA 2015/570
32 Rugby ST ELLALONG, NSW AUSTRALIA
Single Storey Dwelling with Attached Garage
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000570/001&Body=
2015-11-30
2015-11-04
9
DA 2015/571
3 Turpentine CL ROTHBURY, NSW AUSTRALIA
New Dwelling
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000571/001&Body=
2015-11-30
2015-11-04
10
DA 2015/572
1616 Broke RD POKOLBIN, NSW AUSTRALIA
Change of Use – Alteration of Gallery into Restaurant within Existing
mailto:council@cessnock.nsw.gov.au?subject=Development Application Enquiry: 008/2015/00000572/001&Body=
2015-11-30
2015-11-04

Statistics

Average successful run time: 1 minute

Total run time: about 17 hours

Total cpu time used: 5 minutes

Total disk space used: 1.47 MB

History

  • Auto ran revision 098b9aa1 and completed successfully .
    46 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and failed .
    nothing changed in the database
  • Auto ran revision 098b9aa1 and completed successfully .
    5 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    18 records added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 098b9aa1 and completed successfully .
    71 records added, 1 record updated in the database
    5 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

cessnock / scraper.php