planningalerts-scrapers / port_macquarie_hastings

Port Macquarie-Hastings Council Development Applications

The official Dubbo City Council Application Tracker

README.md

Port Macquarie Hastings Council Scraper

This site is a typical .NET site involves the followings. * Cookie - Yes * Pagnation - No, hardcoded to fetch up to first 100 records * Javascript - No * Ajax - Yes * JSON - Yes

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default, today - 1 week) * thismonth * lastmonth

Enjoy

Contributors

Last run completed successfully 2019-02-15.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting data between 09/02/2019 and 16/02/2019, changable via MORPH_PERIOD environment Saving record 11.2019.86.1 - 31 Moran Cl, King Creek 2446 NSW Saving record 12.2019.86.1 - 31 Moran Cl, King Creek 2446 NSW Skipping already saved record 10.2019.86.1 Saving record 10.2019.97.1 - 585 Ocean Dr, North Haven 2443 NSW Skipping already saved record 10.2019.87.1 Skipping already saved record 10.2019.85.1 Skipping already saved record 10.2019.94.1 Saving record 710.2019.6021.1 - Horton St, Port Macquarie 2444 NSW Saving record 680.2019.84.1 - 52 Kalinda Dr, Port Macquarie 2444 NSW Saving record 11.2019.84.1 - 52 Kalinda Dr, Port Macquarie 2444 NSW Saving record 12.2019.84.1 - 52 Kalinda Dr, Port Macquarie 2444 NSW Skipping already saved record 10.2019.84.1 Saving record 10.2019.96.1 - 3 Backler St, Thrumster 2444 NSW Skipping already saved record 680.2019.5016.1 Saving record 21.2019.17.1 - 53 Norman St, Laurieton 2443 NSW Saving record 10.2018.340.2 - 65 Yaluma Dr, Port Macquarie 2444 NSW Saving record 11.2019.83.1 - 45 Lord St, Laurieton 2443 NSW Saving record 12.2019.83.1 - 45 Lord St, Laurieton 2443 NSW Skipping already saved record 10.2019.83.1 Saving record 680.2019.81.1 - 73 Rob Roy Rd, King Creek 2446 NSW Saving record 10.2019.95.1 - 712 Beechwood Rd, Beechwood 2446 NSW Saving record 11.2019.81.1 - 73 Rob Roy Rd, King Creek 2446 NSW Saving record 12.2019.81.1 - 73 Rob Roy Rd, King Creek 2446 NSW Skipping already saved record 10.2019.81.1 Saving record 710.2019.80.1 - 31 Campus St, Thrumster 2444 NSW Saving record 680.2019.80.1 - 31 Campus St, Thrumster 2444 NSW Skipping already saved record 10.2019.80.1 Saving record 15.2019.7043.1 - 49 Songbird Cct, Port Macquarie 2444 NSW Saving record 10.2018.1079.2 - 25 Colonel Barney Dr, Port Macquarie 2444 NSW Saving record 12.2018.401.1 - 34 Meares Cct, Thrumster 2444 NSW Skipping already saved record 710.2019.6020.1 Skipping already saved record 710.2019.6017.1 Skipping already saved record 10.2019.93.1 Skipping already saved record 10.2019.92.1 Skipping already saved record 680.2019.5018.1 Skipping already saved record 10.2019.91.1 Skipping already saved record 10.2019.88.1 Skipping already saved record 10.2019.89.1 Skipping already saved record 10.2019.90.1 Skipping already saved record 710.2019.6018.1 Skipping already saved record 680.2019.5019.1 Skipping already saved record 680.2019.7042.1 Skipping already saved record 15.2019.7042.1 Skipping already saved record 680.2019.5017.1 Skipping already saved record 21.2019.16.1 Skipping already saved record 12.2019.7042.1 Skipping already saved record 710.2019.7041.1 Skipping already saved record 680.2019.7041.1 Skipping already saved record 15.2019.7041.1 Skipping already saved record 12.2019.7041.1 Skipping already saved record 10.2019.82.1

Data

Downloaded 1298 times by

data

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.76 MB) Use the API

rows 10 / 9657

id	council_reference	address	description	info_url	comment_url	date_scraped	date_received
1	787/2015	4 Academic Street THRUMSTER NSW 2444, Australia	Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-02
2	865/2015	5 Admirals Circle LAKEWOOD NSW 2443, Australia	Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-26
3	459/2015	22 Ashdown Drive PORT MACQUARIE NSW 2444, Australia	Modification to Retaining wall associated with Dwelling and Pool	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-17
4	820/2015	24 Ashdown Drive PORT MACQUARIE NSW 2444, Australia	Subdivision (Exempt)	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-10
5	862/2015	338 Bago Road WAUCHOPE NSW 2446, Australia	Dwelling and Shed	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-26
6	832/2015	173 Ballengara Bransdon Road COOPERABUNG NSW 2441, Australia	Additions to Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-13
7	819/2015	33 Bay Street PORT MACQUARIE NSW 2444, Australia	Additions to Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-10
8	872/2015	818 Beechwood Road BEECHWOOD NSW 2446, Australia	Ancillary Building - Shed	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-30
9	836/2015	511 Black Creek Road BLACK CREEK NSW 2439, Australia	Alterations and Additions to Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-17
10	834/2015	Blackbutt Drive WAUCHOPE NSW 2446, Australia	Multi Dwelling Housing with a Combination of Torrens and Strata Title Subdivision	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-16

Statistics

Average successful run time: 6 minutes

Total run time: 5 days

Total cpu time used: 41 minutes

Total disk space used: 3.13 MB

History

Auto ran revision b65e863d and completed successfully 2019-02-15.

run time 7 s

21 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2019-02-14.

run time 7 s

13 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2019-02-13.

run time 6 s

10 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2019-02-12.

run time 5 s

23 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2019-02-11.

run time 6 s

7 records added, 1 record updated in the database

1 page scraped
...
Created on morph.io 2015-12-07

Show complete history

Scraper code

PHP

port_macquarie_hastings / scraper.php

git clone URL