planningalerts-scrapers / port_macquarie_hastings

Port Macquarie-Hastings Council Development Applications

Scrapes datracker.pmhc.nsw.gov.au and www.firebrno.cz

The official Dubbo City Council Application Tracker

README.md

Port Macquarie Hastings Council Scraper

This site is a typical .NET site involves the followings. * Cookie - Yes * Pagnation - No, hardcoded to fetch up to first 100 records * Javascript - No * Ajax - Yes * JSON - Yes

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default, today - 1 week) * thismonth * lastmonth

Enjoy

Contributors

Last run completed successfully 2018-04-19.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting data between 13/04/2018 and 20/04/2018, changable via MORPH_PERIOD environment Saving record 10.2018.303.1 - 1-7 Newport Island Rd, Port Macquarie 2444 NSW Saving record 680.2018.5071.1 - 731 Beechwood Rd, Beechwood 2446 NSW Saving record 12.2012.358.6 - 30 Boundary St, Port Macquarie 2444 NSW Saving record 12.2017.219.1 - 11 Horizons Pky, Port Macquarie 2444 NSW Saving record 12.2017.618.1 - 9 Litchfield Pky, Thrumster 2444 NSW Saving record 10.2018.302.1 - 116 Cameron St, Wauchope 2446 NSW Saving record 10.2018.301.1 - 7 Pinah Pl, King Creek 2446 NSW Skipping already saved record 710.2018.6069.1 Skipping already saved record 10.2018.300.1 Skipping already saved record 10.2018.299.1 Skipping already saved record 10.2018.298.1 Skipping already saved record 10.2018.295.1 Skipping already saved record 10.2018.296.1 Skipping already saved record 17.1993.159.3 Skipping already saved record 17.2015.845.2 Skipping already saved record 680.2015.5110.48 Skipping already saved record 680.2018.5070.1 Skipping already saved record 17.2017.895.1 Skipping already saved record 710.2018.252.1 Skipping already saved record 13.2017.581.1 Skipping already saved record 12.2017.1001.1 Skipping already saved record 11.2017.1001.1 Skipping already saved record 10.2017.925.2 Skipping already saved record 10.2017.399.2 Skipping already saved record 12.2017.330.1 Skipping already saved record 10.2018.294.1 Skipping already saved record 680.2018.5017.2 Skipping already saved record 10.2018.293.1 Skipping already saved record 10.2018.291.1 Skipping already saved record 10.2017.764.2 Skipping already saved record 11.2017.764.2 Skipping already saved record 10.2018.297.1 Skipping already saved record 15.2018.7137.1 Skipping already saved record 12.2017.656.1 Skipping already saved record 12.2018.7133.1 Skipping already saved record 15.2018.7133.1 Skipping already saved record 11.2017.1052.1 Skipping already saved record 11.2017.572.1 Skipping already saved record 10.2018.292.1

Data

Downloaded 989 times by

data

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.82 MB) Use the API

rows 10 / 6236

id	council_reference	address	description	info_url	comment_url	date_scraped	date_received
1	787/2015	4 Academic Street THRUMSTER NSW 2444, Australia	Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-02
2	865/2015	5 Admirals Circle LAKEWOOD NSW 2443, Australia	Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-26
3	459/2015	22 Ashdown Drive PORT MACQUARIE NSW 2444, Australia	Modification to Retaining wall associated with Dwelling and Pool	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-17
4	820/2015	24 Ashdown Drive PORT MACQUARIE NSW 2444, Australia	Subdivision (Exempt)	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-10
5	862/2015	338 Bago Road WAUCHOPE NSW 2446, Australia	Dwelling and Shed	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-26
6	832/2015	173 Ballengara Bransdon Road COOPERABUNG NSW 2441, Australia	Additions to Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-13
7	819/2015	33 Bay Street PORT MACQUARIE NSW 2444, Australia	Additions to Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-10
8	872/2015	818 Beechwood Road BEECHWOOD NSW 2446, Australia	Ancillary Building - Shed	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-30
9	836/2015	511 Black Creek Road BLACK CREEK NSW 2439, Australia	Alterations and Additions to Dwelling	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-17
10	834/2015	Blackbutt Drive WAUCHOPE NSW 2446, Australia	Multi Dwelling Housing with a Combination of Torrens and Strata Title Subdivision	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	https://eservice.hastings.nsw.gov.au/eservice/dialog/daEnquiryInit.do?doc_typ=(10,15)&nodeNum=115765	2015-12-07	2015-11-16

Statistics

Average successful run time: 8 minutes

Total run time: 5 days

Total cpu time used: 39 minutes

Total disk space used: 2.19 MB

History

Auto ran revision b65e863d and completed successfully 2018-04-19.

run time 7 s

7 records added, 1 record updated in the database

42 pages scraped
Auto ran revision b65e863d and completed successfully 2018-04-18.

run time 8 s

13 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2018-04-17.

run time 35 s

20 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2018-04-16.

run time 8 s

30 records added, 1 record updated in the database

1 page scraped
Auto ran revision b65e863d and completed successfully 2018-04-15.

run time 7 s

nothing changed in the database

1 page scraped
...
Created on morph.io 2015-12-07

Show complete history

Scraper code

PHP

port_macquarie_hastings / scraper.php

git clone URL