planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Server - .NET
Cookie tracking - No
Pagnation - Yes
Javascript - No
Ajax - Yes
Clearly defined data within a record - Yes
Scrape detail page - No

Setup MORPH_PERIOD for data recovery, available options are

thisweek (default)
thismonth
lastmonth
~~year (ie. 2017, min 2003)~~

Enjoy

Contributors

Last run failed 2020-01-27 with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting 'Last 14 Days' data, changable via MORPH_PERIOD environment Scraping page 1 of 3 scraper.rb:69:in `parse': invalid date (ArgumentError) from scraper.rb:69:in `block (2 levels) in <main>' from scraper.rb:61:in `each' from scraper.rb:61:in `block in <main>' from scraper.rb:51:in `each' from scraper.rb:51:in `<main>'

Data

Downloaded 971 times by

data

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.67 MB) Use the API

rows 10 / 6011

id	council_reference	address	description	info_url	comment_url	date_scraped	date_received
1	252/D341/17	Wattle Street Brompton, SA	Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.	http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-15
2	040/D295/17	17-23 Ashburton Street Blair Athol, SA	Land Division	http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
3	145/D202/17	Valley View Drive McLaren Vale, SA	1 in to 2 land division	http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
4	170/D195/17	Andrea Avenue Newton, SA	Torrens Title land division. One allotment into three. Existing dwelling to be demolished.	http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
5	170/D196/17	8-10 Gilbert Street NEWTON, SA	1 into 4 Torrens division	http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
6	211/D172/17	37 and 39 Malurus Avenue Lockleys, SA	division of land to create 3 new allotments from the existing two.	http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
7	361/D188/17	19 Uranbo Street Salisbury North, SA	Torrens Land Division. One Allotment into Two Allotments.	http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
8	361/D189/17	7 Gilbert Street Ingle Farm, SA	Land Division	http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-13
9	100/D241/17	9 Rosedale Avenue Morphettville, SA	Division of land to create two new Torrens title allotments	http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-12
10	100/D242/17	19 Lawson Avenue Morphettville, SA	Division of land to create two new Torrens title allotments	http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE	http://www.saplanningportal.sa.gov.au/public_register	2017-10-17	2017-10-12

Statistics

Average successful run time: less than a minute

Total run time: about 9 hours

Total cpu time used: 7 minutes

Total disk space used: 1.71 MB

History

Auto ran revision 77f2429f and failed 2020-01-27.

run time 6 s

nothing changed in the database
Auto ran revision 77f2429f and failed 2020-01-26.

run time 1 minute

nothing changed in the database
Auto ran revision 77f2429f and failed 2020-01-25.

run time 59 s

nothing changed in the database
Auto ran revision 77f2429f and failed 2020-01-24.

run time 1 minute

nothing changed in the database
Auto ran revision 77f2429f and failed 2020-01-23.

run time 5 s

nothing changed in the database
...
Created on morph.io 2017-10-17

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb

git clone URL