planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Saving record 25008781, 96 TRAEGER RD MALLALA SA 5502 Saving record 25005194, 137 GREENOCK RD GREENOCK SA 5360 Saving record 24037645, 5 RIVIERA CT PASADENA SA 5042 Saving record 25010438, LOT 103 SEVENTH AV ALBERTON SA 5014 Saving record 22020497, 288 JERVOIS RD YANKALILLA SA 5203 Saving record 25009616, LOT 141 CORINGLE RD WOOL BAY SA 5575 Saving record 25009491, 23 CREWE ST HENLEY BEACH SA 5022 Saving record 24030917, 7869 MAIN SOUTH RD SECOND VALLEY SA 5204 Saving record 25008215, LOT 5 ESPLANADE ROBE SA 5276 Saving record 25005956, 351 BALD HILLS RD MOUNT BARKER SA 5251 Saving record 25009397, 411 PIRIE BLOCKS RD BUNGAMA SA 5540 Saving record 25003413, 39 WENDT TCE TINTINARA SA 5266 Saving record 24038258, 106 PRINCES HWY NAIRNE SA 5252 Saving record 25009682, 5 -7 AVENUE ST MILLSWOOD SA 5034 Saving record 25012121, 5 -9 STARKE ST LOCK SA 5633 Saving record 24036960, 62 PORT RD WALLAROO MINES SA 5554 Saving record 25011444, 17 SEVENTH AV CHELTENHAM SA 5014 /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:329:in `fetch': 429 => Net::HTTPTooManyRequests for https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticedetail -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:998:in `response_redirect' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:321:in `fetch' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:1323:in `post_form' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:536:in `post' from scraper.rb:25:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3844 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture DevAppProject softgrow odnagroup ianheggie-oaf MichaelBone 93399339 eucodepy tobiasoberrauch Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (24.9 MB) Use the API

rows 10 / 72886

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 25 MB

History

  • Auto ran revision 9fd84356 and failed .
    17 records added, 1 record removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    19 records added, 9 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 12 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb