planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal


South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Saving record 25021989, 63 PRINCES RD KINGSWOOD SA 5062 Saving record 23018037, 22 HUMPHRIES TCE KILKENNY SA 5009 Saving record 25021823, 73 FROGMORE RD KIDMAN PARK SA 5025 Saving record 25004215, LOT 2 AUGUSTA HWY PORT PATERSON SA 5700 Saving record 25023693, LOT 27 CLIPPER PL MANNUM SA 5238 Saving record 25023530, LOT 21 FLINDERS HWY ELLISTON SA 5670 Saving record 25009947, 31 CIRCUIT DR HENDON SA 5014 Saving record 25018256, 6 TURNBULL PL BROMPTON SA 5007 Saving record 25018480, 274 GRAINGERS RD WASLEYS SA 5400 Saving record 25012775, 22B COWAN ST GAWLER SA 5118 Saving record 25018269, 532 NORTON SUMMIT RD TERINGIE SA 5072 Saving record 25019330, 2 MILNE CT EVANSTON PARK SA 5116 Saving record 25006419, 5 SMITH RD GAWLER BELT SA 5118 Saving record 24030574, 1 DENNIS AV ROBE SA 5276 Saving record 23032264, 55 CORNISH TCE WALLAROO SA 5556 Saving record 25017995, LOT 10 AUGUSTA HWY PORT AUGUSTA SA 5700 Saving record 25013747, 10 ROSELLA ST PAYNEHAM SA 5070 Saving record 25013060, 126 -128 WATERFALL GULLY RD WATERFALL GULLY SA 5066 Saving record 25001481, 32 THE ESPLANADE HALLETT COVE SA 5158 /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:329:in `fetch': 429 => Net::HTTPTooManyRequests for https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticedetail -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:998:in `response_redirect' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:321:in `fetch' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:1323:in `post_form' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:536:in `post' from scraper.rb:25:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3940 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture DevAppProject softgrow odnagroup ianheggie-oaf MichaelBone 93399339 eucodepy tobiasoberrauch Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (25.1 MB) Use the API

rows 10 / 73347

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 25.2 MB

History

  • Auto ran revision 9fd84356 and failed .
    19 records added, 19 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 1 record removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    19 records added, 19 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    nothing changed in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 10 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb