planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal


South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Saving record 24022009, 343 YALLUNDA FLAT RD YALLUNDA FLAT SA 5607 Saving record 25022075, 13 KEGWORTH RD MELROSE PARK SA 5039 Saving record 25007378, 10 PRIDMORE RD GLEN OSMOND SA 5064 Saving record 25021750, LOT 7347 MONTGOMERY DR MOUNT BARKER SA 5251 Saving record 25021752, LOT 7350 MONTGOMERY DR MOUNT BARKER SA 5251 Saving record 25026153, 99 SPRINGBANK RD CLAPHAM SA 5062 Saving record 25022813, 50 MACKINNON PDE NORTH ADELAIDE SA 5006 Saving record 25026165, 210 MAIN RD BLACKWOOD SA 5051 Saving record 25022021, LOT 7349 MONTGOMERY DR MOUNT BARKER SA 5251 Saving record 25021440, LOT 7351 MONTGOMERY DR MOUNT BARKER SA 5251 Saving record 25021758, LOT 7348 MONTGOMERY DR MOUNT BARKER SA 5251 Saving record 22023707, 121 ALEXANDER AV EVANSTON PARK SA 5116 Saving record 25015960, 42 HIGH ST ARNO BAY SA 5603 Saving record 25025380, UNIT 13 390 -394 SIR DONALD BRADMAN DR BROOKLYN PARK SA 5032 Saving record 25023502, LOT 101 TAPALIN ST RENMARK WEST SA 5341 Saving record 25015729, 3 QUEEN LOUISA DR LONG FLAT SA 5253 Saving record 25014343, 44 STAR AND ARROW RD COROMANDEL EAST SA 5157 /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:329:in `fetch': 429 => Net::HTTPTooManyRequests for https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticedetail -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:998:in `response_redirect' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:321:in `fetch' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:1323:in `post_form' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:536:in `post' from scraper.rb:25:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3974 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture DevAppProject softgrow odnagroup ianheggie-oaf tobiasoberrauch Jedwards157 MichaelBone 93399339 eucodepy

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (25.2 MB) Use the API

rows 10 / 73512

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 25.2 MB

History

  • Auto ran revision 9fd84356 and failed .
    17 records added, 6 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 12 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 11 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb