planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Saving record 24037451, LOT 3 ALLEN LASHMAR RD ANTECHAMBER BAY SA 5222 Saving record 24041335, 41A AMHERST AV TRINITY GARDENS SA 5068 Saving record 25001242, 33 MALCOLM ST MILLSWOOD SA 5034 Saving record 25001943, 18 DUTCHMAN DR HALLETT COVE SA 5158 Saving record 25002145, 135 SASSANOWSKY RD SUTTONTOWN SA 5291 Saving record 24031252, 103 FISHER ST FULLARTON SA 5063 Saving record 25000443, 19 VICTORIA ST PROSPECT SA 5082 Saving record 25002022, 56 KINTORE AV PROSPECT SA 5082 Saving record 24042612, 56 HINDMARSH RD MCCRACKEN SA 5211 Saving record 24040724, 241 RODS RD KULPARA SA 5552 Saving record 24037455, 8 TORRENS ST TORRENSVILLE SA 5031 Saving record 24040195, 429 PORT RD CROYDON SA 5008 Saving record 24024999, 110 -118 O'SULLIVAN BEACH RD LONSDALE SA 5160 Saving record 25001793, 1136 OLD PORT RD HENDON SA 5014 Saving record 25001988, 4 SARAH CT PENNINGTON SA 5013 Saving record 24041057, 15 MALVERN AV MALVERN SA 5061 Saving record 25000127, 6 HILL ST BURNSIDE SA 5066 /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:329:in `fetch': 429 => Net::HTTPTooManyRequests for https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticedetail -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:998:in `response_redirect' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:321:in `fetch' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:1323:in `post_form' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:536:in `post' from scraper.rb:25:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3768 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture DevAppProject softgrow odnagroup ianheggie-oaf Jedwards157 MichaelBone 93399339 eucodepy tobiasoberrauch

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (24.8 MB) Use the API

rows 10 / 72572

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 24.8 MB

History

  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 13 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    20 records added, 17 records removed, 1 record updated in the database
  • Manually ran revision 9fd84356 and failed .
    19 records added, 19 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb