planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer benrfairless ianheggie-oaf Br3nda

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Using user agent: Ruby/3.2.2 PlanningAlerts scraper for SA Planning Portal/2.0 (https://www.planningalerts.org.au/about) Visiting html page to set cookies: https://plan.sa.gov.au/have_your_say/notified_developments/current_notified_developments ... Delaying for 19.28s to avoid 429 Too Many Requests errors Got 200 response from https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticessummary /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Found 74 applications to process in random order with 15.0...25.0 seconds between requests. Reusing comment email and authority from existing record: 25033619 Saving record 25033619, UNIT 1 G 16 BAGOT ST NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 26013997 Saving record 26013997, 1A HEXHAM AV MYRTLE BANK SA 5064 Reusing comment email and authority from existing record: 26004775 Saving record 26004775, 24 HERMIT ST ROXBY DOWNS SA 5725 Reusing comment email and authority from existing record: 26019464 Saving record 26019464, 19 CLIFTON ST MILLSWOOD SA 5034 Reusing comment email and authority from existing record: 26019543 Saving record 26019543, LOT 11 MEMFORD WAY FLAGSTAFF HILL SA 5159 Reusing comment email and authority from existing record: 26011376 Saving record 26011376, 57 MARINE PDE KINGSTON SE SA 5275 Reusing comment email and authority from existing record: 26017045 Saving record 26017045, 8 FULLER ST WALKERVILLE SA 5081 Reusing comment email and authority from existing record: 26016999 Saving record 26016999, 15 SEAFIELD AV KINGSWOOD SA 5062 Reusing comment email and authority from existing record: 24010498 Saving record 24010498, 77 FREDERICK ST WELLAND SA 5007 Reusing comment email and authority from existing record: 26015315 Saving record 26015315, 496 NELSON RD GULFVIEW HEIGHTS SA 5096 Reusing comment email and authority from existing record: 26013760 Saving record 26013760, 11 GOVER ST NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 23016334 Saving record 23016334, 42 CRANWELL RD SUMMERTOWN SA 5141 Reusing comment email and authority from existing record: 26016152 Saving record 26016152, 53 -55 WEST PWY COLONEL LIGHT GARDENS SA 5041 Reusing comment email and authority from existing record: 26019683 Saving record 26019683, LOT 21 GOOCH RD COOKE PLAINS SA 5261 Reusing comment email and authority from existing record: 26004581 Saving record 26004581, 10 CHARTERIS ST PORT ELLIOT SA 5212 Reusing comment email and authority from existing record: 26019314 Saving record 26019314, 49 GEORGE ST PARKSIDE SA 5063 Reusing comment email and authority from existing record: 26005122 Saving record 26005122, 28 -45 BROUGHAM PL NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 26017458 Saving record 26017458, 15 TORRENS ST COLLEGE PARK SA 5069 Reusing comment email and authority from existing record: 26019048 Saving record 26019048, 34 CRAMOND PWY OAKDEN SA 5086 Reusing comment email and authority from existing record: 26013315 Saving record 26013315, 38 TENNYSON DR BEAUMONT SA 5066 Reusing comment email and authority from existing record: 26019541 Saving record 26019541, 7 WHALEBOAT CT VICTOR HARBOR SA 5211 Reusing comment email and authority from existing record: 26011995 Saving record 26011995, 24 TRAMINER WAY AULDANA SA 5072 Reusing comment email and authority from existing record: 26009594 Saving record 26009594, 297 KENSINGTON RD KENSINGTON PARK SA 5068 Reusing comment email and authority from existing record: 26003102 Saving record 26003102, 462 MAIN NORTH RD BLAIR ATHOL SA 5084 Reusing comment email and authority from existing record: 26020836 Saving record 26020836, 9 ELIZABETH CR CARRICKALINGA SA 5204 Reusing comment email and authority from existing record: 25036289 Saving record 25036289, LOT 3 STROUT RD MCLAREN VALE SA 5171 Reusing comment email and authority from existing record: 26017459 Saving record 26017459, 30 DEEMSTER AV CHRISTIES BEACH SA 5165 Reusing comment email and authority from existing record: 26015678 Saving record 26015678, UNIT 9 238 CHANDLERS HILL RD HAPPY VALLEY SA 5159 Reusing comment email and authority from existing record: 26018307 Saving record 26018307, 5 ELMGLADE RD SPRINGFIELD SA 5062 Reusing comment email and authority from existing record: 26018607 Saving record 26018607, 248 CLIFF VIEW DR WALKER FLAT SA 5238 Reusing comment email and authority from existing record: 26017965 Saving record 26017965, 1132 MONTACUTE RD MONTACUTE SA 5134 Reusing comment email and authority from existing record: 26019388 Saving record 26019388, 16A MILLICENT ST ATHOL PARK SA 5012 Reusing comment email and authority from existing record: 26015121 Saving record 26015121, 139 GILLMAN RD OAKBANK SA 5243 Reusing comment email and authority from existing record: 26014056 Saving record 26014056, 9 LESLIE AV EVANDALE SA 5069 Reusing comment email and authority from existing record: 26006236 Saving record 26006236, 7 INVERNESS WAY WALLAROO SA 5556 Reusing comment email and authority from existing record: 24042219 Saving record 24042219, 150 TATACHILLA RD MCLAREN VALE SA 5171 Reusing comment email and authority from existing record: 25033208 Saving record 25033208, LOT 61 CAPE WILLOUGHBY RD CUTTLEFISH BAY SA 5222 Reusing comment email and authority from existing record: 26011206 Saving record 26011206, 2 THIRTEENTH ST GAWLER SOUTH SA 5118 Reusing comment email and authority from existing record: 26016021 Saving record 26016021, 15 DEW ST MILE END SA 5031 Reusing comment email and authority from existing record: 26014345 Saving record 26014345, 87 -89 OLD SWANPORT RD MURRAY BRIDGE SA 5253 Reusing comment email and authority from existing record: 25036047 Saving record 25036047, 6 ANDERSON ST STRATHALBYN SA 5255 Reusing comment email and authority from existing record: 26014103 Saving record 26014103, 20 SMITH-DORRIEN ST NETHERBY SA 5062 Reusing comment email and authority from existing record: 26020920 Saving record 26020920, 16 BLANCHE AV MAGILL SA 5072 Reusing comment email and authority from existing record: 26011307 Saving record 26011307, 19A PARHAM RD EDEN HILLS SA 5050 Reusing comment email and authority from existing record: 26017039 Saving record 26017039, 1 -1A AUSTIN CR ST GEORGES SA 5064 Reusing comment email and authority from existing record: 26019346 Saving record 26019346, 143 -173 BAINS RD MORPHETT VALE SA 5162 Reusing comment email and authority from existing record: 26021220 Saving record 26021220, 8 ALAWA AV MODBURY NORTH SA 5092 Reusing comment email and authority from existing record: 26016921 Saving record 26016921, 2 PULLEINE RD NAIRNE SA 5252 Reusing comment email and authority from existing record: 25019950 Saving record 25019950, 121 OLD BETHEL RD WILLIAMSTOWN SA 5351 Reusing comment email and authority from existing record: 26000034 Saving record 26000034, 4 ELIZABETH CR HALLETT COVE SA 5158 Reusing comment email and authority from existing record: 26004891 Saving record 26004891, LOT 103 ESPLANADE SEMAPHORE SA 5019 Reusing comment email and authority from existing record: 25024461 Saving record 25024461, 7 MERCOWIE RD CRYSTAL BROOK SA 5523 Reusing comment email and authority from existing record: 26011790 Saving record 26011790, 33 GATES RD HACKHAM SA 5163 Reusing comment email and authority from existing record: 26021477 Saving record 26021477, 26 EDWARD ST GLYNDE SA 5070 Reusing comment email and authority from existing record: 26004599 Saving record 26004599, 11 HAYNES AV RICHMOND SA 5033 Reusing comment email and authority from existing record: 26009234 Saving record 26009234, 29 SALTASH AV CHRISTIES BEACH SA 5165 Reusing comment email and authority from existing record: 26013452 Saving record 26013452, 13 SEVENTH AV ST MORRIS SA 5068 Reusing comment email and authority from existing record: 25024405 Saving record 25024405, 86A -86 PROSPECT RD PROSPECT SA 5082 Reusing comment email and authority from existing record: 26019904 Saving record 26019904, 11B SILVERMERE AV PARADISE SA 5075 Reusing comment email and authority from existing record: 26010051 Saving record 26010051, 29 CAMPANIA RD N KUDLA SA 5115 Reusing comment email and authority from existing record: 26014102 Saving record 26014102, 47 BURTON RD ATHELSTONE SA 5076 Reusing comment email and authority from existing record: 26007870 Saving record 26007870, 310 -314 DIAGONAL RD OAKLANDS PARK SA 5046 Reusing comment email and authority from existing record: 26009466 Saving record 26009466, 360 HILLIER RD HILLIER SA 5116 Reusing comment email and authority from existing record: 26020626 Saving record 26020626, 15 ANTIGUA GR WEST LAKES SA 5021 Reusing comment email and authority from existing record: 26008274 Saving record 26008274, 108 -118 MARION RD BROOKLYN PARK SA 5032 Reusing comment email and authority from existing record: 26007948 Saving record 26007948, 17 WILSON ST ADELAIDE SA 5000 Reusing comment email and authority from existing record: 26016722 Saving record 26016722, 9 EGMONT TCE ERINDALE SA 5066 Reusing comment email and authority from existing record: 26015334 Saving record 26015334, 33 BYARD TCE MITCHELL PARK SA 5043 Reusing comment email and authority from existing record: 26011951 Saving record 26011951, 2 CALCA ST SCEALE BAY SA 5680 Reusing comment email and authority from existing record: 26007677 Saving record 26007677, 38 VALETTA RD KIDMAN PARK SA 5025 Reusing comment email and authority from existing record: 25039789 Saving record 25039789, 44A COOMURRA DR SALISBURY HEIGHTS SA 5109 Reusing comment email and authority from existing record: 26022046 Saving record 26022046, 1080 OLD STURT HWY BERRI SA 5343 Reusing comment email and authority from existing record: 25018172 Saving record 25018172, 1905 DENIAL BAY RD NADIA SA 5690 Reusing comment email and authority from existing record: 26016803 Saving record 26016803, 35A PADDLESTEAMER DR MILANG SA 5256 First Application data retrieved: {"publicNotificationID"=>13612, "applicationID"=>26014345, "applicationDocumentStorageMetadataID"=>0, "propertyAddress"=>"87 -89 OLD SWANPORT RD MURRAY BRIDGE SA 5253", "developmentDescription"=>"1 into 2 Torrens Title Land Division", "closingDate"=>"8/13/2026 12:00:00 AM", "decisionAuthority"=>"State Planning Commission", "relevantAuthority"=>"Rural City of Murray Bridge", "locationCouncil"=>"THE RURAL CITY OF MURRAY BRIDGE", "notificationChannel"=>"DAP", "publicConsultationUrl"=>"", "applicants"=>[{"interestedPartyId"=>1117581, "applicationId"=>26014345, "contactName"=>"Alex Stender ", "address"=>"275 MARION ROAD NORTH PLYMPTON 5037"}]} Found 74 applications that were already in the database, and added 0 new applications. Added 19.28 seconds total delay over 1 requests so server doesn't complain! Count Authority ----- ------------------ 3 Adelaide Hills Council 2 Alexandrina Council 4 City of Adelaide 5 City of Burnside 3 City of Campbelltown 4 City of Charles Sturt 2 City of Marion 5 City of Mitcham 4 City of Norwood, Payneham and St. Peters 8 City of Onkaparinga 3 City of Port Adelaide Enfield 1 City of Prospect 2 City of Salisbury 1 City of Tea Tree Gully 3 City of Unley 1 City of Victor Harbor 3 City of West Torrens 1 Copper Coast Council 1 District Council of Kingston 1 District Council of Streaky Bay 1 District Council of Yankalilla 1 Kangaroo Island Council 1 Mid Murray Council 1 Mount Barker District Council 1 Municipal Council of Roxby Downs 1 Port Pirie Regional Council 4 State Planning Commission 1 The Barossa Council 1 The Berri Barmera Council 1 The Coorong District Council 1 The Corporation of the Town of Walkerville 3 Town of Gawler

Data

Downloaded 4899 times by openaustralia mlandauer dcalde jesse-a-reilly Manasaanand tkrikke repose-lecture AthleticoPrawn softgrow qs4v8thvrq-boop DevAppProject odnagroup ianheggie-oaf 538digital MichaelBone 93399339 eucodepy seegolovemee tobiasoberrauch Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26.1 MB) Use the API

rows 10 / 74809

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority on_notice_from
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 4 days

Total cpu time used: about 1 hour

Total disk space used: 26.1 MB

History

  • Auto ran revision b38f71b9 and completed successfully .
    74 records added, 74 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    74 records added, 68 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    84 records added, 80 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    85 records added, 83 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    99 records added, 99 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb