planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer benrfairless ianheggie-oaf Br3nda

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Using user agent: Ruby/3.2.2 PlanningAlerts scraper for SA Planning Portal/2.0 (https://www.planningalerts.org.au/about) Visiting html page to set cookies: https://plan.sa.gov.au/have_your_say/notified_developments/current_notified_developments ... Delaying for 21.19s to avoid 429 Too Many Requests errors Got 200 response from https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticessummary Found 102 applications to process in random order with 15.0...25.0 seconds between requests. /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Reusing comment email and authority from existing record: 25022946 Saving record 25022946, 38 ROMALO AV MAGILL SA 5072 Reusing comment email and authority from existing record: 25036315 Saving record 25036315, 133 LOWER NELSON RD PORT MACDONNELL SA 5291 Reusing comment email and authority from existing record: 25033338 Saving record 25033338, 253 -255 OLD SOUTH RD OLD REYNELLA SA 5161 Reusing comment email and authority from existing record: 25035259 Saving record 25035259, UNIT 3 4 -8 DEEDS RD CAMDEN PARK SA 5038 Reusing comment email and authority from existing record: 25035892 Saving record 25035892, 1 -7 CAIRNS ST ADELAIDE SA 5000 Reusing comment email and authority from existing record: 25029247 Saving record 25029247, 504 SEAVIEW RD HENLEY BEACH SA 5022 Reusing comment email and authority from existing record: 23011454 Saving record 23011454, 141 COLVILLE RD ALDINGA SA 5173 Reusing comment email and authority from existing record: 25037443 Saving record 25037443, 39 STURT AV CLAPHAM SA 5062 Reusing comment email and authority from existing record: 25034521 Saving record 25034521, 10 ALPHA RD PROSPECT SA 5082 Reusing comment email and authority from existing record: 25031230 Saving record 25031230, 49 WILPENA TCE ALDGATE SA 5154 Reusing comment email and authority from existing record: 25038104 Saving record 25038104, 5 GROVE ST GLENELG EAST SA 5045 Reusing comment email and authority from existing record: 24014558 Saving record 24014558, 23 MAIN NORTH RD CLARE SA 5453 Reusing comment email and authority from existing record: 25036120 Saving record 25036120, 148 PENOLA RD MOUNT GAMBIER SA 5290 Reusing comment email and authority from existing record: 25037931 Saving record 25037931, UNIT 1-2 151 GRAND JUNCTION RD OTTOWAY SA 5013 Reusing comment email and authority from existing record: 25026137 Saving record 25026137, 315 SAMPSON RD MYPONGA BEACH SA 5202 Reusing comment email and authority from existing record: 25027846 Saving record 25027846, 25 FIRST AV FORESTVILLE SA 5035 Reusing comment email and authority from existing record: 25036509 Saving record 25036509, 11 PAULA AV WINDSOR GARDENS SA 5087 Reusing comment email and authority from existing record: 25030450 Saving record 25030450, 73 EDUCATION RD CHANDLERS HILL SA 5159 Reusing comment email and authority from existing record: 25037365 Saving record 25037365, UNIT 1-2 266 -268 GLEN OSMOND RD FULLARTON SA 5063 Reusing comment email and authority from existing record: 25037105 Saving record 25037105, 26 GILBERT ST GOODWOOD SA 5034 Reusing comment email and authority from existing record: 25033743 Saving record 25033743, 88 -88A GRANGE RD WELLAND SA 5007 Reusing comment email and authority from existing record: 25033226 Saving record 25033226, 243 -253 HALIFAX ST ADELAIDE SA 5000 Reusing comment email and authority from existing record: 25037911 Saving record 25037911, LOT 101 SEVENTH ST GAWLER SOUTH SA 5118 Reusing comment email and authority from existing record: 25028811 Saving record 25028811, 40 FARROW CCT SEAFORD SA 5169 Reusing comment email and authority from existing record: 25003185 Saving record 25003185, 18 OXFORD ST HYDE PARK SA 5061 Reusing comment email and authority from existing record: 25017139 Saving record 25017139, 175 CROSS RD WESTBOURNE PARK SA 5041 Reusing comment email and authority from existing record: 25036267 Saving record 25036267, 70 BATH ST GLENELG SOUTH SA 5045 Reusing comment email and authority from existing record: 25034206 Saving record 25034206, 26 VINE ST PROSPECT SA 5082 Reusing comment email and authority from existing record: 25018388 Saving record 25018388, 108 RANGE RD W WILLUNGA SOUTH SA 5172 Reusing comment email and authority from existing record: 25036938 Saving record 25036938, 4 JOHN FALLAS DR COMPTON SA 5291 Reusing comment email and authority from existing record: 25002919 Saving record 25002919, LOT 7 GILES ST MAGILL SA 5072 Reusing comment email and authority from existing record: 25016854 Saving record 25016854, 12 LOUIS AV HAWTHORNDENE SA 5051 Reusing comment email and authority from existing record: 25034939 Saving record 25034939, LOT 12 BILLABONG RD GOOLWA SOUTH SA 5214 Reusing comment email and authority from existing record: 25033458 Saving record 25033458, 39 CHARLES ST PROSPECT SA 5082 Reusing comment email and authority from existing record: 25026160 Saving record 25026160, 1 -3 TURNERS AV HAWTHORNDENE SA 5051 Reusing comment email and authority from existing record: 25024793 Saving record 25024793, 126D BLEWITT SPRINGS RD MCLAREN FLAT SA 5171 Reusing comment email and authority from existing record: 25031903 Saving record 25031903, 45D CLANCY RD GAWLER BELT SA 5118 Reusing comment email and authority from existing record: 25037687 Saving record 25037687, 12 EDWIN ST SPALDING SA 5454 Reusing comment email and authority from existing record: 25035550 Saving record 25035550, 146 RIVER LANE PORT MANNUM SA 5238 Reusing comment email and authority from existing record: 25032957 Saving record 25032957, 15 ANDREW AV MILLSWOOD SA 5034 Reusing comment email and authority from existing record: 25027822 Saving record 25027822, 419 ESPLANADE MOANA SA 5169 Reusing comment email and authority from existing record: 25027756 Saving record 25027756, 2A MARY PENFOLD DR ROSSLYN PARK SA 5072 Reusing comment email and authority from existing record: 25007283 Saving record 25007283, 7 NALARA PL SALISBURY HEIGHTS SA 5109 Reusing comment email and authority from existing record: 25020462 Saving record 25020462, 3E TRELAWNEY ST MOONTA BAY SA 5558 Reusing comment email and authority from existing record: 25034812 Saving record 25034812, 5A COLLIVER ST NORWOOD SA 5067 Reusing comment email and authority from existing record: 24033910 Saving record 24033910, 64 WOOLY BUTT RD IRONBANK SA 5153 Reusing comment email and authority from existing record: 25028987 Saving record 25028987, 9 SCOTT RD LANGHORNE CREEK SA 5255 Reusing comment email and authority from existing record: 25036117 Saving record 25036117, 3 ALFORD RD GLYNDE SA 5070 Reusing comment email and authority from existing record: 25022395 Saving record 25022395, 35 ROCHESTER ST LEABROOK SA 5068 Reusing comment email and authority from existing record: 25034052 Saving record 25034052, 14 KENTON ST LOCKLEYS SA 5032 Reusing comment email and authority from existing record: 25032069 Saving record 25032069, 53 SANDERGROVE RD STRATHALBYN SA 5255 Reusing comment email and authority from existing record: 25011798 Saving record 25011798, 4 EVANS ST WOODVILLE SOUTH SA 5011 Reusing comment email and authority from existing record: 25025368 Saving record 25025368, 69 SIXTH AV ST PETERS SA 5069 Reusing comment email and authority from existing record: 25030959 Saving record 25030959, 6 MORRELL ST ALLENBY GARDENS SA 5009 Reusing comment email and authority from existing record: 25029744 Saving record 25029744, UNIT 1 7 CAIRNS ST NORWOOD SA 5067 Reusing comment email and authority from existing record: 25031836 Saving record 25031836, 7 BURTON ST COWELL SA 5602 Reusing comment email and authority from existing record: 25032521 Saving record 25032521, 77 WHITTY RD TANTANOOLA SA 5280 Reusing comment email and authority from existing record: 25036473 Saving record 25036473, -34.95682050848403 138.60838298427927 Reusing comment email and authority from existing record: 25014351 Saving record 25014351, 15A CARLO ST KIDMAN PARK SA 5025 Reusing comment email and authority from existing record: 25038388 Saving record 25038388, 6 MURRAY ST CLAPHAM SA 5062 Reusing comment email and authority from existing record: 25032680 Saving record 25032680, 23 AMANDA AV FINDON SA 5023 Reusing comment email and authority from existing record: 25037304 Saving record 25037304, 7 AVENEL GARDENS RD MEDINDIE SA 5081 Reusing comment email and authority from existing record: 25036069 Saving record 25036069, 4 EDISON RD TONSLEY SA 5042 Reusing comment email and authority from existing record: 25019038 Saving record 25019038, 6 GEORGE CT MARINO SA 5049 Reusing comment email and authority from existing record: 25028799 Saving record 25028799, 33 ANGAS ST ADELAIDE SA 5000 Reusing comment email and authority from existing record: 25033522 Saving record 25033522, 705 SOUTH RD BLACK FOREST SA 5035 Reusing comment email and authority from existing record: 25036063 Saving record 25036063, 17 OXFORD ST HYDE PARK SA 5061 Reusing comment email and authority from existing record: 24031019 Saving record 24031019, 123 STATELY WAY WALLAROO SA 5556 Reusing comment email and authority from existing record: 25026734 Saving record 25026734, 4 ALLAN ST MARINO SA 5049 Reusing comment email and authority from existing record: 25034850 Saving record 25034850, 38 ORMOND GR TOORAK GARDENS SA 5065 Reusing comment email and authority from existing record: 25035067 Saving record 25035067, 40 SWAN ST GRANGE SA 5022 Reusing comment email and authority from existing record: 25024053 Saving record 25024053, 163 -167 SOUTH RD MILE END SA 5031 Reusing comment email and authority from existing record: 25036635 Saving record 25036635, LOT 2 FOURTH AV ST PETERS SA 5069 Reusing comment email and authority from existing record: 25038441 Saving record 25038441, 167 -169 MCLEAN ST BEVERLEY SA 5009 Reusing comment email and authority from existing record: 25033798 Saving record 25033798, LOT 2 JEFFREY ST NAIRNE SA 5252 Reusing comment email and authority from existing record: 25034135 Saving record 25034135, 19 SALISBURY TCE COLLINSWOOD SA 5081 Reusing comment email and authority from existing record: 23024355 Saving record 23024355, 129 WILLIAMSTOWN RD SANDY CREEK SA 5350 Reusing comment email and authority from existing record: 25013426 Saving record 25013426, 1488 KEYNETON RD KEYNETON SA 5353 Reusing comment email and authority from existing record: 25035239 Saving record 25035239, UNIT 1 4 ROSETTA ST COLLINSWOOD SA 5081 Reusing comment email and authority from existing record: 25037789 Saving record 25037789, 31 LAURA DR NEWTON SA 5074 Reusing comment email and authority from existing record: 25029849 Saving record 25029849, 195 STANLEY ST NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 25036034 Saving record 25036034, LOT 11 SAND MINE RD MOUNT COMPASS SA 5210 Reusing comment email and authority from existing record: 25035947 Saving record 25035947, 11 LIGHT CR MOUNT BARKER SA 5251 Reusing comment email and authority from existing record: 25038750 Saving record 25038750, 2 STURT ST MOUNT GAMBIER SA 5290 Reusing comment email and authority from existing record: 25037939 Saving record 25037939, 7303 HORROCKS HWY WATERVALE SA 5452 Reusing comment email and authority from existing record: 25036271 Saving record 25036271, 40 AVENUE RD HIGHGATE SA 5063 Reusing comment email and authority from existing record: 25038676 Saving record 25038676, 1 SPENCE AV MYRTLE BANK SA 5064 Reusing comment email and authority from existing record: 25030616 Saving record 25030616, 84 MAIN NORTH RD WILLASTON SA 5118 Reusing comment email and authority from existing record: 25035673 Saving record 25035673, 3 INVERGOWRIE AV HIGHGATE SA 5063 Reusing comment email and authority from existing record: 25035554 Saving record 25035554, LOT 31 ADELAIDE RD MANNUM SA 5238 Reusing comment email and authority from existing record: 25035502 Saving record 25035502, 47 THE ANNIE WATT CCT WEST LAKES SHORE SA 5020 Reusing comment email and authority from existing record: 25021939 Saving record 25021939, 38 MARINER DR SHEIDOW PARK SA 5158 Reusing comment email and authority from existing record: 25024778 Saving record 25024778, 42 MELBOURNE ST NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 25036514 Saving record 25036514, 7 CHARLES ST ALLENBY GARDENS SA 5009 Reusing comment email and authority from existing record: 25036945 Saving record 25036945, 3 -7 PARTRIDGE ST GOOLWA SA 5214 Reusing comment email and authority from existing record: 25034589 Saving record 25034589, 236 GLEN OSMOND RD FULLARTON SA 5063 Reusing comment email and authority from existing record: 25033672 Saving record 25033672, LOT 402 FULLARTON RD HIGHGATE SA 5063 Reusing comment email and authority from existing record: 25037065 Saving record 25037065, 11 KONDOPARINGA RD MEADOWS SA 5201 Reusing comment email and authority from existing record: 25033737 Saving record 25033737, LOT 270 FLINDERS AV COLONEL LIGHT GARDENS SA 5041 Reusing comment email and authority from existing record: 25020792 Saving record 25020792, 51 WOODLAND WAY TERINGIE SA 5072 Reusing comment email and authority from existing record: 25028555 Saving record 25028555, LOT 150 LUCINDALE RD STEWART RANGE SA 5271 Reusing comment email and authority from existing record: 25031932 Saving record 25031932, 64 ADELAIDE RD GAWLER SOUTH SA 5118 First Application data retrieved: {"publicNotificationID"=>12436, "applicationID"=>25013426, "applicationDocumentStorageMetadataID"=>0, "propertyAddress"=>"1488 KEYNETON RD KEYNETON SA 5353, LOT 128 KEYNETON RD MOCULTA SA 5353", "developmentDescription"=>"Land Division 5 into 5 Torrens Title", "closingDate"=>"1/27/2026 12:00:00 AM", "decisionAuthority"=>"State Planning Commission", "relevantAuthority"=>"The Barossa Council", "locationCouncil"=>"MID MURRAY COUNCIL,THE BAROSSA COUNCIL", "notificationChannel"=>"DAP", "publicConsultationUrl"=>"", "applicants"=>[{"interestedPartyId"=>905159, "applicationId"=>25013426, "contactName"=>"The Red Creek Trust ", "address"=>"PO Box 74 Angaston 5353"}]} Found 102 applications that were already in the database, and added 0 new applications. Added 21.19 seconds total delay over 1 requests so server doesn't complain! Count Authority ----- ------------------ 2 Adelaide Hills Council 5 Alexandrina Council 4 City of Adelaide 5 City of Burnside 1 City of Campbelltown 10 City of Charles Sturt 2 City of Holdfast Bay 4 City of Marion 6 City of Mitcham 2 City of Mt Gambier 5 City of Norwood, Payneham and St. Peters 7 City of Onkaparinga 2 City of Port Adelaide Enfield 5 City of Prospect 1 City of Salisbury 11 City of Unley 3 City of West Torrens 2 Clare and Gilbert Valleys Council 2 Copper Coast Council 1 District Council of Franklin Harbour 2 District Council of Grant 1 District Council of Yankalilla 1 Light Regional Council 2 Mid Murray Council 3 Mount Barker District Council 1 Naracoorte Lucindale Council 1 Northern Areas Council 6 State Planning Commission 1 The Corporation of the Town of Walkerville 3 Town of Gawler 1 Wattle Range Council

Data

Downloaded 4231 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture tkrikke DevAppProject softgrow odnagroup ianheggie-oaf 93399339 eucodepy seegolovemee tobiasoberrauch Jedwards157 MichaelBone

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (25.4 MB) Use the API

rows 10 / 73908

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority on_notice_from
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 4 days

Total cpu time used: about 1 hour

Total disk space used: 25.5 MB

History

  • Auto ran revision b38f71b9 and completed successfully .
    102 records added, 102 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    102 records added, 102 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and failed .
    nothing changed in the database
  • Auto ran revision b38f71b9 and failed .
    nothing changed in the database
  • Auto ran revision b38f71b9 and completed successfully .
    102 records added, 102 records removed, 1 record updated in the database
  • ...
  • Created on { hostname }

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb