planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Compiling Ruby [1G-----> Using Ruby version: ruby-2.5.3 [1G-----> Installing dependencies using bundler version 1.15.2 [1G Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment [1G Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. [1G Fetching gem metadata from https://rubygems.org/...... [1G Fetching version metadata from https://rubygems.org/. [1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git [1G Using bundler 1.15.2 [1G Fetching connection_pool 2.2.5 [1G Fetching unf_ext 0.0.8.2 [1G Fetching httpclient 2.8.3 [1G Installing connection_pool 2.2.5 [1G Installing unf_ext 0.0.8.2 with native extensions [1G Installing httpclient 2.8.3 [1G Fetching mime-types-data 3.2018.0812 [1G Installing mime-types-data 3.2018.0812 [1G Fetching net-http-digest_auth 1.4.1 [1G Installing net-http-digest_auth 1.4.1 [1G Fetching mini_portile2 2.6.1 [1G Fetching racc 1.6.2 [1G Installing racc 1.6.2 with native extensions [1G Installing mini_portile2 2.6.1 [1G Fetching ntlm-http 0.1.1 [1G Installing ntlm-http 0.1.1 [1G Fetching webrobots 0.1.2 [1G Installing webrobots 0.1.2 [1G Fetching sqlite3 1.4.0 [1G Installing sqlite3 1.4.0 with native extensions [1G Fetching net-http-persistent 3.0.0 [1G Installing net-http-persistent 3.0.0 [1G Fetching mime-types 3.2.2 [1G Installing mime-types 3.2.2 [1G Fetching unf 0.1.4 [1G Installing unf 0.1.4 [1G Fetching domain_name 0.5.20190701 [1G Installing domain_name 0.5.20190701 [1G Fetching http-cookie 1.0.5 [1G Installing http-cookie 1.0.5 [1G Fetching nokogiri 1.12.5 (x86_64-linux) [1G Installing nokogiri 1.12.5 (x86_64-linux) [1G Fetching mechanize 2.7.6 [1G Installing mechanize 2.7.6 [1G Fetching sqlite_magic 0.0.6 [1G Installing sqlite_magic 0.0.6 [1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176) [1G Bundle complete! 2 Gemfile dependencies, 20 gems now installed. [1G Gems in the groups development and test were not installed. [1G Bundled gems are installed into ./vendor/bundle. [1G Bundle completed (16.54s) [1G Cleaning up the bundler cache. [1G Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. [1G-----> Detecting rake tasks [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Saving record 22033647, 163 TWENTYSIXTH ST RENMARK SOUTH SA 5341 Saving record 22034409, 31 MANN TCE NORTH ADELAIDE SA 5006 Saving record 22035399, 240 MAIN RD BLACKWOOD SA 5051 Saving record 22031775, 2A FULFORD TCE LITTLEHAMPTON SA 5250 Saving record 21022201, 75 PANALATINGA RD WOODCROFT SA 5162 Saving record 22018145, 32 TOLMER RD ELIZABETH PARK SA 5113 Saving record 22025802, 60 STANLEY ST ERINDALE SA 5066 Saving record 22027449, 478 SEAVIEW RD HENLEY BEACH SA 5022 Saving record 22040976, 2 RICHARD ST BROOKLYN PARK SA 5032 Saving record 22042250, 4 OXFORD ST WOODVILLE PARK SA 5011 Saving record 22042636, 69 ELLEN ST PORT PIRIE SA 5540 Saving record 23004316, 20 KING ST MINTARO SA 5415 Saving record 23004892, 23 HAYWARD AV TORRENSVILLE SA 5031 Saving record 23005620, 26 CIRCUIT DR HENDON SA 5014 Saving record 23005936, 153 MURRAY ST NURIOOTPA SA 5355 Saving record 23006224, 29 SEAVIEW RD WEST BEACH SA 5024 Saving record 23006243, 46 SMITH ST THEBARTON SA 5031 Saving record 23006266, UNIT 6 28-30 SEAVIEW RD WEST BEACH SA 5024 Saving record 23005870, 83 AVENUE RD CLARENCE GARDENS SA 5039 Saving record 23006923, 20 HISSAR AV SALISBURY NORTH SA 5108 Saving record 23005727, 2 THOMAS ST NAIRNE SA 5252 Saving record 23005495, 24 FIRST ST MAGILL SA 5072 Saving record 23005141, 7 PRINCE ALBERT ST ALBERT PARK SA 5014 Saving record 23004125, 2-8 GREENWITH RD GOLDEN GROVE SA 5125 Saving record 23003408, 36 MAITLAND ST MITCHAM SA 5062 Saving record 22042844, 6 PETERS AV FULHAM GARDENS SA 5024 Saving record 22043290, 213 GLEN OSMOND RD FREWVILLE SA 5063 Saving record 22039569, 1A FIFTH AV CHELTENHAM SA 5014 Saving record 22035349, 8A MCLAUCHLAN RD WINDSOR GARDENS SA 5087 Saving record 22035319, 10 ADELE AV PARADISE SA 5075 Saving record 22033547, 107-109 O'CONNELL ST NORTH ADELAIDE SA 5006 Saving record 22038444, 26 HIGHFIELD AV ST GEORGES SA 5064 Saving record 22036595, 10 TAIT ST RENOWN PARK SA 5008 Saving record 22041021, 22 FIELD ST WHYALLA PLAYFORD SA 5600 Saving record 22040393, 18 GRANT AV ROSE PARK SA 5067 Saving record 23005204, LOT 321 PARROT HILL RD MOCULTA SA 5353 Saving record 23004856, 10 WINCHESTER ST ST PETERS SA 5069 Saving record 23005644, 36 CALLINGTON RD CALLINGTON SA 5254 Saving record 23000604, 18 BRIAN HURN CR ANGASTON SA 5353 Saving record 23002134, 46 MONMOUTH RD WESTBOURNE PARK SA 5041 Saving record 23002725, 334 MAGILL RD KENSINGTON PARK SA 5068 Saving record 23007311, LOT 1 ST GEORGES PL ALBERTON SA 5014 Saving record 23006497, 152-154 PIMPALA RD MORPHETT VALE SA 5162 Saving record 23005853, LOT 31 MACCLESFIELD RD MEADOWS SA 5201 Saving record 23006137, 4 ORCHID DR ROBE SA 5276 Saving record 23002706, 7 TECOMA CL MOUNT GAMBIER SA 5290 Saving record 23002803, 77 WATTS RD BRUKUNGA SA 5252 Saving record 23002828, 54 INVERNESS WAY WALLAROO SA 5556 Saving record 23003282, 4 SHELLEY AV PLYMPTON PARK SA 5038 Saving record 23003650, 8-18 HINDMARSH RD MCCRACKEN SA 5211 Saving record 23003867, 153 RODERT RD MEADOWS SA 5201 Saving record 23003922, 176 FIRST AV ROYSTON PARK SA 5070 Saving record 23003938, LOT 396 BLUE GUM ST MOUNT BARKER SA 5251 Saving record 23003939, LOT 398 BLUE GUM ST MOUNT BARKER SA 5251 Saving record 23001917, 24 NORTHBROOK AV MITCHAM SA 5062 Saving record 23005441, 102 SHEPHERDSON RD MOUNT GAMBIER SA 5290 Saving record 23004868, 60 AURORA CCT MEADOWS SA 5201 Saving record 23005210, 2A CENTRE WAY BELAIR SA 5052 Saving record 23003944, LOT 397 BLUE GUM ST MOUNT BARKER SA 5251 Saving record 23003950, LOT 394 BLUE GUM ST MOUNT BARKER SA 5251 Saving record 23003953, LOT 393 BLUE GUM ST MOUNT BARKER SA 5251 Saving record 23004245, UNIT 3 5 WILSON ST MOUNT GAMBIER SA 5290 Saving record 23004508, 48 PRICE AV LOWER MITCHAM SA 5062 Saving record 23004326, 71 DODSON RD ENCOUNTER BAY SA 5211 Saving record 23005375, 18 NORTHBROOK AV MITCHAM SA 5062 Saving record 23001481, 9 PONTON ST SALISBURY SA 5108 Saving record 23000756, 16 THE ESPLANADE BAUDIN BEACH SA 5222 Saving record 23003940, 9 EVERSLEY AV ENFIELD SA 5085 Saving record 23003217, 259 RANGE RD N UPPER HERMITAGE SA 5131 Saving record 22032539, 288 MARINE PDE PINKS BEACH SA 5275 Saving record 22030333, 126 PORTRUSH RD PAYNEHAM SOUTH SA 5070 Saving record 22021757, 211 ROB LOXTON RD WALKER FLAT SA 5238 Saving record 21019569, 94 CHILDERS ST NORTH ADELAIDE SA 5006 Saving record 22031936, 11 GOODALL RD PARA HILLS SA 5096 Saving record 22036576, LOT 12 GOWRIE ST TORRENS PARK SA 5062 Saving record 22035305, 95 COROMANDEL PDE COROMANDEL VALLEY SA 5051 Saving record 22042761, 31 LERWICK AV HAZELWOOD PARK SA 5066 Saving record 23003370, 44 GUNTHER PDE PASADENA SA 5042 Saving record 23003203, 89 LEDGER RD BEVERLEY SA 5009 Saving record 23000160, 10 KENIHANS RD HAPPY VALLEY SA 5159 Saving record 23001427, LOT 35 OLD PORT RD PORT ADELAIDE SA 5015 Saving record 23004791, 69-71 GRAND JUNCTION RD ROSEWATER SA 5013 Saving record 23005521, 7A BURNLEY ST FULHAM SA 5024 Saving record 23003525, 23 CORNELL AV VALLEY VIEW SA 5093 Saving record 23006680, 19 ST GEORGES AV GLANDORE SA 5037 Saving record 23005376, 119 OSMANLI RD D'ESTREES BAY SA 5223 Saving record 22041887, 10 WARWICK ST LARGS NORTH SA 5016 Saving record 22043126, 7 TOOGOOD AV BEVERLEY SA 5009 Saving record 22039636, 28 RALSTON ST NORTH ADELAIDE SA 5006 Saving record 22000202, 75 HENLEY BEACH RD MILE END SA 5031 Saving record 22022923, 54 CLIFFORD ST TORRENSVILLE SA 5031 Saving record 22027904, 18A WARE ST THEBARTON SA 5031 Saving record 22032466, 102 GULF POINT DR NORTH HAVEN SA 5018 Saving record 22010999, 752 JOLLY WAY POLISH HILL RIVER SA 5453 Saving record 22031821, 35 GARNET AV BLACKWOOD SA 5051 Saving record 22037932, 14 CORMAC ST PORT HUGHES SA 5558 Saving record 22043343, 103 BLACKWOOD PARK BVD CRAIGBURN FARM SA 5051 Saving record 22041998, 63 PIGEON FLAT RD BORDERTOWN SA 5268 Saving record 23004869, 3 GREEN ST ST MORRIS SA 5068 Saving record 23004663, 9 SALTFLEET ST PORT NOARLUNGA SA 5167 Saving record 23005834, LOT 11 BLOMBERY RD ARNO BAY SA 5603 Saving record 23006273, UNIT 2 48 ROSE TCE WAYVILLE SA 5034 Saving record 23005875, 354 MARION RD NORTH PLYMPTON SA 5037 Saving record 23001744, 28 LOCHNESS AV TORRENS PARK SA 5062 Saving record 23001955, 33A PENNO PDE N BELAIR SA 5052 Saving record 22041872, 143-145 MAIN RD MCLAREN VALE SA 5171 Saving record 22042963, 45-47 FLINDERS DR CAPE JERVIS SA 5204 Saving record 22036615, 24 BEACH ST GRANGE SA 5022 Saving record 22040741, SITE 7 16 LADY BAY RD NORMANVILLE SA 5204 Saving record 22029830, UNIT 10 20 TROON DR NORMANVILLE SA 5204 Saving record 21028693, 431 SIR CONDOR LAUCKE WAY NURIOOTPA SA 5355 Saving record 21034335, 61 GAWLER RD VIRGINIA SA 5120 Saving record 22007065, 226 GAWLER-ONE TREE HILL RD EVANSTON PARK SA 5116 Saving record 22006867, LOT 15 KENT ST HAWTHORN SA 5062 Saving record 22039111, 11376 AUGUSTA HWY WARNERTOWN SA 5540 Saving record 22041120, 84 SHERIDAN RD WESTERN RIVER SA 5223 Saving record 23000823, 57 COWAN ST GAWLER SA 5118 Saving record 23001283, 12 RYMILL RD SOMERTON PARK SA 5044 Saving record 23004054, 69 FAIRFORD ST UNLEY SA 5061 Saving record 23004375, 86 HYDE RD NORTH SHIELDS SA 5607 Saving record 23004485, 17 UNION ST LARGS BAY SA 5016 Saving record 23004837, 47 WILLIAM ST WEST CROYDON SA 5008 Saving record 23002543, 27 JOHNSTONE ST GLENGOWRIE SA 5044 Saving record 23002738, LOT 1623 WILLIAM SCOTT CCT CRAIGBURN FARM SA 5051 Saving record 23003334, 17 GLENHUNTLEY ST WOODVILLE SOUTH SA 5011 Saving record 22040467, 231 ESPLANADE SEACLIFF SA 5049 Saving record 22037933, 124 DAYS RD FERRYDEN PARK SA 5010 Saving record 22042666, 43 AVENUE RD PARADISE SA 5075 Saving record 22033778, 48 MORTLOCK TCE PORT LINCOLN SA 5606 Saving record 22031188, 16 OATES ST PORT HUGHES SA 5558 Saving record 22021332, 704-712 PORT WAKEFIELD RD PARAFIELD GARDENS SA 5107 Saving record 22039957, LOT 209 KENTDALE ST GRANGE SA 5022 Saving record 22041369, 18 ROYAL AV BURNSIDE SA 5066 Saving record 22041373, 18 ROYAL AV BURNSIDE SA 5066 Saving record 22041376, 18A ROYAL AV BURNSIDE SA 5066 Saving record 23003191, 8 MULWALA RD COMPTON SA 5291 Saving record 23001423, 24 SERENITY DR SHEIDOW PARK SA 5158 Saving record 23000320, 11 LURLINE AV GILLES PLAINS SA 5086 Saving record 23000733, LOT 3 BARBARA ST PORT ELLIOT SA 5212 Saving record 23002224, 32 STIRLING CT STRATHALBYN SA 5255 Saving record 23004641, LOT 3 SILO RD ELLISTON SA 5670 Saving record 23004063, 19 MERGHINY DR CEDUNA SA 5690 Saving record 22041414, 48 BRUNEL DR MODBURY HEIGHTS SA 5092 Saving record 22039012, 12 STIEF ST WYOMI SA 5275 Saving record 22042806, 23 YOUNG ST BURNSIDE SA 5066 Saving record 22043187, 43A ESPLANADE CHRISTIES BEACH SA 5165 Saving record 22043338, 45 SEA VISTA RD NEPEAN BAY SA 5223 Saving record 22029904, 10 MARINE CT CARRICKALINGA SA 5204 Saving record 22031184, 39 MOUNT OSMOND RD MOUNT OSMOND SA 5064 Saving record 22031134, 71 SCHOFIELDS FTRK STRATHALBYN SA 5255 Saving record 22042671, 1A HERON WAY HALLETT COVE SA 5158 Saving record 22042696, 36 RUSSELL TCE WOODVILLE PARK SA 5011 Saving record 23004042, 77 MYRTLE RD SEACLIFF SA 5049 Saving record 23001461, 10 MEDWAY ST FULLARTON SA 5063 Saving record 23002913, 18 ORCA PL ENCOUNTER BAY SA 5211

Data

Downloaded 2666 times by openaustralia dcalde mlandauer jesse-a-reilly DevAppProject softgrow MichaelBone 93399339 tobiasoberrauch Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (24.2 MB) Use the API

rows 10 / 71364

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 24.2 MB

History

  • Auto ran revision 9fd84356 and completed successfully .
    155 records added, 155 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and completed successfully .
    155 records added, 155 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and completed successfully .
    164 records added, 150 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and completed successfully .
    158 records added, 146 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and completed successfully .
    155 records added, 155 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb