planningalerts-scrapers / sunshine_coast

Sunshine Coast Council Development Applications


Sunshine coast council scraper

This site is a typical .NET site involves the followings.

  • Cookie - No
  • JSON - Yes
  • Structured HTML - No (especially address is a nightmare)

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default, last 14 days)
  • thismonth
  • lastmonth

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting 'Last 14 Days' data, changable via MORPH_PERIOD environment Saving record POS22/0204, 26 Access Crescent Coolum Beach Saving record POS22/0217, Stage 1, 33 Palmwoods School Rd PALMWOODS Saving record OPW22/0647, 7 Gossamer Dr BUDERIM Saving record MCU23/0006, 138 Old Gympie Rd MOOLOOLAH VALLEY Saving record OPW22/0319.01, 1, 3 and 5 Commercial Rd KULUIN Saving record OPW23/0010, 6 & 8 Mooloolah Rd, MOOLOOLAH VALLEY Saving record OPW23/0011, 21 Whispering Gum Ave, EUMUNDI Saving record RAL23/0002, 614 Yandina Bli Bli Rd MAROOCHY RIVER Saving record OPW23/0012, 2 Marigold St CURRIMUNDI Saving record REC15/0043.04, 14-16 Jorl Cout BUDERIM Saving record POS22/0234, 333 and 343 Forestry Rd LANDSBOROUGH Saving record OPW23/0013, 40 Mahogany Dr PELICAN WATERS Saving record MCU23/0007, 11 Kookaburra Crescent, Bokarina Saving record OPW23/0014, 11 Kookaburra Crescent, Bokarina Saving record POS22/0244, 514 & 588 North Arm Yandina Creek Rd YANDINA CREEK Saving record POS23/0004, Stage 2 Saving record POS23/0006, 12 Greber Road BEERWAH Saving record OPW21/0647.02, 37 and 39 Moreton Pde KINGS BEACH Saving record OPW21/0735.01, Twin Waters Resort 270 Ocean Dr TWIN WATERS Saving record MCU23/0008, 4/29-31 Fred Chaplin Cct CORBOULD PARK Saving record OPW23/0015, 2 Kuran St MAROOCHYDORE Saving record MCU23/0009, 1062 Maleny-Kenilworth Rd ELAMAN CREEK Saving record RAL23/0003, 6 Alexandra St KENILWORTH Saving record OPW23/0016, 1046-1052 Yandina Coolum Rd MAROOCHY RIVER Saving record MCU15/0214.24, 27 Green Turtle Place MARCOOLA Saving record RAL18/0009.38, 7 Shoalwater Cct PELICAN WATERS Saving record MCU23/0010, 10 Harvest Rd YANDINA Saving record OPW23/0017, 10 Harvest Rd YANDINA Saving record OPW23/0018, 3 James St DICKY BEACH Saving record RAL18/0173.06, Panorama Estate Saving record OPW23/0020, 7 and 5 Candle Vine St MALENY Saving record OPW23/0023, 6 Alexandra St KENILWORTH Saving record RAL23/0004, Peter Crosby Way PALMVIEW Saving record POS22/0237, Stage 2 Saving record POS22/0240, 92 Roderick Street MOFFAT BEACH Saving record POS22/0245, 118 Millwell Road Maroochydore Saving record OPW23/0019, 49 Yellow Cedar Place PALMWOODS Saving record REC02/0208.03, 160 Towen Mount Rd TOWEN MOUNTAIN Saving record MCU23/0011, 1-23 Arcoona Rd COOLUM BEACH Saving record OPW23/0021, 1-23 Arcoona Rd COOLUM BEACH Saving record REC99/8119.23, 37B CICADA CLOSE BUDERIM Saving record OPW23/0022, Maclamond Drive PELICAN WATERS Saving record MCU04/0205.01, 86 Palmwoods-Montville Rd Palmwoods Saving record OPW23/0024, 8 Clifton Pl EUMUNDI Saving record MCU22/0028.01, 20 Queen of Colonies Pde MOFFAT BEACH Saving record MCU17/0148.08, Curve Birtinya 105/1 Mantra Esp BIRTINYA Saving record OPW23/0025, 33 Stark Lane FOREST GLEN Saving record MCU13/0019.07, Sunshine Cove Way Maroochydore Saving record MCU17/2067.02, 60 Dalton Drive MAROOCHYDORE Saving record OPW23/0026, Harmony Release 20 Saving record OPW22/0095.01, 40 Menzies Dr PACIFIC PARADISE Saving record MCU19/0115.01, 1-3 Kessling Ave KUNDA PARK

Data

Downloaded 13004 times by slav123 openaustralia LoveMyData dcalde mlandauer DevAppProject WilliamYun jesse-a-reilly TheZepto spatialsteve dsccramb equivalentideas yourneighbourhood ajoh3691 Jedwards157 EMBLAPROJECTS Corris224

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3.89 MB) Use the API

rows 10 / 10664

date_scraped description info_url date_received council_reference address comment_url
2012-08-27
Indoor Recreation Sports Centre - Studdock Investments Pty Ltd & Cricket Generation Pty Ltd C/
2012-08-01
MCU12/0111
84 Sandalwood Lane, FOREST GLEN, QLD
2012-08-27
Extension to Relevant Period
2012-08-01
132006.1360.03
8 William St, NOOSAVILLE, QLD
2012-08-27
Generally in Accordance
2012-08-01
132007.1841.04
26 Eenie Creek Rd, NOOSAVILLE, QLD
2012-08-27
Generally in Accordance
2012-08-01
2008/56M0014.02
476 Nicklin Way, WURTULLA, QLD
2012-08-27
45 Primary School Crt - Conversion of Dual Occupancy to Offices - Martin Kurvink
2012-08-02
MCU12/0112
43-45 Primary School Ct, MAROOCHYDORE, QLD
2012-08-27
Warehouse - Uptons Salvage Trading Pty Ltd
2012-08-02
MCU12/0113
31 Moroney Pl, BEERWAH, QLD
2012-08-27
62 Duporth Ave MAROOCHYDORE - Generally In Accordance
2012-08-03
MCU02/0088.05
52 Duporth Ave, MAROOCHYDORE, QLD
2012-08-27
Extension to Relevant Period - Furno's Pty Ltd
2012-08-03
MCU07/0214.03
88 Burnett St, BUDERIM, QLD
2012-08-27
Development Application for Material Change of Use to establish an Sport and Recreation (Naval Cadet Unit)
2012-08-03
1998/510053.01
Maleny-Stanley River Rd, MALENY, QLD
2012-08-27
Infrastructure Agreement
2012-08-06
MCU10/0052.01
0 Parklakes Dr, BLI BLI, QLD

Statistics

Average successful run time: less than a minute

Total run time: 30 days

Total cpu time used: about 1 hour

Total disk space used: 3.93 MB

History

  • Auto ran revision 6a6ca4ed and completed successfully .
    52 records added, 52 records removed in the database
  • Auto ran revision 6a6ca4ed and completed successfully .
    57 records added, 46 records removed in the database
  • Auto ran revision 6a6ca4ed and completed successfully .
    52 records added, 41 records removed in the database
  • Auto ran revision 6a6ca4ed and completed successfully .
    49 records added, 41 records removed in the database
  • Auto ran revision 6a6ca4ed and completed successfully .
    45 records added, 45 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

sunshine_coast / scraper.rb