planningalerts-scrapers / wollondilly

Wollondilly Shire Council Development Applications


Wollondilly now has a custom web application which has an API. So, we're using that.

Contributors LoveMyData mlandauer tseyiokorodudu

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.5.1 -----> Installing dependencies using bundler version 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/......  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching unf_ext 0.0.7.6  Fetching httpclient 2.8.3  Installing connection_pool 2.2.2  Installing httpclient 2.8.3  Fetching mime-types-data 3.2019.0331  Installing mime-types-data 3.2019.0331  Fetching net-http-digest_auth 1.4.1  Fetching mini_portile2 2.4.0  Installing mini_portile2 2.4.0  Installing net-http-digest_auth 1.4.1  Fetching ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing ntlm-http 0.1.1  Fetching sqlite3 1.3.13  Installing webrobots 0.1.2  Fetching net-http-persistent 3.0.1  Installing unf_ext 0.0.7.6 with native extensions  Installing net-http-persistent 3.0.1  Fetching mime-types 3.2.2  Installing mime-types 3.2.2  Installing sqlite3 1.3.13 with native extensions  Fetching nokogiri 1.10.3  Installing nokogiri 1.10.3 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.6  Installing mechanize 2.7.6  Bundle complete! 2 Gemfile dependencies, 19 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (26.31s)  Cleaning up the bundler cache.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Using Australian proxy... Storing DA/2002/54824/2 - 26 Station Street THIRLMERE, NSW Storing DA/2005/54657/5 - 29 Bobs Range Road ORANGEVILLE, NSW Storing DA/2013/301/3 - 246 Donalds Range Road RAZORBACK, NSW Storing DA/2018/339/6 - 990 Picton Road WILTON, NSW Storing DA/2018/352/2 - 30 Greenacre Drive TAHMOOR, NSW Storing DA/2019/393/2 - 1 Devitt Crescent THE OAKS, NSW Storing DA/2020/502/2 - 1117 Argyle Street WILTON, NSW Storing DA/2021/107/2 - 450 Arina Road BARGO, NSW Storing DA/2021/525/2 - 7 Gumtree Terrace TAHMOOR, NSW Storing DA/2022/1/1 - 1150 Werombi Road THERESA PARK, NSW Storing DA/2022/2/1 - 37 Resthaven Way SILVERDALE, NSW Storing DA/2022/3/1 - 103 Ambervale Drive ORANGEVILLE, NSW Storing DA/2022/4/1 - 46 Green Hills Drive SILVERDALE, NSW Storing DA/2022/5/1 - 46 Green Hills Drive SILVERDALE, NSW Storing DA/2022/6/1 - 115 Reservoir Road BARGO, NSW Storing DA/2022/7/1 - 16 Torulosa Road THE OAKS, NSW Storing DA/2022/8/1 - 1060 Barkers Lodge Road OAKDALE, NSW Storing DA/2022/9/1 - 20 Mortimer Street YANDERRA, NSW Storing DA/2022/10/1 - 2 Lewis Street SILVERDALE, NSW Storing DA/2022/11/1 - 98 Stonebrook Meadows Road RAZORBACK, NSW Storing DA/2022/12/1 - Marsh Road SILVERDALE, NSW Storing DA/2022/13/1 - Marsh Road SILVERDALE, NSW Storing DA/2022/14/1 - 24 Highland Crescent THIRLMERE, NSW Storing DA/2022/15/1 - 27 Ridgehaven Road SILVERDALE, NSW Storing DA/2022/16/1 - 38 Woodlands Way ORANGEVILLE, NSW Storing DA/2022/17/1 - 70 Frankum Drive ORANGEVILLE, NSW Storing DA/2022/18/1 - 70 Tickle Drive THIRLMERE, NSW Storing DA/2022/19/1 - 50 Hawthorne Road BARGO, NSW Storing DA/2022/20/1 - 19 The Acres Way TAHMOOR, NSW Storing DA/2022/21/1 - 25 Wrightson Way DOUGLAS PARK, NSW Storing DA/2022/22/1 - 25 Wrightson Way DOUGLAS PARK, NSW Storing DA/2022/23/1 - 75 Johnston Road BARGO, NSW Storing DA/2022/24/1 - 6 Resthaven Way SILVERDALE, NSW Storing DA/2022/25/1 - 125 Fallons Road ORANGEVILLE, NSW Storing DA/2022/26/1 - Macquarie Place TAHMOOR, NSW Storing DA/2022/27/1 - 113 Menangle Road MENANGLE, NSW Storing DA/2022/28/1 - 2770 Remembrance Driveway TAHMOOR, NSW Storing DA/2022/29/1 - 324 Avoca Road SILVERDALE, NSW Storing DA/2022/30/1 - 37 Thirlmere Way TAHMOOR, NSW Storing DA/2022/31/1 - 10 Attunga Close PICTON, NSW Storing DA/2022/32/1 - 7 Gumtree Terrace TAHMOOR, NSW Storing DA/2022/33/1 - 587 Spring Creek Road MOUNT HUNTER, NSW Storing DA/2022/34/1 - 587 Spring Creek Road MOUNT HUNTER, NSW Storing DA/2022/35/1 - 90 West Parade THIRLMERE, NSW Storing DA/2022/36/1 - 2 Major Roberts Avenue TAHMOOR, NSW Storing DA/2022/37/1 - 70 Thurns Road RAZORBACK, NSW Storing DA/2022/38/1 - 70 Thurns Road RAZORBACK, NSW Storing DA/2022/39/1 - 55 Bronzewing Street TAHMOOR, NSW Storing DA/2022/40/1 - 2 Eliza Place PICTON, NSW Storing DA/2022/41/1 - 60 Binalong Road BELIMBLA PARK, NSW Storing DA/2022/42/1 - 2 McIntosh Street THE OAKS, NSW Storing DA/2022/43/1 - Marsh Road SILVERDALE, NSW Storing DA/2022/44/1 - 1428 Burragorang Road OAKDALE, NSW Storing CDC/2022/5/1 - 39 Kyeema Street PICTON, NSW Storing CDC/2022/6/1 - 61 Tiger Street SILVERDALE, NSW Storing CDC/2022/7/1 - 33 Torumba Circuit SILVERDALE, NSW Storing CDC/2022/8/1 - 37 Bullen Drive SILVERDALE, NSW Storing CDC/2022/9/1 - 71 Fairway Drive WILTON, NSW Storing CDC/2022/10/1 - 14 Green Hills Drive SILVERDALE, NSW Storing CDC/2022/11/1 - 52 Walmsley Crescent SILVERDALE, NSW Storing CDC/2022/14/1 - 27 Taylors Road SILVERDALE, NSW

Data

Downloaded 520 times by openaustralia dcalde muzzie86

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (420 KB) Use the API

rows 10 / 1232

council_reference address description info_url date_scraped date_received on_notice_from on_notice_to lat lng
DA/2015/605/2
21 Station Master Avenue THIRLMERE, NSW
Modification to condition 24(5) for purpose of installing dog-proof fencing to prevent dogs from entering the conservation area
2020-07-29
2020-07-09
-34.2015815866067
150.57718975772
DA/2017/7/2
40 Burragorang Street THE OAKS, NSW
Modification - design changes
2020-07-29
2020-07-09
2020-07-15
2020-07-29
-34.0785271073153
150.566013656916
DA/2018/755/3
18 Torumba Circuit SILVERDALE, NSW
Modification - Modify condition 13 (1)
2020-07-29
2020-07-10
-33.9125745023159
150.604898799426
DA/2019/597/3
85 Eagles Road RAZORBACK, NSW
Modification - design changes
2020-07-29
2020-07-06
2020-07-10
2020-07-24
-34.1315649588424
150.672930057128
DA/2019/658/2
2 Kalangara Road SILVERDALE, NSW
Modification - design changes
2020-07-29
2020-07-03
-33.911332578772
150.604277567584
DA/2020/170/2
120 Burrells Road MENANGLE, NSW
Modification - include temporary occupation
2020-07-29
2020-07-07
-34.1561082189654
150.691282209498
DA/2020/231/2
8 Hassall Road BUXTON, NSW
Modification - Design changes
2020-07-29
2020-07-10
-34.2487649560524
150.540986009254
DA/2020/315/1
529 Old Razorback Road CAWDOR, NSW
Dwelling additions and s68 connect to existing
2020-07-29
2020-06-29
-34.1159615115923
150.668708408434
DA/2020/316/1
5 Camellia Court PICTON, NSW
Secondary dwelling & retaining walls
2020-07-29
2020-06-29
-34.1813143878095
150.614690808844
DA/2020/317/1
5 Dillies Lane TAHMOOR, NSW
Dwelling house
2020-07-29
2020-06-29
-34.2187083600634
150.604580262409

Statistics

Average successful run time: 3 minutes

Total run time: about 18 hours

Total cpu time used: 10 minutes

Total disk space used: 766 KB

History

  • Auto ran revision 95d5b551 and completed successfully .
    61 records added, 58 records removed in the database
  • Auto ran revision 95d5b551 and completed successfully .
    58 records added, 57 records removed in the database
  • Auto ran revision 95d5b551 and completed successfully .
    59 records added, 59 records removed in the database
  • Auto ran revision 95d5b551 and completed successfully .
    68 records added, 68 records removed in the database
  • Auto ran revision 95d5b551 and completed successfully .
    79 records added, 76 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

wollondilly / scraper.rb