LoveMyData / wollondilly

Wollondilly Shire Council Development Applications


Wollondilly Shire Council Scraper

Wollondilly Shire Council involves the followings * Server - .NET but Java backend * Cookie tracking - Yes - JSESSION * Pagnation - No - However, query is based on 'date' submit via POST * Javascript - No * Clearly defined data within a row - No * iFrame - Why??

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default, today -7 days) * thismonth * lastmonth

Enjoy

Contributors LoveMyData

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.0.0-p481 -----> Installing dependencies using bundler 1.13.7  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Rubygems 2.0.14 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation.  Installing unf_ext 0.0.7.2 with native extensions  Installing httpclient 2.8.2.4  Installing mime-types-data 3.2016.0521  Installing net-http-digest_auth 1.4  Installing net-http-persistent 2.9.4  Installing mini_portile2 2.1.0  Installing pkg-config 1.1.7  Installing ntlm-http 0.1.1  Installing webrobots 0.1.2  Installing sqlite3 1.3.11 with native extensions  Using bundler 1.13.7  Installing unf 0.1.4  Installing mime-types 3.1  Installing nokogiri 1.6.8 with native extensions  Installing sqlite_magic 0.0.6  Installing domain_name 0.5.20160826  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Installing http-cookie 1.0.2  Installing mechanize 2.7.5  Bundle complete! 2 Gemfile dependencies, 19 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (29.33s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Scraping from 20/07/2017 to 27/07/2017, changable via MORPH_PERIOD variable Saving record 471/2017, 29 Albatross Bend WILTON, NSW, 2571 Saving record 472/2017, 32 Albatross Bend WILTON, NSW, 2571 Skipping already saved record 452/2017 Skipping already saved record 451/2017 Saving record 480/2017, 11 Barrallier Avenue TAHMOOR, NSW, 2573 Saving record 463/2017, 17 Barrallier Avenue TAHMOOR, NSW, 2573 Saving record 477/2017, 19 Barrallier Avenue TAHMOOR, NSW, 2573 Saving record 478/2017, 13 Belford Circuit TAHMOOR, NSW, 2573 Saving record 481/2017, 7 Bottlebrush Close PICTON, NSW, 2571 Saving record 484/2017, 109 Burragorang Road MOUNT HUNTER, NSW, 2570 Saving record 528/2016, 12 Caley Street THE OAKS, NSW, 2570 Saving record 473/2017, 19 Campbell Street PICTON, NSW, 2571 Saving record 468/2017, 3 Cristina Place SILVERDALE, NSW, 2752 Saving record 482/2017, 20 Frankum Drive ORANGEVILLE, NSW, 2570 Saving record 483/2017, 3 Gaudry Street THE OAKS, NSW, 2570 Saving record 467/2017, 22 Greenfields Place THERESA PARK, NSW, 2570 Saving record 470/2017, 1 Heritage Drive APPIN, NSW, 2560 Saving record 465/2017, 38 Kalangara Road SILVERDALE, NSW, 2752 Saving record 475/2017, 22 Kennedy Street APPIN, NSW, 2560 Saving record 469/2017, 33 Koolahs Street APPIN, NSW, 2560 Saving record 462/2017, 144 Lyrebird Road PHEASANTS NEST, NSW, 2574 Saving record 485/2017, 23 Macquarie Place TAHMOOR, NSW, 2573 Skipping already saved record 454/2017 Skipping already saved record 456/2017 Skipping already saved record 457/2017 Saving record 466/2017, 59 Oaks Road THIRLMERE, NSW, 2572 Skipping already saved record 459/2017 Saving record 461/2017, 60 Skarrats Lane LAKESLAND, NSW, 2572 Saving record 474/2017, 12 Station Street MENANGLE, NSW, 2568 Skipping already saved record 458/2017 Skipping already saved record 460/2017 Saving record 479/2017, 22 The Grange PICTON, NSW, 2571 Skipping already saved record 453/2017 Saving record 476/2017, 105 Thirlmere Way TAHMOOR, NSW, 2573 Skipping already saved record 455/2017 Saving record 260/2014, 100 Yarran Road BARGO, NSW, 2574 Saving record 464/2017, 87 York Street TAHMOOR, NSW, 2573

Data

Downloaded 111 times by LoveMyData

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (110 KB) Use the API

rows 10 / 363

council_reference address description info_url comment_url date_scraped date_received
57630/2001
38 Appin Road APPIN, NSW, 2560
S96 to Modify Site Layout
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-28
170/2017
74 Appin Road APPIN, NSW, 2560
Continued Use and Change of Use to Manager's Residence
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-21
185/2017
224 Appin Road APPIN, NSW, 2560
Rear Awning over Existing Concrete
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-29
199/2017
25 Arden Road BUXTON, NSW, 2571
Detached Secondary Dwelling & S68 Connect to Existing SMF New Secondary Dwelling New Connect Existing Sewage Management Facil
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-31
148/2017
1115 Burragorang Road BELIMBLA PARK, NSW, 2570
Ancillary Buildings to Dwelling (Shed and Swimming Pool) New Out Building - non-habitable New Swimming Pool
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-15
850/2016
1119 Burragorang Road BELIMBLA PARK, NSW, 2570
S96 Modification
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-10
534/2016
190 Burragorang Road BROWNLOW HILL, NSW, 2570
S96 - Amend External Cladding
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-13
171/2017
190 Burragorang Road BROWNLOW HILL, NSW, 2570
S68 AWTS and Solid Fuel Heater New Septic Tank - Aerated Sub Surface New Solid Fuel Heater
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-21
174/2017
440 Cawdor Road CAWDOR, NSW, 2570
Poultry Farm inc Sheds,Office,Workers Amenities,Earthworks & Infrastru Development of a Livestock Intensive Industry (Poultry) including 8 x Sheds,2 x Machinery Sheds,Composting & Litter Sheds,Office,Workers Amenity,Bulk Earthworks & Associated Infrastructure
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-23
173/2017
220 Charlies Point Road BARGO, NSW, 2574
Erection of a Shed for Use as a Depot
mailto:council@wollondilly.nsw.gov.au
2017-04-11
2017-03-22

Statistics

Average successful run time: less than a minute

Total run time: about 2 hours

Total cpu time used: 2 minutes

Total disk space used: 480 KB

History

  • Auto ran revision ba961f2e and completed successfully .
    27 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    21 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    7 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    13 records added in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    1 record added in the database
  • Auto ran revision ba961f2e and completed successfully .
    7 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    4 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    4 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    7 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    13 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    9 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    1 record added in the database
  • Auto ran revision ba961f2e and completed successfully .
    4 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    8 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    4 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Manually ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Manually ran revision ba961f2e and completed successfully .
    23 records added in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Manually ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    8 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    9 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    4 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    5 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    4 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    1 record added in the database
  • Auto ran revision ba961f2e and completed successfully .
    2 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    5 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    1 record added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    7 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    3 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    5 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    6 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    5 records added in the database
  • Auto ran revision ba961f2e and failed .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Auto ran revision ba961f2e and completed successfully .
    7 records added in the database
  • Auto ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Manually ran revision ba961f2e and completed successfully .
    nothing changed in the database
  • Manually ran revision ba961f2e and completed successfully .
    25 records added in the database
  • Manually ran revision ba961f2e and failed .
    nothing changed in the database
  • Manually ran revision ba961f2e and completed successfully .
    31 records added in the database
  • Manually ran revision ba961f2e and failed .
    nothing changed in the database
  • Manually ran revision ba961f2e and failed .
    nothing changed in the database
  • Manually ran revision 0685b9cb and failed .
    nothing changed in the database
  • Manually ran revision 303a1d92 and failed .
    nothing changed in the database
  • Manually ran revision 656ac2e8 and failed .
    nothing changed in the database
    1 page scraped
  • Manually ran revision 9b3347e9 and failed .
    nothing changed in the database
  • Manually ran revision 6e35abfa and failed .
    nothing changed in the database
  • Manually ran revision abaaacd3 and failed .
    nothing changed in the database
  • Manually ran revision c725d972 and failed .
    nothing changed in the database
    1 page scraped
  • Manually ran revision 71bcd238 and failed .
    nothing changed in the database
  • Manually ran revision cf382e88 and failed .
    nothing changed in the database
  • Manually ran revision cb924fce and failed .
    nothing changed in the database
  • Manually ran revision f94d5b16 and failed .
    nothing changed in the database
  • Manually ran revision f94d5b16 and failed .
    nothing changed in the database
  • Manually ran revision 7493951a and failed .
  • Manually ran revision bf7c03bc and failed .
    nothing changed in the database
  • Created on morph.io

Scraper code

Ruby

wollondilly / scraper.rb