planningalerts-scrapers / wollondilly

Wollondilly Shire Council Development Applications

README.md

Wollondilly now has a custom web application which has an API. So, we're using that.

Contributors LoveMyData mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.5.1 -----> Installing dependencies using bundler version 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/......  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching unf_ext 0.0.7.6  Fetching httpclient 2.8.3  Installing connection_pool 2.2.2  Installing httpclient 2.8.3  Installing unf_ext 0.0.7.6 with native extensions  Fetching mime-types-data 3.2019.0331  Installing mime-types-data 3.2019.0331  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching mini_portile2 2.4.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Installing mini_portile2 2.4.0  Fetching webrobots 0.1.2  Installing webrobots 0.1.2  Fetching sqlite3 1.3.13  Fetching net-http-persistent 3.0.1  Installing net-http-persistent 3.0.1  Installing sqlite3 1.3.13 with native extensions  Fetching mime-types 3.2.2  Installing mime-types 3.2.2  Fetching nokogiri 1.10.3  Installing nokogiri 1.10.3 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.6  Installing mechanize 2.7.6  Bundle complete! 2 Gemfile dependencies, 19 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (21.46s)  Cleaning up the bundler cache.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Storing DA/1997/173/4 - 129 Silica Road YANDERRA, NSW Storing DA/2014/331/2 - 30 Bronzewing Street West TAHMOOR, NSW Storing DA/2016/168/2 - 6 Adelaide Place SILVERDALE, NSW Storing DA/2016/412/6 - 1 Cooper Drive OAKDALE, NSW Storing DA/2017/123/2 - 115 Stratford Road TAHMOOR, NSW Storing DA/2018/202/2 - 235 Braddocks Road WEROMBI, NSW Storing DA/2018/615/2 - 80 Quarry Road THE OAKS, NSW Storing DA/2019/452/3 - 120 Silverdale Road THE OAKS, NSW Storing DA/2019/650/2 - 19 Wellers Road BARGO, NSW Storing DA/2020/132/2 - 38 The Acres Way TAHMOOR, NSW Storing DA/2020/619/2 - 93 Avon Dam Road BARGO, NSW Storing DA/2020/804/2 - 180 Biffins Lane CAWDOR, NSW scraper.rb:23:in `block in <main>': undefined method `>=' for nil:NilClass (NoMethodError) from scraper.rb:21:in `each' from scraper.rb:21:in `<main>'

Data

Downloaded 228 times by openaustralia dcalde muzzie86

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (290 KB) Use the API

rows 10 / 843

council_reference address description info_url date_scraped date_received on_notice_from on_notice_to lat lng
DA/2015/605/2
21 Station Master Avenue THIRLMERE, NSW
Modification to condition 24(5) for purpose of installing dog-proof fencing to prevent dogs from entering the conservation area
https://tracking.wollondilly.nsw.gov.au/detail/010.2015.00000605.002
2020-07-29
2020-07-09
-34.2015815866067
150.57718975772
DA/2017/7/2
40 Burragorang Street THE OAKS, NSW
Modification - design changes
https://tracking.wollondilly.nsw.gov.au/detail/010.2017.00000007.002
2020-07-29
2020-07-09
2020-07-15
2020-07-29
-34.0785271073153
150.566013656916
DA/2018/755/3
18 Torumba Circuit SILVERDALE, NSW
Modification - Modify condition 13 (1)
https://tracking.wollondilly.nsw.gov.au/detail/010.2018.00000755.003
2020-07-29
2020-07-10
-33.9125745023159
150.604898799426
DA/2019/597/3
85 Eagles Road RAZORBACK, NSW
Modification - design changes
https://tracking.wollondilly.nsw.gov.au/detail/010.2019.00000597.003
2020-07-29
2020-07-06
2020-07-10
2020-07-24
-34.1315649588424
150.672930057128
DA/2019/658/2
2 Kalangara Road SILVERDALE, NSW
Modification - design changes
https://tracking.wollondilly.nsw.gov.au/detail/010.2019.00000658.002
2020-07-29
2020-07-03
-33.911332578772
150.604277567584
DA/2020/170/2
120 Burrells Road MENANGLE, NSW
Modification - include temporary occupation
https://tracking.wollondilly.nsw.gov.au/detail/010.2020.00000170.002
2020-07-29
2020-07-07
-34.1561082189654
150.691282209498
DA/2020/231/2
8 Hassall Road BUXTON, NSW
Modification - Design changes
https://tracking.wollondilly.nsw.gov.au/detail/010.2020.00000231.002
2020-07-29
2020-07-10
-34.2487649560524
150.540986009254
DA/2020/315/1
529 Old Razorback Road CAWDOR, NSW
Dwelling additions and s68 connect to existing
https://tracking.wollondilly.nsw.gov.au/detail/010.2020.00000315.001
2020-07-29
2020-06-29
-34.1159615115923
150.668708408434
DA/2020/316/1
5 Camellia Court PICTON, NSW
Secondary dwelling & retaining walls
https://tracking.wollondilly.nsw.gov.au/detail/010.2020.00000316.001
2020-07-29
2020-06-29
-34.1813143878095
150.614690808844
DA/2020/317/1
5 Dillies Lane TAHMOOR, NSW
Dwelling house
https://tracking.wollondilly.nsw.gov.au/detail/010.2020.00000317.001
2020-07-29
2020-06-29
-34.2187083600634
150.604580262409

Statistics

Average successful run time: 4 minutes

Total run time: about 12 hours

Total cpu time used: 5 minutes

Total disk space used: 630 KB

History

  • Auto ran revision 06a19fe9 and failed .
    12 records added, 12 records removed in the database
  • Auto ran revision 06a19fe9 and failed .
    12 records added, 11 records removed in the database
  • Auto ran revision 06a19fe9 and failed .
    11 records added, 11 records removed in the database
  • Auto ran revision 06a19fe9 and failed .
    11 records added, 11 records removed in the database
  • Auto ran revision 06a19fe9 and failed .
    12 records added, 12 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

wollondilly / scraper.rb