otherchirps / blue-mountains

Blue Mountains City Council Development Applications


Scrapes development applications from the Blue Mountains City Council in a format suitable for use with PlanningAlerts.

Contributors mlandauer CloCkWeRX henare otherchirps

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... -----> Ruby app detected -----> Compiling Ruby/Rack -----> Using Ruby version: ruby-1.9.3 -----> Installing dependencies using 1.7.12  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching additional metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Installing i18n 0.6.0  Installing Ascii85 1.0.1  Installing aasm 3.0.8  Installing multi_json 1.1.0  Installing arel 3.0.2  Installing builder 3.0.0  Installing mime-types 1.17.2  Installing addressable 2.2.7  Installing tzinfo 0.3.33  Installing multi_xml 0.5.1  Installing awesome_print 1.0.2  Installing uuidtools 2.1.3  Installing choice 0.1.4  Installing chronic 0.6.7  Installing json 1.6.5  Installing fastercsv 1.5.4  Installing multipart-post 1.1.5  Installing oauth 0.4.5  Installing ruby-ole 1.2.11.2  Installing log4r 1.1.9  Installing to_regexp 0.1.1  Installing posix-spawn 0.3.6  Installing upsert 0.3.4  Installing fast-stemmer 1.0.1  Installing hashie 1.2.0  Installing gdata 1.1.2  Installing unf_ext 0.0.4  Installing htmlentities 4.3.1  Installing httpauth 0.1  Installing httpclient 2.2.4  Installing nokogiri 1.5.0  Installing net-http-digest_auth 1.1.1  Installing net-http-persistent 2.5.2  Installing ntlm-http 0.1.1  Installing ruby-rc4 0.1.5  Installing shoulda-context 1.0.0  Installing shoulda-matchers 1.0.0  Installing hpricot 0.8.4  Installing polylines 0.1.0  Installing rack 1.4.1  Installing rubyzip 0.9.4  Installing simple_oauth 0.2.0  Installing stringex 1.4.0  Installing trollop 1.16.2  Installing stackdeck 0.2.0  Installing tmail 1.2.7.1  Using bundler 1.7.12  Installing activesupport 3.2.6  Installing sqlite3 1.3.9  Installing httparty 0.8.3  Installing money 5.0.0  Installing jwt 0.1.4  Installing rfgraph 0.3  Installing libxml-ruby 2.8.0  Installing typhoeus 0.3.3  Installing faraday 0.8.1  Installing todonotes 0.1.0  Installing spreadsheet 0.6.5.7  Installing unix_utils 0.0.14  Installing webrobots 0.0.12  Installing unf 0.0.4  Installing sanitize 2.0.3  Installing pdf-reader 1.0.0  Installing shoulda 3.0.1  Installing activemodel 3.2.6  Installing fixed_width-multibyte 0.2.3  Installing hash_digest 1.0.0  Installing sqlite_magic 0.0.3  Installing sinew 1.0.3  Installing fletcher 0.4.1  Installing monster_mash 0.3.0  Installing oauth2 0.5.2  Installing aws-sdk 1.6.1  Installing domain_name 0.5.2  Installing twitter 4.5.0  Installing pismo 0.7.2  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults)  Installing activeresource 3.2.6  Installing activerecord 3.2.6  Installing alchemy_api 0.1.2  Installing google-spreadsheet-ruby 0.1.8  Installing google_drive 0.3.1  Installing highrise 3.0.3  Installing mechanize 2.5.1  Installing active_record_inline_schema 0.5.7  Installing roo 1.10.0  Installing remote_table 2.0.2  Installing errata 1.1.1  Installing data_miner 2.3.4  Your bundle is complete!  Gems in the groups development and test were not installed.  It was installed into ./vendor/bundle  Post-install message from httparty:  When you HTTParty, you must party hard!  Post-install message from pdf-reader:  ********************************************  v1.0.0 of PDF::Reader introduced a new page-based API. There are extensive  examples showing how to use it in the README and examples directory.  For detailed documentation, check the rdocs for the PDF::Reader,  PDF::Reader::Page and PDF::Reader::ObjectHash classes.  The old API is marked as deprecated but will continue to work with no  visible warnings for now.  ********************************************  Bundle completed (43.73s)  Cleaning up the bundler cache.  -----> Discovering process types  Procfile declares types -> scraper  Default process types for Ruby -> rake, console, web Injecting scraper and running... Skipping header row

Data

Downloaded 2 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (28 KB) Use the API

rows 10 / 67

info_url comment_url council_reference on_notice_from on_notice_to address description date_scraped
X/433/2015
2015-06-03
2015-07-03
43 - 51 Great Western Highway, LEURA, NSW
A residential flat building (28 units)
2015-06-15
X/518/2015
2015-06-12
2015-06-26
59 Godson Avenue, BLACKHEATH, NSW
Convert garage to studio and alterations
2015-06-15
X/338/2015
2015-05-27
2015-06-26
52 - 106 Thirroul Avenue, BLACKHEATH, NSW
Additions and alterations to the school building including provision of pedestrian ramps
2015-06-15
X/472/2015
2015-06-12
2015-06-26
17 Roberts Parade, HAWKESBURY HEIGHTS, NSW
An inground swimming pool and a deck
2015-06-15
X/520/2015
2015-06-12
2015-06-26
326 Great Western Highway, LAWSON, NSW
Dual occupancy (detached)
2015-06-15
X/506/2015
2015-06-12
2015-06-26
67 Victoria Street, MOUNT VICTORIA, NSW
A single storey dwelling and a solid fuel heater
2015-06-15
X/449/2015
2015-06-12
2015-06-26
45 Toulon Avenue, WENTWORTH FALLS, NSW
A pergola
2015-06-15
X/500/2015
2015-06-11
2015-06-25
3 McCall Avenue, BLACKHEATH, NSW
A single storey dwelling, detached garage and solid fuel heater
2015-06-15
XM/799/2014/A
2015-06-11
2015-06-25
35 Second Street, BLACKHEATH, NSW
S96 and cc modification - adds/alts to dwelling, a detached carport and a solid fuel heater
2015-06-15
X/499/2015
2015-06-11
2015-06-25
31 Kingfisher Crescent, BULLABURRA, NSW
An awning
2015-06-15

Statistics

Average successful run time: 1 minute

Total run time: 2 minutes

Total cpu time used: less than 10 seconds

Total disk space used: 55.6 KB

History

  • Manually ran revision 30ab78c0 and completed successfully .
    11 records added in the database
  • Manually ran revision 30ab78c0 and completed successfully .
    56 records added in the database
    9 pages scraped
  • Created on morph.io

Scraper code

Ruby

blue-mountains / scraper.rb