LoveMyData / blacktown_city

Blacktown City Council Development Applications

Scrapes services.blacktown.nsw.gov.au


Contributors LoveMyData henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching unf_ext 0.0.7.4  Fetching httpclient 2.8.3  Fetching mime-types-data 3.2016.0521  Installing mime-types-data 3.2016.0521  Installing httpclient 2.8.3  Installing unf_ext 0.0.7.4 with native extensions  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching net-http-persistent 2.9.4  Installing net-http-persistent 2.9.4  Fetching mini_portile2 2.2.0  Installing mini_portile2 2.2.0  Fetching ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing ntlm-http 0.1.1  Installing webrobots 0.1.2  Fetching sqlite3 1.3.13  Fetching mime-types 3.1  Installing sqlite3 1.3.13 with native extensions  Installing mime-types 3.1  Fetching nokogiri 1.8.0  Installing nokogiri 1.8.0 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20170404  Installing domain_name 0.5.20170404  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.5  Installing mechanize 2.7.5  Bundle complete! 2 Gemfile dependencies, 18 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (26.28s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Skipping already saved record DA-17-01903 Skipping already saved record DA-17-02388 Skipping already saved record DA-18-00127 Skipping already saved record DA-18-00161 Skipping already saved record DA-18-00317 Skipping already saved record DA-18-00341 Skipping already saved record DA-18-00457 Skipping already saved record DA-18-00494 Skipping already saved record DA-18-00500 Skipping already saved record DA-18-00559 Skipping already saved record DA-18-00561 Skipping already saved record DA-18-00614 Saved record DA-18-00639 Saved record DA-18-00668 Skipping already saved record DA-18-00673 Skipping already saved record DA-18-00679 Skipping already saved record DA-18-00683 Skipping already saved record DA-18-00687 Skipping already saved record DA-18-00699 Skipping already saved record DA-18-00717 Saved record DA-18-00725 Skipping already saved record DA-18-00732 Skipping already saved record DA-18-00761 Skipping already saved record DA-18-00852 Skipping already saved record DA-18-00859 Skipping already saved record DA-18-00864 Skipping already saved record DA-18-00868 Saved record DA-18-00870 Skipping already saved record DA-18-00881 Skipping already saved record DA-18-00949 Skipping already saved record DA-18-00955 Skipping already saved record DA-18-00956 Skipping already saved record DA-18-00958 Skipping already saved record DA-18-00961 Skipping already saved record DA-18-00963 Skipping already saved record DA-18-00964 Skipping already saved record DA-18-00965 Skipping already saved record DA-18-00966 Skipping already saved record DA-18-00967 Skipping already saved record DA-18-00968 Skipping already saved record DA-18-00969 Skipping already saved record DA-18-00970 Skipping already saved record DA-18-00971 Skipping already saved record DA-18-00981 Skipping already saved record DA-18-00982 Skipping already saved record DA-18-00983 Skipping already saved record DA-18-00985 Skipping already saved record DA-18-00986 Skipping already saved record DA-18-00990 Skipping already saved record DA-18-00991 Skipping already saved record DA-18-00992 Skipping already saved record DA-18-00993 Skipping already saved record DA-18-00994 Skipping already saved record DA-18-00995 Skipping already saved record DA-18-00997 Skipping already saved record DA-18-00998 Saved record DA-18-00999 Saved record DA-18-01001 Saved record DA-18-01003 Saved record DA-18-01004 Saved record DA-18-01005 Saved record DA-18-01006 Saved record DA-18-01007 Saved record DA-18-01008 Saved record DA-18-01009 Saved record DA-18-01010 Saved record DA-18-01011 Saved record DA-18-01012 Saved record DA-18-01013 Saved record DA-18-01014 Saved record DA-18-01015 Saved record DA-18-01016 Saved record DA-18-01017 Saved record DA-18-01018 Saved record DA-18-01021 Skipping already saved record MOD-17-00272 Skipping already saved record MOD-18-00115 Skipping already saved record MOD-18-00202 Skipping already saved record MOD-18-00224 Skipping already saved record SPP-18-00007

Data

Downloaded 559 times by LoveMyData MikeRalphson Lyndall equivalentideas

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.99 MB) Use the API

rows 10 / 5356

council_reference address description date_received date_scraped info_url comment_url communt_url
DA-16-03444
44 Darlington Street STANHOPE GARDENS NSW 2768
Awning
2016-04-14
2016-04-23
DA-16-03436
45 Matthew Crescent BLACKTOWN NSW 2148
Demolition
2016-04-12
2016-04-23
DA-16-02936
154 Best Road SEVEN HILLS NSW 2147
Other Development
2016-02-05
2016-04-23
DA-16-03442
15 Medwin Place QUAKERS HILL NSW 2763
Dwelling - First Floor Addition
2016-04-13
2016-04-23
DA-16-03275
Oatley Close BLACKTOWN NSW 2148
Change of use
2016-03-17
2016-04-23
DA-16-03440
49 Sebastian Crescent COLEBEE NSW 2761
Change of use
2016-04-13
2016-04-23
DA-16-03443
6 Kippax Place SHALVEY NSW 2770
Dwelling - Alterations/Additions
2016-04-14
2016-04-23
DA-16-03439
Pomeroy Street SCHOFIELDS NSW 2762
Dwelling - 2 Storeys, Garage
2016-04-13
2016-04-23
MOD-15-02715
3 Joshua Way DEAN PARK NSW 2761
S96(1A) - Minimal enviornmental impact
2015-12-30
2016-04-23
DA-16-03457
66 Arnott Road MARAYONG NSW 2148
Dwelling - First Floor Addition
2016-04-15
2016-04-23

Statistics

Average successful run time: less than a minute

Total run time: 1 day

Total cpu time used: 14 minutes

Total disk space used: 2.02 MB

History

  • Auto ran revision 08951a26 and completed successfully .
    23 records added in the database
    1 page scraped
  • Auto ran revision 08951a26 and completed successfully .
    1 record added in the database
    1 page scraped
  • Auto ran revision 08951a26 and completed successfully .
    3 records added in the database
    1 page scraped
  • Auto ran revision 08951a26 and completed successfully .
    12 records added in the database
  • Auto ran revision 08951a26 and completed successfully .
    13 records added in the database
    1 page scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

blacktown_city / scraper.rb