planningalerts-scrapers / parramatta

Parramatta City Council Development Applications


Parramatta Council Scraper

  • Scrape detail page - Yes
  • Clearly defined data within a row - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • 7 (days)
  • 14 (days) - default
  • thismonth
  • lastmonth

Enjoy

Note: XML feed on the site appears to be broken

Forked from ScraperWiki

Contributors henare LoveMyData equivalentideas jamezpolley mlandauer

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.4.1 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.  Retrying dependency api due to error (2/4): Bundler::HTTPError Network error while fetching https://index.rubygems.org/api/v1/dependencies?gems=addressable%2Cconnection_pool%2Ccrack%2Cdomain_name%2Chashdiff%2Chttp-cookie%2Chttpclient%2Cmechanize%2Cmime-types%2Cmime-types-data%2Cmini_portile2%2Cnet-http-digest_auth%2Cnet-http-persistent%2Cnokogiri%2Cntlm-http%2Cpublic_suffix%2Csafe_yaml%2Cscraperwiki%2Csqlite3%2Csqlite_magic%2Ctimecop%2Cunf%2Cunf_ext%2Cvcr%2Cwebmock%2Cwebrobots (execution expired)...........  Fetching version metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching public_suffix 3.1.0  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching safe_yaml 1.0.5  Installing connection_pool 2.2.2  Installing safe_yaml 1.0.5  Fetching unf_ext 0.0.7.5  Installing public_suffix 3.1.0  Fetching hashdiff 0.4.0  Installing hashdiff 0.4.0  Fetching httpclient 2.8.3  Fetching mime-types-data 3.2018.0812  Installing unf_ext 0.0.7.5 with native extensions  Installing httpclient 2.8.3  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching mini_portile2 2.3.0  Installing mini_portile2 2.3.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing webrobots 0.1.2  Fetching sqlite3 1.3.13  Installing sqlite3 1.3.13 with native extensions  Fetching timecop 0.9.1  Installing timecop 0.9.1  Fetching vcr 5.0.0  Installing vcr 5.0.0  Fetching net-http-persistent 3.0.0  Installing net-http-persistent 3.0.0  Fetching crack 0.4.3  Installing crack 0.4.3  Fetching addressable 2.6.0  Installing addressable 2.6.0  Fetching nokogiri 1.8.5  Installing nokogiri 1.8.5 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching webmock 3.6.0  Installing webmock 3.6.0  Fetching sqlite_magic 0.0.6  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Gem::RemoteFetcher::UnknownHostError: timed out  (https://rubygems.org/gems/mime-types-data-3.2018.0812.gem)  An error occurred while installing mime-types-data (3.2018.0812), and Bundler  cannot continue.  Make sure that `gem install mime-types-data -v '3.2018.0812'` succeeds before  bundling.  In Gemfile:  mechanize was resolved to 2.7.6, which depends on  mime-types was resolved to 3.2.2, which depends on  mime-types-data  Bundler Output: Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.  Retrying dependency api due to error (2/4): Bundler::HTTPError Network error while fetching https://index.rubygems.org/api/v1/dependencies?gems=addressable%2Cconnection_pool%2Ccrack%2Cdomain_name%2Chashdiff%2Chttp-cookie%2Chttpclient%2Cmechanize%2Cmime-types%2Cmime-types-data%2Cmini_portile2%2Cnet-http-digest_auth%2Cnet-http-persistent%2Cnokogiri%2Cntlm-http%2Cpublic_suffix%2Csafe_yaml%2Cscraperwiki%2Csqlite3%2Csqlite_magic%2Ctimecop%2Cunf%2Cunf_ext%2Cvcr%2Cwebmock%2Cwebrobots (execution expired)...........  Fetching version metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching public_suffix 3.1.0  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching safe_yaml 1.0.5  Installing connection_pool 2.2.2  Installing safe_yaml 1.0.5  Fetching unf_ext 0.0.7.5  Installing public_suffix 3.1.0  Fetching hashdiff 0.4.0  Installing hashdiff 0.4.0  Fetching httpclient 2.8.3  Fetching mime-types-data 3.2018.0812  Installing unf_ext 0.0.7.5 with native extensions  Installing httpclient 2.8.3  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching mini_portile2 2.3.0  Installing mini_portile2 2.3.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing webrobots 0.1.2  Fetching sqlite3 1.3.13  Installing sqlite3 1.3.13 with native extensions  Fetching timecop 0.9.1  Installing timecop 0.9.1  Fetching vcr 5.0.0  Installing vcr 5.0.0  Fetching net-http-persistent 3.0.0  Installing net-http-persistent 3.0.0  Fetching crack 0.4.3  Installing crack 0.4.3  Fetching addressable 2.6.0  Installing addressable 2.6.0  Fetching nokogiri 1.8.5  Installing nokogiri 1.8.5 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching webmock 3.6.0  Installing webmock 3.6.0  Fetching sqlite_magic 0.0.6  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Gem::RemoteFetcher::UnknownHostError: timed out  (https://rubygems.org/gems/mime-types-data-3.2018.0812.gem)  An error occurred while installing mime-types-data (3.2018.0812), and Bundler  cannot continue.  Make sure that `gem install mime-types-data -v '3.2018.0812'` succeeds before  bundling.   In Gemfile:  mechanize was resolved to 2.7.6, which depends on  mime-types was resolved to 3.2.2, which depends on  mime-types-data  !  ! Failed to install gems via Bundler.  !

Data

Downloaded 8275 times by slav123 openaustralia DevAppProject WilliamYun LoveMyData irwin-r jesse-a-reilly tatol86 TheZepto wbchilds razz969 brickworksbp z37214728 Jedwards157 jayzhouny Jeremy-Charlton Dave9129 ajoh3691 equivalentideas jacroft408 henare kadrach horizons444 hoyathorwood

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (9.35 MB) Use the API

rows 10 / 24080

description info_url date_received council_reference address comment_url date_scraped date
Removal 2 Trees
2010-12-09
TA/732/2010
5 Willoughby Street EPPING NSW 2121
2010-12-12 01:15:22
2010-12-09 00:00:00
Removal of 1 Tree
2010-12-09
TA/733/2010
18 / 47 Wentworth Avenue WESTMEAD NSW 2145
2010-12-12 01:15:22
2010-12-09 00:00:00
Removal of 2 Trees
2010-12-09
TA/734/2010
30 Brabyn Street NORTH PARRAMATTA NSW 2151
2010-12-12 01:15:22
2010-12-09 00:00:00
Removal of 1 Tree
2010-12-09
TA/735/2010
37 - 43 Stewart Street DUNDAS VALLEY NSW 2117
2010-12-12 01:15:22
2010-12-09 00:00:00
Section 96(1a) modification to an approved dwelling including raising floor levels by 300mm
2010-12-09
DA/485/2010/A
48 - 50 Meadows Street MERRYLANDS NSW 2160
2010-12-12 01:15:21
2010-12-09 00:00:00
Demolition and construction of a 2 storey residential flat building under the Affordable Rental Housing SEPP containing 12 apartments over basement carparking with access from Barangaroo Road
2010-12-09
DA/997/2010
4 Barangaroo Road TOONGABBIE NSW 2146
2010-12-12 01:15:21
2010-12-09 00:00:00
Demolition, tree removal and construction of a 3 storey residential flat building containing 12 units over basement carparking with strata subdivision
2010-12-09
DA/998/2010
34 Napier Street PARRAMATTA NSW 2150
2010-12-12 01:15:21
2010-12-09 00:00:00
Construction of a two storey dwelling house and tree removal
2010-12-09
CC/750/2010
14 Lewis Street EPPING NSW 2121
2010-12-12 01:15:21
2010-12-09 00:00:00
Demolition, tree removal and construction of a 2 storey dual occupancy development with a 1
2010-12-09
CC/751/2010
19 Rhodes Avenue GUILDFORD NSW 2161
2010-12-12 01:15:21
2010-12-09 00:00:00
Alterations and additions to existing dwelling
2010-12-09
CD/308/2010
30 Taywood Avenue WINSTON HILLS NSW 2153
2010-12-12 01:15:21
2010-12-09 00:00:00

Statistics

Average successful run time: 2 minutes

Total run time: 12 days

Total cpu time used: about 1 hour

Total disk space used: 18.3 MB

History

  • Auto ran revision f4c0592b and failed .
    nothing changed in the database
  • Auto ran revision f4c0592b and completed successfully .
    67 records added, 62 records removed in the database
  • Auto ran revision f4c0592b and completed successfully .
    62 records added, 51 records removed in the database
  • Auto ran revision f4c0592b and completed successfully .
    62 records added, 62 records removed in the database
  • Auto ran revision f4c0592b and completed successfully .
    62 records added, 61 records removed in the database
  • ...
  • Forked from ScraperWiki

Show complete history

Scraper code

Ruby

parramatta / scraper.rb