Wollongong Council Scraper

  • Server - ePathway
  • Cookie tracking - Yes
  • Pagnation - Yes
  • Javascript - No
  • Clearly defined data within a row - No

Enjoy

Contributors mlandauer otherchirps henare CloCkWeRX LoveMyData equivalentideas

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.4.1 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Fetching public_suffix 3.0.3  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching safe_yaml 1.0.5  Installing connection_pool 2.2.2  Installing safe_yaml 1.0.5  Installing public_suffix 3.0.3  Fetching unf_ext 0.0.7.6  Fetching hashdiff 0.3.9  Installing hashdiff 0.3.9  Fetching httpclient 2.8.3  Fetching mime-types-data 3.2019.0331  Installing mime-types-data 3.2019.0331  Installing httpclient 2.8.3  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching mini_portile2 2.4.0  Installing mini_portile2 2.4.0  Installing unf_ext 0.0.7.6 with native extensions  Fetching ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing webrobots 0.1.2  Installing ntlm-http 0.1.1  Fetching sqlite3 1.4.1  Fetching timecop 0.9.1  Installing timecop 0.9.1  Installing sqlite3 1.4.1 with native extensions  Fetching vcr 4.0.0  Installing vcr 4.0.0  Fetching net-http-persistent 3.0.1  Installing net-http-persistent 3.0.1  Fetching crack 0.4.3  Installing crack 0.4.3  Fetching addressable 2.6.0  Installing addressable 2.6.0  Fetching mime-types 3.2.2  Installing mime-types 3.2.2  Fetching nokogiri 1.10.3  Installing nokogiri 1.10.3 with native extensions  Fetching webmock 3.5.1  Installing webmock 3.5.1  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.6  Installing mechanize 2.7.6  Bundle complete! 5 Gemfile dependencies, 27 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (28.35s)  Cleaning up the bundler cache.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Saving record DA-2018/1441/A - 5 Mountain Road, AUSTINMER NSW 2515 Saving record DA-2019/444 - 30 Mountain Road, AUSTINMER NSW 2515 Saving record DA-2019/486 - 10 Napier Street, BALGOWNIE NSW 2519 Saving record DA-2019/393 - Bulli Raceway, Princes Highway, BULLI NSW 2516 Saving record DA-2019/409 - 22 Hospital Road, BULLI NSW 2516 Saving record DA-2019/439 - 51 Westmacott Parade, BULLI NSW 2516 Saving record DA-2018/666/B - 23 Somerville Street, BULLI NSW 2516 Saving record DA-2019/446 - 63 Buttenshaw Drive, COLEDALE NSW 2515 Saving record DA-2019/470 - 18 Young Street, COLEDALE NSW 2515 Saving record DA-2019/70/A - 5 Coxs Avenue, CORRIMAL NSW 2518 Saving record DA-2019/482 - 2/1 Midgley Street, CORRIMAL NSW 2518 Saving record DA-2018/1610 - 9 Bundarra Place, DAPTO NSW 2530 Saving record DA-2019/404 - 3 Brook Street, DAPTO NSW 2530 Saving record DA-2019/465 - Dapto Citizens Bowls Club, 102-106 Fowlers Road, DAPTO NSW 2530 Saving record DA-2019/467 - 2 Barellan Avenue, DAPTO NSW 2530 Saving record DA-2019/59 - 134 Cawley Street, EAST CORRIMAL NSW 2518 Saving record DA-2019/449 - Mt Ousley Pre School, 3 David Crescent, FAIRY MEADOW NSW 2519 Saving record DA-2019/416 - 105 Walang Avenue, FIGTREE NSW 2525 Saving record DA-2017/597/A - 3 Bellevue Road, FIGTREE NSW 2525 Saving record DA-2019/443 - 13 Sleigh Street, FIGTREE NSW 2525 Saving record DA-2019/474 - 2 Indigo Way, FIGTREE NSW 2525 Saving record DA-2019/475 - 7 Gadsden Way, FIGTREE NSW 2525 Saving record DA-2019/483 - 52 Valley Drive, FIGTREE NSW 2525 Saving record DA-2019/428 - 26 Foley Street, GWYNNEVILLE NSW 2500 Saving record DA-2019/386 - 47 The Crescent, HELENSBURGH NSW 2508 Saving record DA-2019/448 - 4 Gardiner Place, HELENSBURGH NSW 2508 Saving record DA-2019/485 - 14 Frew Avenue, HELENSBURGH NSW 2508 Saving record DA-2019/462 - 42 Homestead Drive, HORSLEY NSW 2530 Saving record DA-2019/464 - 13 Greenbrook Place, HORSLEY NSW 2530 Saving record DA-2019/473 - 142 Horsley Drive, HORSLEY NSW 2530 Saving record DA-2019/463 - 33 Elouera Crescent, KANAHOOKA NSW 2530 Saving record DA-2019/447 - 143 Gipps Road, KEIRAVILLE NSW 2500 Saving record DA-2019/469 - 55 Grey Street, KEIRAVILLE NSW 2500 Saving record DA-2018/42/A - 13 Ambleside Avenue, MOUNT KEIRA NSW 2500 Saving record DA-2019/450 - 3 Kirkwood Place, MOUNT KEMBLA NSW 2526 Saving record DA-2019/461 - 48 Ramah Avenue, MOUNT PLEASANT NSW 2519 Saving record DA-2019/451 - 22 Railway Crescent, NORTH WOLLONGONG NSW 2500 Saving record DA-2019/460 - 12/9-11 Bode Avenue, NORTH WOLLONGONG NSW 2500 Saving record DA-2019/396 - 8 Nimbin Street, RUSSELL VALE NSW 2517 Saving record DA-2019/426 - Lysaghts, Springhill Road, SPRING HILL NSW 2500 Saving record DA-2015/952/B - 2C Pass Avenue, THIRROUL NSW 2515 Saving record DA-2019/455 - 27 Fords Road, THIRROUL NSW 2515 Saving record DA-2019/471 - 27 The Waves, THIRROUL NSW 2515 Saving record DA-2019/419 - 5 Waverley Drive, UNANDERRA NSW 2526 Saving record DA-2019/472 - 20 Blackman Parade, UNANDERRA NSW 2526 Saving record DA-2019/480 - 7 Beatus Street, UNANDERRA NSW 2526 Saving record DA-2019/453 - 2-4 Highway Avenue, WEST WOLLONGONG NSW 2500 Saving record DA-2019/466 - 33 Cochrane Street, WEST WOLLONGONG NSW 2500 Saving record DA-2019/479 - 12 Immarna Avenue, WEST WOLLONGONG NSW 2500 Saving record DA-2003/1805/B - 4/63-65 Crown Street, WOLLONGONG NSW 2500 Saving record DA-2019/436 - 10 Market Place, WOLLONGONG NSW 2500 Saving record DA-2010/1000/C - 72-76 Crown Street, WOLLONGONG NSW 2500 Saving record DA-2019/445 - 2-4 Reef Avenue, WOMBARRA NSW 2515 Saving record DA-2019/442 - Old Wongawilli School, 39 Wongawilli Road, WONGAWILLI NSW 2530 Saving record DA-2019/456 - 109 Campbell Street, WOONONA NSW 2517

Data

Downloaded 2496 times by openaustralia LoveMyData DevAppProject WilliamYun jesse-a-reilly mattykn TheZepto henare MikeRalphson gnobes bigsmaug mkel4474 razz969 Jedwards157 derman232

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.06 MB) Use the API

rows 10 / 4990

date_received council_reference description address info_url comment_url date_scraped
2014-03-13
DA-2004/175/N
193 lot torrens title subdivision Modification N - to delete the creation of a small public reserve and consolidation
Lot 511 DP 1184600
2014-04-09
2014-03-20
DA-2014/303
Residential - alterations and additions
17 The Grove, AUSTINMER NSW 2515
2014-04-09
2014-03-26
DA-2014/327
Residential - demolition of existing structures and construction of dual occupancy (attached) and subdivision - Torrens title - two residential lots
77 Pioneer Road, BELLAMBI NSW 2518
2014-04-09
2010-08-18
MP-2010/147
Hi-Tech Holistic Cancer and Medical Hospital Facility and Ancilliary uses
Lot 4 Warwick Street, BERKELEY NSW 2506
2014-04-09
2013-11-13
DA-2013/1361
Refurbishment of community hall and construction of car parking and childrens playground
Community Park, Winnima Way, BERKELEY NSW 2506
2014-04-09
2014-03-25
DA-2014/329
Residential - demolition of carport, alterations and additions to existing dwelling including garage
98 Nottingham Street, BERKELEY NSW 2506
2014-04-09
2014-04-03
DA-2014/364
Residential - driveway and modify existing retaining walls.
24 Whimbrel Avenue, BERKELEY NSW 2506
2014-04-09
2014-03-03
DA-2014/211
Residential - dwelling to create dual occupancy and two lot Torrens title subdivision
190 Princes Highway, BULLI NSW 2516
2014-04-09
2014-03-24
DA-2012/523/A
Residential - secondary dwelling house Modification A - to change the footprint of the building, change verandah shape and window changes
8 Somerville Street, BULLI NSW 2516
2014-04-09
2014-04-01
DA-2014/345
Residential - dwelling house
21 Wilkies Street, BULLI NSW 2516
2014-04-09

Statistics

Average successful run time: 1 minute

Total run time: 29 days

Total cpu time used: about 1 hour

Total disk space used: 3.99 MB

History

  • Auto ran revision a5d4d3f7 and completed successfully .
    55 records added, 55 records removed in the database
  • Auto ran revision a5d4d3f7 and completed successfully .
    55 records added, 51 records removed in the database
  • Auto ran revision a5d4d3f7 and completed successfully .
    54 records added, 54 records removed in the database
  • Auto ran revision a5d4d3f7 and completed successfully .
    63 records added, 60 records removed in the database
  • Auto ran revision a5d4d3f7 and completed successfully .
    65 records added, 54 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

wollongong / scraper.rb