planningalerts-scrapers / multiple_atdis

Scrapes multiple authorities that have ATDIS feed


This is a scraper that runs on Morph. To get started see the documentation

Contributors mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  The git source `git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git` uses the `git` protocol, which transmits data without encryption. Disable this warning with `bundle config git.allow_insecure true`, or switch to the `https` protocol to keep your data secure.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/.  Fetching git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git  Fetching concurrent-ruby 1.1.5  Fetching minitest 5.11.3  Fetching thread_safe 0.3.6  Installing minitest 5.11.3  Installing thread_safe 0.3.6  Installing concurrent-ruby 1.1.5  Fetching public_suffix 3.1.0  Installing public_suffix 3.1.0  Fetching multi_json 1.13.1  Installing multi_json 1.13.1  Fetching unf_ext 0.0.7.6  Fetching mime-types-data 3.2019.0331  Installing mime-types-data 3.2019.0331  Installing unf_ext 0.0.7.6 with native extensions  Fetching netrc 0.11.0  Installing netrc 0.11.0  Fetching rgeo 2.0.1  Fetching httpclient 2.8.3  Installing rgeo 2.0.1 with native extensions  Installing httpclient 2.8.3  Fetching sqlite3 1.4.1  Installing sqlite3 1.4.1 with native extensions  Using bundler 1.15.2  Fetching safe_yaml 1.0.5  Installing safe_yaml 1.0.5  Fetching hashdiff 0.4.0  Installing hashdiff 0.4.0  Fetching timecop 0.9.1  Installing timecop 0.9.1  Fetching vcr 5.0.0  Installing vcr 5.0.0  Fetching tzinfo 1.2.5  Installing tzinfo 1.2.5  Fetching addressable 2.6.0  Fetching i18n 1.6.0  Installing i18n 1.6.0  Fetching mime-types 3.2.2  Installing addressable 2.6.0  Installing mime-types 3.2.2  Fetching unf 0.1.4  Fetching crack 0.4.3  Installing unf 0.1.4  Fetching rgeo-geojson 2.1.1  Installing rgeo-geojson 2.1.1  Fetching activesupport 5.2.3  Installing crack 0.4.3  Installing activesupport 5.2.3  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching webmock 3.6.0  Installing webmock 3.6.0  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching activemodel 5.2.3  Installing activemodel 5.2.3  Fetching rest-client 2.0.2  Installing rest-client 2.0.2  Fetching atdis 0.4.1  Installing atdis 0.4.1  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Fetching scraperwiki 3.0.2  Installing scraperwiki 3.0.2  Fetching scraperwiki-morph 0.1.1  Installing scraperwiki-morph 0.1.1  Using atdisplanningalertsfeed 0.1.0 from git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git (at master@a7a7ce7)  Bundle complete! 4 Gemfile dependencies, 34 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from i18n:  HEADS UP! i18n 1.1 changed fallbacks to exclude default locale.  But that may break your application.  Please check your Rails app for 'config.i18n.fallbacks = true'.  If you're using I18n (>= 1.1.0) and Rails (< 5.2.2), this should be  'config.i18n.fallbacks = [I18n.default_locale]'.  If not, fallbacks will be broken in your app by I18n 1.1.x.  For more info see:  https://github.com/svenfuchs/i18n/releases/tag/v1.1.0  Post-install message from hashdiff:  The HashDiff constant used by this gem conflicts with another gem of a similar name. As of version 1.0 the HashDiff constant will be completely removed and replaced by Hashdiff. For more information see https://github.com/liufengyun/hashdiff/issues/45.  Bundle completed (19.18s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... The HashDiff constant used by this gem conflicts with another gem of a similar name. As of version 1.0 the HashDiff constant will be completely removed and replaced by Hashdiff. For more information see https://github.com/liufengyun/hashdiff/issues/45. Collecting ATDIS feed data for bega_valley... D, [2020-04-07T01:12:05.451716 #17] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=3 D, [2020-04-07T01:12:20.623970 #17] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=4 D, [2020-04-07T01:12:32.920175 #17] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=5 D, [2020-04-07T01:12:43.132940 #17] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=6 W, [2020-04-07T01:12:43.562999 #17] WARN -- : No/invalid pagination, assuming no records/aborting Storing 010.2020.00000097.001 - 43-45 Main ST, MERIMBULA, NSW 2548 Storing 010.2020.00000098.001 - 12-14 Yuppara ST, TATHRA, NSW 2550 Storing 010.2020.00000095.001 - 29 Weemilah DR, PAMBULA BEACH, NSW 2549 Storing 010.2020.00000096.001 - 17 Quondolo ST, PAMBULA, NSW 2549 Storing 010.2020.00000100.001 - 245 Black Range RD, BLACK RANGE, NSW 2550 Storing 010.2020.00000094.001 - 10 Monaro ST, MERIMBULA, NSW 2548 Storing 010.2020.00000099.001 - 15 Erle PL, SOUTH PAMBULA, NSW 2549 Storing 010.2020.00000072.001 - 146 Nutleys Creek RD, BERMAGUI, NSW 2546 Storing 010.2020.00000076.001 - 39 Carp ST, BEGA, NSW 2550 Storing 010.2020.00000090.001 - 226 Upper Kiah RD, KIAH, NSW 2551 Storing 010.2020.00000092.001 - 16 Edna DR, TATHRA, NSW 2550 Storing 010.2020.00000091.001 - 31 Gordon ST, QUAAMA, NSW 2550 Storing 010.2020.00000088.001 - 24 Bega ST, COBARGO, NSW 2550 Storing 010.2020.00000079.001 - 10 Kurrajong CR, KALARU, NSW 2550 Storing 010.2009.00000563.002 - 235 Wanatta LA, WOLUMLA, NSW 2550 Storing 010.2020.00000093.001 - 15 Northview CL, BLACK RANGE, NSW 2550 Storing 010.2019.00000203.002 - 200 Wallaga Lake RD, WALLAGA LAKE, NSW 2546 Storing 010.2020.00000082.001 - 39 McDonald RD, WANDELLA, NSW 2550 Storing 010.2020.00000081.001 - Edrom RD, EDROM, NSW 2551 Storing 010.2016.00000412.003 - 142-144 Imlay ST, EDEN, NSW 2551 Storing 010.2020.00000087.001 - 4 Lauren CT, TURA BEACH, NSW 2548 Storing 010.2020.00000064.001 - 17 Bournda CCT, TURA BEACH, NSW 2548 Storing 010.2020.00000070.001 - 511 Wallagoot LA, WALLAGOOT, NSW 2550 Storing 010.2020.00000086.001 - 54 Bournda CCT, TURA BEACH, NSW 2548 Storing 010.2020.00000084.001 - 5 Bega ST, COBARGO, NSW 2550 Storing 010.2020.00000085.001 - 33 Weemilah DR, PAMBULA BEACH, NSW 2549 Storing 010.2020.00000083.001 - 2 Landing RD, BROADWATER, NSW 2549 Storing 010.2020.00000078.001 - 1 Francis Hollis DR, TATHRA, NSW 2550 Storing 010.2020.00000080.001 - 298 Barrabooka RD, TANJA, NSW 2550 Storing 010.2020.00000077.001 - 50 Ocean View DR, BERMAGUI, NSW 2546 Storing 010.2020.00000075.001 - 1 Wattle ST, BERMAGUI, NSW 2546 Storing 010.2020.00000074.001 - 25 King ST, SOUTH PAMBULA, NSW 2549 Storing 010.2020.00000067.001 - 121 Boundary RD, BEGA, NSW 2550 Storing 010.2018.00000370.001 - 38 Parbery AVE, BERMAGUI, NSW 2546 Storing 010.2020.00000069.001 - 40 Boyd ST, EDEN, NSW 2551 Storing 010.2020.00000068.001 - 127 Rosedale LA, KANOONA, NSW 2550 Storing 010.2020.00000073.001 - 518 Yowrie RD, YOWRIE, NSW 2550 Storing 010.1988.00001490.002 - 11 Toallo ST, PAMBULA, NSW 2549 Storing 010.2020.00000066.001 - 35 Bay ST, TATHRA, NSW 2550 Storing 010.2020.00000065.001 - 6 Whale Cove CCT, EDEN, NSW 2551 Storing 010.2016.00000476.002 - 12 Quondolo ST, PAMBULA, NSW 2549 Collecting ATDIS feed data for ballina... D, [2020-04-07T01:13:23.569184 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=3 D, [2020-04-07T01:13:36.636418 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=4 D, [2020-04-07T01:13:50.995684 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=5 D, [2020-04-07T01:14:02.852524 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=6 D, [2020-04-07T01:14:19.062660 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=7 D, [2020-04-07T01:14:30.913430 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=8 D, [2020-04-07T01:14:42.085083 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=9 D, [2020-04-07T01:14:53.638924 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=10 D, [2020-04-07T01:15:08.400204 #17] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=11 D, [2020-04-07T01:15:18.998483 #17] DEBUG -- : Fetching Storing 010.2020.00000193.001 - 2 Spring Creek PL, WOLLONGBAR, NSW 2477 Storing 010.2020.00000206.001 - 14 Simpson AVE, WOLLONGBAR, NSW 2477 Storing 010.2019.00000023.002 - 3 Tipperary PL, BALLINA, NSW 2478 Storing 010.2016.00000242.002 - 25 Kellie-Ann CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000205.001 - 9 Parkview CIR, ALSTONVILLE, NSW 2477 Storing 010.2020.00000211.001 - 31 Outrigger RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000204.001 - 9 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000203.001 - 8 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000128.001 - 2/13 Alison AVE, LENNOX HEAD, NSW 2478 Storing 010.2019.00000502.003 - 2-12 Greenwood PL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000202.001 - 12 Latta AVE, BALLINA, NSW 2478 Storing 010.2019.00000796.002 - 224 Newrybar Swamp RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000201.001 - 16 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000200.001 - 12 Latta AVE, BALLINA, NSW 2478 Storing 018.2020.00005021.001 - 68 Pine AVE, EAST BALLINA, NSW 2478 Storing 010.2020.00000196.001 - 121 Montwood DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000210.001 - 17 Claire CCT, WEST BALLINA, NSW 2478 Storing 010.2020.00000207.001 - 13 Trelawney AVE, WOLLONGBAR, NSW 2477 Storing 010.2020.00000165.001 - 8 Fieldcrest PL, LENNOX HEAD, NSW 2478 Storing 010.2010.00000613.007 - 13 Alison AVE, LENNOX HEAD, NSW 2478 Storing 010.2020.00000198.001 - 11 Outrigger RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000197.001 - Humpty Back RD, MCLEANS RIDGES, NSW 2480 Storing 010.2020.00000137.001 - 39 Farrelly AVE, CUMBALUM, NSW 2478 Storing 010.2020.00000208.001 - 24 Barlows RD, WEST BALLINA, NSW 2478 Storing 010.2020.00000136.001 - 49 Bentwing PL, TINTENBAR, NSW 2478 Storing 010.2020.00000160.001 - Ascot RD, BALLINA, NSW 2478 Storing 010.2020.00000177.001 - Howards RD, ALSTONVILLE, NSW 2477 Storing 010.2020.00000191.001 - 953 River DR, KEITH HALL, NSW 2478 Storing 010.2020.00000189.001 - 1 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000172.001 - 62 Horizon DR, WEST BALLINA, NSW 2478 Storing 010.2020.00000167.001 - 40 De-Havilland CR, BALLINA, NSW 2478 Storing 010.2020.00000195.001 - 1/6 Pelican PL, EAST BALLINA, NSW 2478 Storing 010.2020.00000173.001 - 84 Hellyar DR, WOLLONGBAR, NSW 2477 Storing 018.2020.00005023.001 - 105 Tamar ST, BALLINA, NSW 2478 Storing 010.2020.00000188.001 - 59 Northumberland DR, EAST BALLINA, NSW 2478 Storing 010.2020.00000153.001 - 68 De-Havilland CR, BALLINA, NSW 2478 Storing 010.2020.00000170.001 - 445 Sneaths RD, ALSTONVALE, NSW 2477 Storing 010.2020.00000184.001 - 3 Dent CR, CUMBALUM, NSW 2478 Storing 010.2020.00000186.001 - 5 Hill ST, EAST BALLINA, NSW 2478 Storing 010.2020.00000187.001 - 29 Teven RD, ALSTONVILLE, NSW 2477 Storing 010.2020.00000185.001 - 24 Opal CR, ALSTONVILLE, NSW 2477 Storing 010.2020.00000178.001 - 69 Martin ST, BALLINA, NSW 2478 Storing 010.2009.00000133.004 - 46 Fenwick DR, EAST BALLINA, NSW 2478 Storing 010.2019.00000815.002 - 49 Farrelly AVE, CUMBALUM, NSW 2478 Storing 010.2020.00000199.001 - 6 Piper DR, BALLINA, NSW 2478 Storing 010.2020.00000183.001 - 873 Marom Creek RD, MAROM CREEK, NSW 2480 Storing 010.2020.00000180.001 - 60 Avalon AVE, WOLLONGBAR, NSW 2477 Storing 010.2020.00000182.001 - 23 Alison AVE, LENNOX HEAD, NSW 2478 Storing 018.2020.00005025.001 - Pacific HWY, WEST BALLINA, NSW 2478 Storing 010.2020.00000194.001 - 148 North Creek RD, BALLINA, NSW 2478 Storing 010.2020.00000192.001 - 550-578 River St, WEST BALLINA, NSW 2478 Storing 018.2020.00005026.001 - 11 Martins LA, KNOCKROW, NSW 2479 Storing 010.2020.00000179.001 - 4 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000181.001 - 20 Ashland ST, ALSTONVILLE, NSW 2477 Storing 010.2020.00000190.001 - 100 North Creek RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000176.001 - 54 Old Byron Bay RD, NEWRYBAR, NSW 2479 Storing 010.2020.00000174.001 - 21 Crane ST, BALLINA, NSW 2478 Storing 010.2020.00000175.001 - 30 Richmond ST, WARDELL, NSW 2477 Storing 010.2020.00000171.001 - 110 North Creek RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000159.001 - 43 Leach CR, WEST BALLINA, NSW 2478 Storing 010.2020.00000150.001 - 2 Camden ST, BALLINA, NSW 2478 Storing 010.2020.00000149.001 - 24 Canal RD, BALLINA, NSW 2478 Storing 010.2020.00000139.001 - 2 The Crest, LENNOX HEAD, NSW 2478 Storing 010.2018.00000511.002 - 31 Richmond ST, WARDELL, NSW 2477 Storing 010.2020.00000125.001 - 135-155 River ST, BALLINA, NSW 2478 Storing 010.2020.00000164.001 - 18 Oakwood DR, BALLINA, NSW 2478 Storing 010.2020.00000161.001 - 6 Latta AVE, BALLINA, NSW 2478 Storing 010.2020.00000163.001 - 1/131 Cherry ST, BALLINA, NSW 2478 Storing 010.2020.00000169.001 - 4 Skinner ST, BALLINA, NSW 2478 Storing 010.2020.00000168.001 - 56 Dolphin DR, WEST BALLINA, NSW 2478 Storing 010.2018.00000432.002 - 2 Old Pacific HWY, NEWRYBAR, NSW 2479 Storing 010.2020.00000166.001 - 430 Old Byron Bay RD, NEWRYBAR, NSW 2479 Storing 010.2020.00000158.001 - 108 Riverside DR, WEST BALLINA, NSW 2478 Storing 010.2020.00000162.001 - 15 Bentinck ST, BALLINA, NSW 2478 Storing 010.2017.00000531.003 - Fernleigh RD, BROOKLET, NSW 2479 Storing 010.2020.00000148.001 - 427 Friday Hut RD, BROOKLET, NSW 2479 Storing 010.2020.00000156.001 - 1/12 Rayner LA, LENNOX HEAD, NSW 2478 Storing 010.2020.00000147.001 - 4 Triton St, LENNOX HEAD, NSW 2478 Storing 010.2020.00000142.001 - 48 Tobin CL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000146.001 - 11 Linderman ST, WEST BALLINA, NSW 2478 Storing 010.2020.00000154.001 - 57 Pine AVE, EAST BALLINA, NSW 2478 Storing 010.2019.00000284.002 - 84 Meridian DR, COOLGARDIE, NSW 2478 Storing 010.2020.00000140.001 - 2/10 Tanamera DR, ALSTONVILLE, NSW 2477 Storing 010.2020.00000134.001 - 26 Fig Tree Hill DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000155.001 - 4 Willie Wagtail PL, ALSTONVILLE, NSW 2477 Storing 010.2020.00000135.001 - 127 Platypus DR, URALBA, NSW 2477 Storing 010.2020.00000138.001 - 96 Stoneyhurst DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000132.001 - 160 Moylans LA, EMPIRE VALE, NSW 2478 Storing 010.2020.00000152.001 - 4 Cowrie St, LENNOX HEAD, NSW 2478 Storing 010.2020.00000131.001 - 4 Lakeview CCT, EAST BALLINA, NSW 2478 Storing 010.2020.00000151.001 - 21 Leach CR, WEST BALLINA, NSW 2478 Storing 010.2020.00000141.001 - 5 Clovelly PL, WOLLONGBAR, NSW 2477 Storing 010.2019.00000439.002 - 31 Millbrook TCE, WOLLONGBAR, NSW 2477 Storing 010.2020.00000145.001 - 2 Bellevue AVE, EAST BALLINA, NSW 2478 Storing 010.2020.00000124.001 - 42 Silver Gull DR, EAST BALLINA, NSW 2478 Storing 010.2020.00000133.001 - 40 Sunray CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000130.001 - 26 Perkins CL, CUMBALUM, NSW 2478 Storing 010.2020.00000144.001 - 14 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000129.001 - 41 Snapper DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000143.001 - 19 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000157.001 - 4 Loedna PL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000126.001 - 48 Killarney CR, SKENNARS HEAD, NSW 2478 Storing 010.2020.00000127.001 - 10 Sunray CR, LENNOX HEAD, NSW 2478 Collecting ATDIS feed data for bathurst... D, [2020-04-07T01:16:16.214234 #17] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=3 D, [2020-04-07T01:16:28.052208 #17] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=4 D, [2020-04-07T01:16:40.695038 #17] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=5 D, [2020-04-07T01:16:54.737780 #17] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=6 D, [2020-04-07T01:17:04.448629 #17] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=7 W, [2020-04-07T01:17:05.967106 #17] WARN -- : No/invalid pagination, assuming no records/aborting Storing 010.2017.00000394.002 - 1557 Rockley RD, FOSTERS VALLEY, NSW 2795 Storing 010.2020.00000094.001 - 4 Austin PL, EGLINTON, NSW 2795 Storing 018.2020.00000049.001 - 67 Newlands CR, KELSO, NSW 2795 Storing 018.2020.00000043.001 - 22 Newlands CR, KELSO, NSW 2795 Storing 010.2020.00000088.001 - 102-104 Gilmour ST, KELSO, NSW 2795 Storing 010.2020.00000099.001 - 115 Samuel WAY, THE LAGOON, NSW 2795 Storing 010.2020.00000103.001 - 113 Durham ST, BATHURST, NSW 2795 Storing 010.2020.00000091.001 - 6 Peard CL, EGLINTON, NSW 2795 Storing 010.2020.00000081.001 - 20 Clairvaux LA, KELSO, NSW 2795 Storing 010.2020.00000101.001 - 92 Freemantle RD, EGLINTON, NSW 2795 Storing 010.2020.00000079.001 - 2 Links PL, ROBIN HILL, NSW 2795 Storing 010.2020.00000085.001 - 63 Corporation AVE, ROBIN HILL, NSW 2795 Storing 010.2020.00000077.001 - 38 Coates DR, KELSO, NSW 2795 Storing 010.2020.00000104.001 - 195 Russell ST, BATHURST, NSW 2795 Storing 010.2020.00000110.001 - 4040 O'Connell RD, KELSO, NSW 2795 Storing 010.2020.00000089.001 - 32 Country WAY, ABERCROMBIE, NSW 2795 Storing 010.2020.00000097.001 - 101 Peel ST, BATHURST, NSW 2795 Storing 010.2020.00000109.001 - 573 Eleven Mile DR, EGLINTON, NSW 2795 Storing 010.2020.00000106.001 - 114 Rankin ST, BATHURST, NSW 2795 Storing 018.2020.00000048.001 - 46 Graham DR, KELSO, NSW 2795 Storing 010.2020.00000107.001 - Tarana RD, BREWONGLE, NSW 2795 Storing 010.2020.00000098.001 - 219 Peregrine RD, BILLYWILLINGA, NSW 2795 Storing 018.2020.00000047.001 - 43 Sunbright RD, KELSO, NSW 2795 Storing 010.2020.00000100.001 - 16 Fitzroy ST, PEEL, NSW 2795 Storing 010.2020.00000087.001 - 214 Sydney RD, KELSO, NSW 2795 Storing 010.2020.00000102.001 - 56 Russell ST, BATHURST, NSW 2795 Storing 018.2020.00000036.001 - 23 Sunbright RD, KELSO, NSW 2795 Storing 018.2020.00000044.001 - 20 Bolton ST, KELSO, NSW 2795 Storing 018.2020.00000045.001 - 49 Hampden Park RD, KELSO, NSW 2795 Storing 010.2020.00000095.001 - 170 William ST, BATHURST, NSW 2795 Storing 010.2020.00000080.001 - 31 Blue Ridge DR, WHITE ROCK, NSW 2795 Storing 010.2020.00000096.001 - 1/20 Corporation AVE, ROBIN HILL, NSW 2795 Storing 010.2020.00000086.001 - 1360 Lagoon RD, THE LAGOON, NSW 2795 Storing 010.2020.00000093.001 - 58 Lee ST, KELSO, NSW 2795 Storing 010.2020.00000092.001 - 5350 Great Western HWY, RAGLAN, NSW 2795 Storing 018.2020.00000040.001 - 219 Turondale RD, DURAMANA, NSW 2795 Storing 010.2020.00000090.001 - 180 Stewart ST, BATHURST, NSW 2795 Storing 018.2020.00000042.001 - 32 Halfpenny DR, KELSO, NSW 2795 Storing 010.2018.00000405.002 - 118 Howick ST, BATHURST, NSW 2795 Storing 018.2020.00000039.001 - 93 Stanley ST, BATHURST, NSW 2795 Storing 010.2020.00000084.001 - 67 Lambert ST, BATHURST, NSW 2795 Storing 010.2020.00000082.001 - 168 Rankin ST, BATHURST, NSW 2795 Storing 010.2020.00000078.001 - 72 Hamilton ST, EGLINTON, NSW 2795 Collecting ATDIS feed data for berrigan... Storing 143.20.DA.D5-M - 11 TOWN BEACH ROAD, TOCUMWAL, NSW 2714 Storing 144.20.DA.D6 - 5 MURRAY STREET, FINLEY, NSW 2712 Storing 142.20.DA.DM - 31 WIRRAWAY COURT, TOCUMWAL, NSW 2714 Storing 140.20.DA.D1 - 19 GYPSIE CRESCENT, BAROOGA, NSW 3644 Storing 139.20.DA.D5 - 26 AMAROO AVENUE, BAROOGA, NSW 3644 Storing 138.20.DA.D5 - 20 MURRAY STREET, TOCUMWAL, NSW 2714 Storing 137.20.DA.DO - 3075 MULWALA-BAROOGA ROAD, MULWALA, NSW 3644 Storing 136.20.DA.D5 - 21 BRIDGET STREET, FINLEY, NSW 2713 Storing 134.20.DA.D5 - 52 MURRAY STREET, FINLEY, NSW 2713 Storing 135.20.DA.D5 - 1 COREE STREET, FINLEY, NSW 2713 Storing 132.20.DA.DM - 193 QUICKS ROAD, BAROOGA, NSW 2714 Storing 133.20.DA.DM - 35 WIRRAWAY COURT, TOCUMWAL, NSW 2714 Storing 131.20.DA.DM - M - PLATYPUS DRIVE, BAROOGA, NSW 3644 Storing 130.20.DA.D9 - 148 MCNAIRS ROAD, FINLEY, NSW 2713 Collecting ATDIS feed data for dubbo... D, [2020-04-07T01:17:19.424022 #17] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=3 D, [2020-04-07T01:17:24.196042 #17] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=4 D, [2020-04-07T01:17:28.521615 #17] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-04-07&lodgement_date_start=2020-03-08&page=5 D, [2020-04-07T01:17:32.907021 #17] DEBUG -- : Fetching Storing 010.2020.00000153.001 - 35R Buckhobble RD, DUBBO, NSW 2830 Storing 010.2020.00000152.001 - 144L Obley RD, DUBBO, NSW 2830 Storing 010.2020.00000151.001 - 140R Wellington RD, WONGARBON, NSW 2831 Storing 010.2019.00000560.002 - Sheraton RD, DUBBO, NSW 2830 Storing 010.2020.00000150.001 - 15 Sandringham PL, DUBBO, NSW 2830 Storing 016.2020.00000016.001 - 70-74 Church ST, DUBBO, NSW 2830 Storing 010.2020.00000149.001 - 61 Stream AVE, DUBBO, NSW 2830 Storing 010.2020.00000146.001 - 116 Yaruga ST, DUBBO, NSW 2830 Storing 010.2020.00000135.001 - 6 Peabody PL, DUBBO, NSW 2830 Storing 010.2020.00000148.001 - 14 Howard AVE, DUBBO, NSW 2830 Storing 010.2020.00000147.001 - 80 Argyle AVE, DUBBO, NSW 2830 Storing 010.2014.00000329.004 - Gilgandra RD, BROCKLEHURST, NSW 2830 Storing 010.2020.00000145.001 - 48 Tamworth ST, DUBBO, NSW 2830 Storing 010.2020.00000123.001 - 10 Kent PL, DUBBO, NSW 2830 Storing 010.2020.00000143.001 - 6R Numbardie DR, DUBBO, NSW 2830 Storing 010.2020.00000125.001 - 101 Gobolion ST, WELLINGTON, NSW 2820 Storing 010.2020.00000128.001 - Nardoo ST, BALLIMORE, NSW 2830 Storing 010.2020.00000121.001 - 15 Buninyong RD, DUBBO, NSW 2830 Storing 010.2020.00000142.001 - 15 Bomen ST, BALLIMORE, NSW 2830 Storing 010.2020.00000140.001 - 9 Charmere PL, DUBBO, NSW 2830 Storing 010.2020.00000139.001 - 9 Thomas Smith PL, DUBBO, NSW 2830 Storing 010.2020.00000144.001 - 8 Tolmer TCE, DUBBO, NSW 2830 Storing 010.2020.00000001.002 - 182 Cobra ST, DUBBO, NSW 2830 Storing 010.2020.00000138.001 - 28 Nancarrow ST, DUBBO, NSW 2830 Storing 010.2020.00000137.001 - 76 Wingewarra ST, DUBBO, NSW 2830 Storing 016.2020.00000014.001 - 7 Freshwater DR, DUBBO, NSW 2830 Storing 016.2020.00000015.001 - 8 Yarra PL, DUBBO, NSW 2830 Storing 010.2020.00000141.001 - 54 Magnolia BVD, DUBBO, NSW 2830 Storing 010.2019.00000419.002 - 24 Hawthorn ST, DUBBO, NSW 2830 Storing 010.2020.00000136.001 - 250B Darling ST, DUBBO, NSW 2830 Storing 010.2020.00000134.001 - 33R The Falls RD, DUBBO, NSW 2830 Storing 010.2019.00000613.002 - 156 Yaruga ST, DUBBO, NSW 2830 Storing 010.2020.00000133.001 - 45 High ST, DUBBO, NSW 2830 Storing 010.2020.00000130.001 - 19 Beni ST, DUBBO, NSW 2830 Storing 016.2020.00000013.001 - 177-191 Macquarie ST, DUBBO, NSW 2830 Storing 010.2020.00000118.001 - 4 Sunvale AVE, DUBBO, NSW 2830 Storing 010.2020.00000132.001 - 48 Roma RD, NORTH YEOVAL, NSW 2868 Storing 010.2020.00000131.001 - 5 Nagle DR, DUBBO, NSW 2830 Storing 010.2020.00000129.001 - 62 Bundemar ST, WONGARBON, NSW 2831 Storing 010.2020.00000127.001 - 2R Gilgandra RD, DUBBO, NSW 2830 Storing 010.2020.00000126.001 - 9 Andrew PL, DUBBO, NSW 2830 Storing 010.2020.00000124.001 - 31 Sterling ST, DUBBO, NSW 2830 Storing 010.2019.00000150.002 - 15 Wirraway CL, DUBBO, NSW 2830 Storing 010.2020.00000119.001 - 21 Bradford CRCT, DUBBO, NSW 2830 Storing 010.2020.00000120.001 - 19 Bradford CRCT, DUBBO, NSW 2830 Storing 010.2020.00000122.001 - 8 Richard Ryan PL, DUBBO, NSW 2830 Storing 010.2020.00000117.001 - 66-72 Windsor PDE, DUBBO, NSW 2830 Storing 010.2020.00000116.001 - 20L Sheraton RD, DUBBO, NSW 2830 Storing 010.2020.00000115.001 - 110 Bultje ST, DUBBO, NSW 2830 Collecting ATDIS feed data for kiama... D, [2020-04-07T01:17:48.505908 #17] DEBUG -- : Fetching Storing 010.2018.00000206.002 - 16 Drualla RD, JAMBEROO, NSW 2533 Storing 010.2017.00000168.002 - 30 Kaleula CR, KIAMA, NSW 2533 Storing 010.2020.00000043.001 - 4 Bland PL, GERROA, NSW 2534 Storing 010.2020.00000045.001 - 17 Flinders AV, KIAMA DOWNS, NSW 2533 Storing 010.2020.00000042.001 - 44A Anembo CR, KIAMA HEIGHTS, NSW 2533 Storing 010.2020.00000040.001 - 1 Newing CRCT, KIAMA DOWNS, NSW 2533 Storing 010.2020.00000041.001 - 69 Headland DR, GERROA, NSW 2534 Storing 010.2020.00000039.001 - 4 Collins LANE, KIAMA, NSW 2533 Storing 010.2017.00000215.003 - 8 Blackwood ST, GERRINGONG, NSW 2534 Storing 010.2020.00000038.001 - 9 Willawa AV, GERRINGONG, NSW 2534 Storing 010.2020.00000037.001 - 22 King ST, KIAMA, NSW 2533 Storing 010.2020.00000036.001 - 7 Cameron CR, KIAMA DOWNS, NSW 2533 Collecting ATDIS feed data for upper_hunter... Storing 023.2009.00001429.002 - 9843 Golden Highway (SRW 209), CASSILIS, NSW 2329 Storing 023.2020.00002760.001 - 135 Liverpool TCE, MURRURUNDI, NSW 2338 Storing 023.2020.00002759.001 - 35 Nandowra RD, MOOBI, NSW 2337 Storing 023.2020.00002761.001 - 53 Allan Cunningham RD, SCONE, NSW 2337 Storing 023.2010.00002142.002 - 1946 Flaggs RD, GUNGAL, NSW 2333 Collecting ATDIS feed data for armidale... Storing 270571 - 90-96 Marsh Street, ARMIDALE, NSW 2350 Collecting ATDIS feed data for ashfield... Collecting ATDIS feed data for leeton... leeton: ERROR: Timed out connecting to server /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:731:in `rescue in transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:647:in `transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient.rb:67:in `get' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/model.rb:76:in `read_url' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/feed.rb:71:in `applications' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:53:in `block in <main>' Collecting ATDIS feed data for muswellbrook... scraper.rb:47:in `each' scraper.rb:47:in `<main>' Storing 017.2020.00000009.001 - 20B Grey Gum RD, DENMAN, NSW 2328 Storing 005.2020.00000031.001 - 1040 Bureen RD, DENMAN, NSW 2328 Storing 005.2020.00000030.001 - 148 Sydney ST, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000029.001 - 20B Grey Gum RD, DENMAN, NSW 2328 Storing 005.2020.00000027.001 - 99 Woodland Ridge RD, MUSCLE CREEK, NSW 2333 Storing 005.2020.00000026.001 - 5 Honeyeater CL, DENMAN, NSW 2328 Storing 017.2020.00000007.001 - 4 Henry Dangar DR, MUSWELLBROOK, NSW 2333 Collecting ATDIS feed data for walgett... Collecting ATDIS feed data for newcastle... walgett: ERROR: Failed to open TCP connection to myhorizon.walgett.nsw.gov.au:80 (Connection refused - connect(2) for "myhorizon.walgett.nsw.gov.au" port 80) /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect' /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/net/http.rb:879:in `block in connect' /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout' /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/timeout.rb:101:in `timeout' /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/net/http.rb:878:in `connect' /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/net/http.rb:863:in `do_start' /app/vendor/ruby-2.3.4/lib/ruby/2.3.0/net/http.rb:852:in `start' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in `transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient.rb:67:in `get' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/model.rb:76:in `read_url' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/feed.rb:71:in `applications' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:53:in `block in <main>' scraper.rb:47:in `each' scraper.rb:47:in `<main>' newcastle: ERROR: Timed out connecting to server /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:731:in `rescue in transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:647:in `transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient.rb:67:in `get' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/model.rb:76:in `read_url' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/feed.rb:71:in `applications' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:53:in `block in <main>' scraper.rb:47:in `each' scraper.rb:47:in `<main>' scraper.rb:66:in `<main>': There were earlier errors. See output for details (RuntimeError)

Data

Downloaded 4697 times by openaustralia DevAppProject mattykn vietbaracon

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.97 MB) Use the API

rows 10 / 4061

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to authority_label
010.2019.00000164.001
119 Lochview Farm RD, LOCHIEL, NSW 2549
Installation of above ground swimming pool
2019-06-06
2019-05-07
bega_valley
010.2012.00000402.002
5 Bells PL, DOCTOR GEORGE MOUNTAIN, NSW 2550
Modification Erection of a dwelling & detached garage/studio & carport
2019-06-06
2019-05-07
2019-05-23
2019-06-06
bega_valley
018.2019.00005032.001
10 Willie Wagtail PL, ALSTONVILLE, NSW 2477
PCA CDC - New Dwelling - Single Residence
2019-06-06
2019-05-07
ballina
010.2019.00000283.001
2/188-202 Southern Cross DR, BALLINA, NSW 2478
Change of Use and Alterations to Commercial Building Change of use from a high technology industry to general industry involving the assembly, manufacture and fitting of prosthetic limbs and alterations to the building
2019-06-06
2019-05-07
ballina
010.2019.00000284.001
84 Meridian DR, COOLGARDIE, NSW 2478
New Dwelling - Single Residence Construction of a two storey dwelling house and associated studio as depicted in plans by Dean Poile Designs for 64 Meridian Drive Coolgardie dated 2 May 2019
2019-06-06
2019-05-07
ballina
010.2019.00000139.001
141 Keppel ST, BATHURST, NSW 2795
Stage 1 - Five lot torens title subdivision (four residential lots and one development lot) Stage 2 - Seven lot strata title subdivision
2019-06-06
2019-05-07
bathurst
018.2019.00000051.001
10 Newlands CR, KELSO, NSW 2795
Construction of a Single storey dwelling with attached garage
2019-06-06
2019-05-07
bathurst
010.2019.00000138.001
82 Graham DR, KELSO, NSW 2795
Dual Occupancy and two lot residential subdivision
2019-06-06
2019-05-07
bathurst
010.2019.00000078.001
21 Boyd ST, MINNAMURRA, NSW 2533
dwelling & air conditioning unit
2019-06-06
2019-05-07
kiama
010.2019.00000080.001
133 Headland DR, GERROA, NSW 2534
detached storage shed
2019-06-06
2019-05-07
2019-05-23
2019-06-06
kiama

Statistics

Average successful run time: 6 minutes

Total run time: 1 day

Total cpu time used: 13 minutes

Total disk space used: 5.01 MB

History

  • Auto ran revision 987eb14c and failed .
    275 records added, 272 records removed in the database
  • Auto ran revision 987eb14c and failed .
    273 records added, 259 records removed in the database
  • Auto ran revision 987eb14c and failed .
    276 records added, 272 records removed in the database
  • Auto ran revision 987eb14c and failed .
    287 records added, 287 records removed in the database
  • Auto ran revision 987eb14c and failed .
    304 records added, 289 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

multiple_atdis / scraper.rb