planningalerts-scrapers / multiple_atdis

Scrapes multiple authorities that have ATDIS feed


This is a scraper that runs on Morph. To get started see the documentation

Contributors mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  The git source `git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git` uses the `git` protocol, which transmits data without encryption. Disable this warning with `bundle config git.allow_insecure true`, or switch to the `https` protocol to keep your data secure.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.16.2). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/.  Fetching git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git  Fetching concurrent-ruby 1.1.5  Fetching minitest 5.11.3  Fetching thread_safe 0.3.6  Installing thread_safe 0.3.6  Installing minitest 5.11.3  Fetching public_suffix 3.1.0  Fetching multi_json 1.13.1  Installing concurrent-ruby 1.1.5  Fetching unf_ext 0.0.7.6  Installing unf_ext 0.0.7.6 with native extensions  Installing multi_json 1.13.1  Installing public_suffix 3.1.0  Fetching mime-types-data 3.2019.0331  Fetching netrc 0.11.0  Installing mime-types-data 3.2019.0331  Installing netrc 0.11.0  Fetching rgeo 2.0.1  Installing rgeo 2.0.1 with native extensions  Fetching httpclient 2.8.3  Installing httpclient 2.8.3  Fetching sqlite3 1.4.1  Installing sqlite3 1.4.1 with native extensions  Using bundler 1.15.2  Fetching safe_yaml 1.0.5  Installing safe_yaml 1.0.5  Fetching hashdiff 0.4.0  Installing hashdiff 0.4.0  Fetching timecop 0.9.1  Installing timecop 0.9.1  Fetching vcr 5.0.0  Installing vcr 5.0.0  Fetching tzinfo 1.2.5  Installing tzinfo 1.2.5  Fetching i18n 1.6.0  Fetching addressable 2.6.0  Installing i18n 1.6.0  Installing addressable 2.6.0  Fetching mime-types 3.2.2  Installing mime-types 3.2.2  Fetching unf 0.1.4  Fetching crack 0.4.3  Installing unf 0.1.4  Fetching rgeo-geojson 2.1.1  Installing rgeo-geojson 2.1.1  Installing crack 0.4.3  Fetching activesupport 5.2.3  Installing activesupport 5.2.3  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching webmock 3.6.0  Installing webmock 3.6.0  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching activemodel 5.2.3  Fetching rest-client 2.0.2  Installing activemodel 5.2.3  Installing rest-client 2.0.2  Fetching atdis 0.4.1  Installing atdis 0.4.1  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Fetching scraperwiki 3.0.2  Installing scraperwiki 3.0.2  Fetching scraperwiki-morph 0.1.1  Installing scraperwiki-morph 0.1.1  Using atdisplanningalertsfeed 0.1.0 from git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git (at master@a7a7ce7)  Bundle complete! 4 Gemfile dependencies, 34 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Post-install message from i18n:  HEADS UP! i18n 1.1 changed fallbacks to exclude default locale.  But that may break your application.  Please check your Rails app for 'config.i18n.fallbacks = true'.  If you're using I18n (>= 1.1.0) and Rails (< 5.2.2), this should be  'config.i18n.fallbacks = [I18n.default_locale]'.  If not, fallbacks will be broken in your app by I18n 1.1.x.  For more info see:  https://github.com/svenfuchs/i18n/releases/tag/v1.1.0  Post-install message from hashdiff:  The HashDiff constant used by this gem conflicts with another gem of a similar name. As of version 1.0 the HashDiff constant will be completely removed and replaced by Hashdiff. For more information see https://github.com/liufengyun/hashdiff/issues/45.  Bundle completed (18.21s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... The HashDiff constant used by this gem conflicts with another gem of a similar name. As of version 1.0 the HashDiff constant will be completely removed and replaced by Hashdiff. For more information see https://github.com/liufengyun/hashdiff/issues/45. Collecting ATDIS feed data for bega_valley... D, [2020-01-23T03:37:48.978986 #15] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=3 D, [2020-01-23T03:38:00.461879 #15] DEBUG -- : Fetching Storing 010.2020.00000020.001 - 598 Yankees Gap RD, BEMBOKA, NSW 2550 Storing 010.2020.00000016.001 - 37 Culgoa CR, PAMBULA BEACH, NSW 2549 Storing 018.2020.00002006.001 - 96 Motbey RD, WANDELLA, NSW 2550 Storing 010.2020.00000011.001 - 5-7 Corunna ST, BERMAGUI, NSW 2546 Storing 010.2020.00000013.001 - 29 The Dress CIR, TURA BEACH, NSW 2548 Storing 010.2020.00000012.001 - Cattle Bay RD, EDEN, NSW 2551 Storing 018.2020.00002007.001 - 25 Blacket LA, BEGA, NSW 2550 Storing 010.2020.00000010.001 - 239 Back Creek RD, NETHERCOTE, NSW 2549 Storing 010.2020.00000009.001 - 14 Dolphin Cove DR, TURA BEACH, NSW 2548 Storing 010.2020.00000008.001 - 8 Robin CL, MERIMBULA, NSW 2548 Storing 018.2020.00002001.001 - 31 Merimbula DR, MERIMBULA, NSW 2548 Storing 010.2020.00000019.001 - 9 Golf RD, BERMAGUI, NSW 2546 Storing 010.2020.00000017.001 - Eagles Nest RD, BROGO, NSW 2550 Storing 010.2020.00000018.001 - Candelo Bega RD, CANDELO, NSW 2550 Storing 018.2020.00002003.001 - 1 Gillespie ST, COBARGO, NSW 2550 Storing 018.2020.00002004.001 - 3 Gillespie ST, COBARGO, NSW 2550 Storing 010.2020.00000015.001 - 114-116 Bega ST, TATHRA, NSW 2550 Storing 010.2020.00000014.001 - 12 Little Bega ST, BEGA, NSW 2550 Storing 018.2020.00002002.001 - 47 Old Wallagoot RD, KALARU, NSW 2550 Storing 010.2020.00000003.001 - 7 Blue Wren PL, BERMAGUI, NSW 2546 Storing 010.2020.00000007.001 - 193 Bournda Park WY, WALLAGOOT, NSW 2550 Storing 010.2019.00000217.002 - Lagoon ST, BEGA, NSW 2550 Storing 010.2020.00000005.001 - 740 Angledale RD, ANGLEDALE, NSW 2550 Storing 010.2020.00000006.001 - 17 Kurrajong CR, KALARU, NSW 2550 Storing 010.2020.00000004.001 - 92 O'Connells Point RD, WALLAGA LAKE, NSW 2546 Storing 010.2020.00000002.001 - 30 Florabunda PL, NETHERCOTE, NSW 2549 Storing 010.2020.00000001.001 - 3 Thompson DR, TATHRA, NSW 2550 Storing 010.2019.00000457.001 - Polacks Flat RD, BEMBOKA, NSW 2550 Collecting ATDIS feed data for ballina... D, [2020-01-23T03:38:23.075888 #15] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=3 D, [2020-01-23T03:38:32.979923 #15] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=4 D, [2020-01-23T03:38:42.958874 #15] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=5 D, [2020-01-23T03:38:51.657418 #15] DEBUG -- : Fetching Storing 010.2020.00000017.001 - 272 Sneaths RD, WOLLONGBAR, NSW 2477 Storing 010.2020.00000014.001 - 70 Scanlan LA, KINVARA, NSW 2478 Storing 010.2020.00000031.001 - 6 Cowrie St, LENNOX HEAD, NSW 2478 Storing 010.2020.00000013.001 - 20 Marine ST, BALLINA, NSW 2478 Storing 010.2020.00000012.001 - 48 Stoneyhurst DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000030.001 - 10 Bridge DR, WARDELL, NSW 2477 Storing 018.2020.00005002.001 - 2-30 Redford DR, SKENNARS HEAD, NSW 2478 Storing 010.2020.00000011.001 - 25 Towalbyn PL, URALBA, NSW 2477 Storing 010.2020.00000029.001 - 119 Montwood DR, LENNOX HEAD, NSW 2478 Storing 010.2019.00000416.002 - 211 Keith Hall LA, KEITH HALL, NSW 2478 Storing 010.2020.00000028.001 - 15 Bugden AVE, ALSTONVILLE, NSW 2477 Storing 010.2020.00000027.001 - 47 Stoneyhurst DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000026.001 - 153 Riverside DR, WEST BALLINA, NSW 2478 Storing 010.2020.00000016.001 - 15 Leach CR, WEST BALLINA, NSW 2478 Storing 018.2020.00005004.001 - 91 Hutley DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000025.001 - The Coast RD, SKENNARS HEAD, NSW 2478 Storing 010.2020.00000024.001 - 3 Selkirk PL, WOLLONGBAR, NSW 2477 Storing 010.2020.00000018.001 - 37 Bentinck ST, BALLINA, NSW 2478 Storing 010.2020.00000009.001 - 34 Burnet ST, BALLINA, NSW 2478 Storing 010.2016.00000464.004 - 2A Eclipse CR, CUMBALUM, NSW 2478 Storing 010.2020.00000023.001 - 55 Avalon AVE, WOLLONGBAR, NSW 2477 Storing 010.2020.00000022.001 - 32 Farrelly AVE, CUMBALUM, NSW 2478 Storing 010.2020.00000005.001 - 39 Pearces Creek Hall RD, PEARCES CREEK, NSW 2477 Storing 010.2020.00000003.001 - 27 Linderman ST, WEST BALLINA, NSW 2478 Storing 010.2020.00000002.001 - 7 Granada PDE, ALSTONVILLE, NSW 2477 Storing 010.2020.00000004.001 - 107-109 Fox ST, BALLINA, NSW 2478 Storing 010.2020.00000021.001 - 13 Scarlett CT, WOLLONGBAR, NSW 2477 Storing 010.2020.00000020.001 - 24 Suncrest AVE, ALSTONVILLE, NSW 2477 Storing 018.2019.00005109.001 - 8 Caloola RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000019.001 - 81 Bentinck ST, BALLINA, NSW 2478 Storing 010.2019.00000227.002 - 275 Hutley DR, SKENNARS HEAD, NSW 2478 Storing 010.2019.00000111.002 - 24 Mosman CH, LENNOX HEAD, NSW 2478 Storing 010.2020.00000010.001 - 1 Granada PDE, ALSTONVILLE, NSW 2477 Storing 010.2020.00000015.001 - 166 River ST, BALLINA, NSW 2478 Storing 010.2020.00000007.001 - 33 Horizon DR, WEST BALLINA, NSW 2478 Storing 010.2019.00000471.002 - 621 Pearces Creek RD, ALSTONVALE, NSW 2477 Storing 010.2020.00000006.001 - 1056 Tamarind DR, TINTENBAR, NSW 2478 Storing 010.2020.00000008.001 - 33 Millbrook TCE, WOLLONGBAR, NSW 2477 Storing 018.2019.00005106.001 - 8 Millbrook TCE, WOLLONGBAR, NSW 2477 Storing 010.2017.00000420.003 - 14 Suvla ST, EAST BALLINA, NSW 2478 Storing 018.2020.00005001.001 - 92 Tamar ST, BALLINA, NSW 2478 Storing 010.2017.00000509.002 - 25 Byron ST, LENNOX HEAD, NSW 2478 Storing 010.2020.00000001.001 - 29 Quays DR, WEST BALLINA, NSW 2478 Collecting ATDIS feed data for bathurst... D, [2020-01-23T03:39:04.656440 #15] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=3 D, [2020-01-23T03:39:08.086500 #15] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=4 D, [2020-01-23T03:39:09.821104 #15] DEBUG -- : Fetching Storing 018.2020.00000014.001 - 7 Wallace WAY, KELSO, NSW 2795 Storing 018.2020.00000013.001 - 15 Campbell CL, LLANARTH, NSW 2795 Storing 010.2020.00000016.001 - 45 James Barnet DR, KELSO, NSW 2795 Storing 010.2020.00000017.001 - 5 Darling ST, EGLINTON, NSW 2795 Storing 010.2020.00000018.001 - 357 College RD, ORTON PARK, NSW 2795 Storing 018.2020.00000015.001 - 76 Windemere RD, ROBIN HILL, NSW 2795 Storing 018.2020.00000012.001 - 10 Darvall DR, KELSO, NSW 2795 Storing 018.2020.00000011.001 - 21 Sunbright RD, KELSO, NSW 2795 Storing 010.2020.00000005.001 - 105 Keppel ST, BATHURST, NSW 2795 Storing 010.2020.00000015.001 - 4 Glazebrooks RD, THE LAGOON, NSW 2795 Storing 010.2020.00000014.001 - 669 White Rock RD, WHITE ROCK, NSW 2795 Storing 018.2020.00000009.001 - 28 Fraser DR, EGLINTON, NSW 2795 Storing 018.2020.00000010.001 - 6 Hyacinth WAY, LLANARTH, NSW 2795 Storing 010.2020.00000013.001 - 20 Vale RD, SOUTH BATHURST, NSW 2795 Storing 018.2020.00000008.001 - 60 Mendel DR, KELSO, NSW 2795 Storing 010.2020.00000012.001 - 73 White Rock RD, WHITE ROCK, NSW 2795 Storing 010.2020.00000011.001 - 3786 O'Connell RD, KELSO, NSW 2795 Storing 010.2020.00000007.001 - 8 Darvall DR, KELSO, NSW 2795 Storing 010.2020.00000008.001 - 201 Hartwood AVE, ROBIN HILL, NSW 2795 Storing 018.2020.00000007.001 - 11 Cain DR, KELSO, NSW 2795 Storing 018.2020.00000006.001 - 8 Hyacinth WAY, LLANARTH, NSW 2795 Storing 010.2020.00000009.001 - 42 Durham ST, BATHURST, NSW 2795 Storing 010.2020.00000010.001 - 93 William ST, BATHURST, NSW 2795 Storing 010.2020.00000001.001 - 250 Rocket ST, BATHURST, NSW 2795 Storing 018.2020.00000005.001 - 12 Meagher ST, LLANARTH, NSW 2795 Storing 010.2020.00000004.001 - 9 Glenhaven CR, PERTHVILLE, NSW 2795 Storing 010.2020.00000003.001 - 390 Panorama AVE, MITCHELL, NSW 2795 Storing 010.2020.00000002.001 - 40 Hampden Park RD, KELSO, NSW 2795 Storing 018.2020.00000004.001 - 6 Wallace WAY, KELSO, NSW 2795 Storing 018.2020.00000003.001 - 40 William ST, BATHURST, NSW 2795 Storing 018.2020.00000002.001 - 4173 O'Connell RD, WHITE ROCK, NSW 2795 Storing 018.2020.00000001.001 - 32 Governors PDE, WINDRADYNE, NSW 2795 Storing 018.2019.00000210.001 - 33 Newlands CR, KELSO, NSW 2795 Collecting ATDIS feed data for berrigan... Storing 105.20.DA.D5 - 135 MURRAY STREET, FINLEY, NSW 2713 Storing 104.20.DA.DM - MURRAY STREET, FINLEY, NSW 2714 Storing 102.20.DA.D9 - MURRAY STREET, FINLEY, NSW 3644 Storing 103.20.DA.DM - 16 MURRAY STREET, FINLEY, NSW 2714 Storing 101.20.DA.D1 - 25 MURRAY STREET, FINLEY, NSW 2713 Storing 100.20.DA.DM - 1243 LARKINS ROAD, FINLEY, NSW 2713 Storing 99.20.DA.DM - 16 HUGHES STREET, FINLEY, NSW 3644 Storing 98.20.DA.DM - MURRAY STREET, FINLEY, NSW 2714 Storing 97.20.DA.D5 - 79 MURRAY STREET, FINLEY, NSW 3644 Collecting ATDIS feed data for dubbo... D, [2020-01-23T03:39:21.716197 #15] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=3 D, [2020-01-23T03:39:25.732436 #15] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-01-23&lodgement_date_start=2019-12-24&page=4 D, [2020-01-23T03:39:29.212026 #15] DEBUG -- : Fetching Storing 010.2020.00000029.001 - 12L Yarrandale RD, DUBBO, NSW 2830 Storing 010.2020.00000030.001 - 1L Buddens RD, DUBBO, NSW 2830 Storing 010.2005.00000439.002 - 218R Obley RD, TOONGI, NSW 2830 Storing 010.2017.00000082.002 - 36 Sterling ST, DUBBO, NSW 2830 Storing 010.2020.00000027.001 - 66-68 Narragal ST, GEURIE, NSW 2818 Storing 010.2020.00000025.001 - 5 Macintyre CRCT, DUBBO, NSW 2830 Storing 010.2020.00000024.001 - 82 Argyle AVE, DUBBO, NSW 2830 Storing 010.2020.00000028.001 - 12 Hovea CT, DUBBO, NSW 2830 Storing 010.2020.00000021.001 - Dunedoo RD, DUBBO, NSW 2830 Storing 010.2020.00000031.001 - 17L Rifle Range RD, DUBBO, NSW 2830 Storing 010.2020.00000018.001 - 56 Windsor PDE, DUBBO, NSW 2830 Storing 010.2020.00000022.001 - 74 Twickenham DR, DUBBO, NSW 2830 Storing 010.2020.00000026.001 - 6 Mangrove CRES, DUBBO, NSW 2830 Storing 016.2020.00000002.001 - 66 Argyle AVE, DUBBO, NSW 2830 Storing 010.2020.00000023.001 - Argyle AVE, DUBBO, NSW 2830 Storing 010.2020.00000020.001 - 43 Gobolion ST, WELLINGTON, NSW 2820 Storing 010.2020.00000019.001 - 27 Taylor ST, DUBBO, NSW 2830 Storing 010.2020.00000012.001 - 91 Champagne DR, DUBBO, NSW 2830 Storing 016.2020.00000004.001 - 64 Argyle AVE, DUBBO, NSW 2830 Storing 016.2020.00000003.001 - 78 Argyle AVE, DUBBO, NSW 2830 Storing 010.2020.00000017.001 - 93 Birch AVE, DUBBO, NSW 2830 Storing 010.2020.00000006.001 - 80L Peak Hill RD, DUBBO, NSW 2830 Storing 010.2020.00000011.001 - 97R Old Mendooran RD, DUBBO, NSW 2830 Storing 010.2020.00000015.001 - 15 Snowy CR, DUBBO, NSW 2830 Storing 010.2020.00000016.001 - 16 Clews ST, DUBBO, NSW 2830 Storing 010.2020.00000014.001 - 212L Mogriguy RD, EUMUNGERIE, NSW 2822 Storing 010.2020.00000008.001 - 104 Talbragar ST, DUBBO, NSW 2830 Storing 010.2020.00000010.001 - Bradford CRCT, DUBBO, NSW 2830 Storing 010.2020.00000013.001 - Toongi RD, TOONGI, NSW 2830 Storing 016.2019.00000072.002 - 78 Champagne DR, DUBBO, NSW 2830 Storing 016.2019.00000075.002 - 86 Champagne DR, DUBBO, NSW 2830 Storing 010.2020.00000005.001 - Boundary RD, DUBBO, NSW 2830 Storing 010.2020.00000002.001 - 1 Aquarius CT, DUBBO, NSW 2830 Storing 010.2020.00000009.001 - 6 Gwydir CT, DUBBO, NSW 2830 Storing 010.2019.00000189.002 - 3 Naman ST, DUBBO, NSW 2830 Storing 010.2020.00000007.001 - 8 Waverly DR, DUBBO, NSW 2830 Storing 010.2020.00000003.001 - 93 Birch AVE, DUBBO, NSW 2830 Storing 010.2020.00000004.001 - 98 Macquarie ST, DUBBO, NSW 2830 Storing 010.2020.00000001.001 - 182 Cobra ST, DUBBO, NSW 2830 Collecting ATDIS feed data for kiama... kiama: ERROR: Timed out connecting to server Collecting ATDIS feed data for upper_hunter... /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:731:in `rescue in transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:647:in `transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient.rb:67:in `get' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/model.rb:76:in `read_url' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/feed.rb:71:in `applications' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:53:in `block in <main>' scraper.rb:47:in `each' scraper.rb:47:in `<main>' Storing 023.2020.00002755.001 - Pages River RD, MURRURUNDI, NSW 2338 Storing 023.2004.00002069.002 - 102 Dry Creek RD, DRY CREEK, NSW 2337 Storing 023.2020.00002754.001 - 240 Wallington RD, PARKVILLE, NSW 2337 Collecting ATDIS feed data for armidale... Storing 268129 - 5 Yellowtail Court, ARMIDALE, NSW 2350 Storing 268109 - 100 Niagara Street, ARMIDALE, NSW 2350 Collecting ATDIS feed data for ashfield... Storing 010.2020.00000006.001 - 28 Edward Street, SUMMER HILL, NSW 2130 Storing 010.2020.00000005.001 - 126 Holden Street, ASHFIELD, NSW 2131 Storing 010.2020.00000004.001 - 21 Smith Street, SUMMER HILL, NSW 2130 Storing 010.2019.00000084.002 - 79 Smith Street, SUMMER HILL, NSW 2130 Storing 010.2020.00000001.001 - 7 Dalhousie Street, HABERFIELD, NSW 2045 Storing 010.2020.00000205.001 - 91 Hawthorne Parade, HABERFIELD, NSW 2045 Collecting ATDIS feed data for leeton... leeton: ERROR: Timed out connecting to server Collecting ATDIS feed data for muswellbrook... /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:731:in `rescue in transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:647:in `transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient.rb:67:in `get' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/model.rb:76:in `read_url' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/feed.rb:71:in `applications' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:53:in `block in <main>' scraper.rb:47:in `each' scraper.rb:47:in `<main>' Storing 017.2020.00000003.001 - 1 Tocal WY, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000005.001 - 29 Sydney ST, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000004.001 - Palace ST, DENMAN, NSW 2328 Storing 005.2020.00000003.001 - Widden Valley RD, DENMAN, NSW 2328 Storing 005.2020.00000002.001 - 31 Weemala PL, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000001.001 - 18 Grimes CL, DENMAN, NSW 2328 Collecting ATDIS feed data for walgett... Collecting ATDIS feed data for newcastle... newcastle: ERROR: Timed out connecting to server /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:731:in `rescue in transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:647:in `transmit' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute' /app/vendor/bundle/ruby/2.3.0/gems/rest-client-2.0.2/lib/restclient.rb:67:in `get' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/model.rb:76:in `read_url' /app/vendor/bundle/ruby/2.3.0/gems/atdis-0.4.1/lib/atdis/feed.rb:71:in `applications' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.3.0/bundler/gems/atdisplanningalertsfeed-a7a7ce749691/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:53:in `block in <main>' scraper.rb:47:in `each' scraper.rb:47:in `<main>' scraper.rb:66:in `<main>': There were earlier errors. See output for details (RuntimeError)

Data

Downloaded 3720 times by openaustralia DevAppProject mattykn

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.62 MB) Use the API

rows 10 / 3298

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to authority_label
010.2019.00000164.001
119 Lochview Farm RD, LOCHIEL, NSW 2549
Installation of above ground swimming pool
2019-06-06
2019-05-07
bega_valley
010.2012.00000402.002
5 Bells PL, DOCTOR GEORGE MOUNTAIN, NSW 2550
Modification Erection of a dwelling & detached garage/studio & carport
2019-06-06
2019-05-07
2019-05-23
2019-06-06
bega_valley
018.2019.00005032.001
10 Willie Wagtail PL, ALSTONVILLE, NSW 2477
PCA CDC - New Dwelling - Single Residence
2019-06-06
2019-05-07
ballina
010.2019.00000283.001
2/188-202 Southern Cross DR, BALLINA, NSW 2478
Change of Use and Alterations to Commercial Building Change of use from a high technology industry to general industry involving the assembly, manufacture and fitting of prosthetic limbs and alterations to the building
2019-06-06
2019-05-07
ballina
010.2019.00000284.001
84 Meridian DR, COOLGARDIE, NSW 2478
New Dwelling - Single Residence Construction of a two storey dwelling house and associated studio as depicted in plans by Dean Poile Designs for 64 Meridian Drive Coolgardie dated 2 May 2019
2019-06-06
2019-05-07
ballina
010.2019.00000139.001
141 Keppel ST, BATHURST, NSW 2795
Stage 1 - Five lot torens title subdivision (four residential lots and one development lot) Stage 2 - Seven lot strata title subdivision
2019-06-06
2019-05-07
bathurst
018.2019.00000051.001
10 Newlands CR, KELSO, NSW 2795
Construction of a Single storey dwelling with attached garage
2019-06-06
2019-05-07
bathurst
010.2019.00000138.001
82 Graham DR, KELSO, NSW 2795
Dual Occupancy and two lot residential subdivision
2019-06-06
2019-05-07
bathurst
010.2019.00000078.001
21 Boyd ST, MINNAMURRA, NSW 2533
dwelling & air conditioning unit
2019-06-06
2019-05-07
kiama
010.2019.00000080.001
133 Headland DR, GERROA, NSW 2534
detached storage shed
2019-06-06
2019-05-07
2019-05-23
2019-06-06
kiama

Statistics

Average successful run time: 6 minutes

Total run time: 1 day

Total cpu time used: 10 minutes

Total disk space used: 4.66 MB

History

  • Auto ran revision 987eb14c and failed .
    169 records added, 163 records removed in the database
  • Auto ran revision 987eb14c and failed .
    188 records added, 163 records removed in the database
  • Auto ran revision 987eb14c and failed .
    163 records added, 139 records removed in the database
  • Auto ran revision 987eb14c and failed .
    139 records added, 125 records removed in the database
  • Auto ran revision 987eb14c and failed .
    153 records added, 153 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

multiple_atdis / scraper.rb