planningalerts-scrapers / multiple_atdis

Scrapes multiple authorities that have ATDIS feed


This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

To run the tests

bundle exec rake

To run style and coding checks

bundle exec rubocop

Contributors mlandauer dependabot[bot]

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Collecting ATDIS feed data for bega_valley... D, [2020-10-01T14:01:24.970811 #16] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=3 D, [2020-10-01T14:01:35.923161 #16] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=4 D, [2020-10-01T14:01:46.785373 #16] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=5 D, [2020-10-01T14:01:57.519370 #16] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=6 D, [2020-10-01T14:02:08.053085 #16] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=7 D, [2020-10-01T14:02:18.705759 #16] DEBUG -- : Fetching http://datracker.begavalley.nsw.gov.au/ATDIS/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=8 W, [2020-10-01T14:02:19.123870 #16] WARN -- : No/invalid pagination, assuming no records/aborting Storing 010.2020.00000383.001 - 5076 Princes HWY, EDEN, NSW 2551 Storing 010.2020.00000352.001 - 39 Kerrisons LA, BEGA, NSW 2550 Storing 010.2020.00000392.001 - 117-119 Tura Beach DR, TURA BEACH, NSW 2548 Storing 010.2020.00000376.001 - 150 Auckland ST, BEGA, NSW 2550 Storing 010.2020.00000390.001 - 2 Burrawong PL, BARRAGGA BAY, NSW 2546 Storing 010.2020.00000363.001 - 264 Newtown RD, BEGA, NSW 2550 Storing 018.2020.00002089.001 - 2 Mumbulla ST, BERMAGUI, NSW 2546 Storing 010.2019.00000171.002 - 3 Oakleaf PL, MILLINGANDI, NSW 2549 Storing 010.2020.00000389.001 - 3 Jacaranda PL, MIRADOR, NSW 2548 Storing 010.2020.00000386.001 - 53 Parker ST, BEGA, NSW 2550 Storing 010.2020.00000387.001 - Racecourse RD, BERMAGUI, NSW 2546 Storing 010.2020.00000388.001 - 6 Gwainurra GR, PAMBULA BEACH, NSW 2549 Storing 010.2020.00000380.001 - 216-218 Imlay ST, EDEN, NSW 2551 Storing 010.2020.00000397.001 - 2 Nelson ST, BEMBOKA, NSW 2550 Storing 010.2020.00000396.001 - 60 Sutherlands RD, YOWRIE, NSW 2550 Storing 010.2020.00000401.001 - 67 Kameruka ST, BEMBOKA, NSW 2550 Storing 010.2020.00000385.001 - 51 Bega ST, TATHRA, NSW 2550 Storing 010.2020.00000379.001 - 26 Jacaranda PL, MIRADOR, NSW 2548 Storing 010.2020.00000400.001 - 14 Casey Jayne CT, TURA BEACH, NSW 2548 Storing 010.2020.00000382.001 - 10972 Princes HWY, COBARGO, NSW 2550 Storing 010.2020.00000373.001 - Mine LA, WOLUMLA, NSW 2550 Storing 010.2020.00000369.001 - 7 Cockatoo CT, MIRADOR, NSW 2548 Storing 010.2020.00000354.001 - 2-8 Bellbrook CR, WALLAGA LAKE, NSW 2546 Storing 010.2020.00000399.001 - 199 Ballantyne RD, NETHERCOTE, NSW 2549 Storing 010.2020.00000398.001 - 54 Wumbalwarra DR, BEGA, NSW 2550 Storing 018.2020.00002097.001 - 16 Tricarpa LA, COOLAGOLITE, NSW 2550 Storing 010.2020.00000395.001 - 31 Trumpeter AVE, EDEN, NSW 2551 Storing 010.2020.00000384.001 - 87 Jenny LA, NETHERCOTE, NSW 2549 Storing 010.2017.00000416.003 - 2 Snowy Mountains HWY, BEMBOKA, NSW 2550 Storing 010.2020.00000355.001 - 18 Main ST, MERIMBULA, NSW 2548 Storing 010.2020.00000394.001 - 164 Mirador DR, MIRADOR, NSW 2548 Storing 010.2020.00000381.001 - 573 Yankees Gap RD, BEMBOKA, NSW 2550 Storing 010.2020.00000378.001 - 2 Wrens RD, MURRAH, NSW 2546 Storing 010.2020.00000372.001 - 28 Cosham CL, EDEN, NSW 2551 Storing 010.2015.00000463.002 - 6 Melaleuca CT, TURA BEACH, NSW 2548 Storing 010.2020.00000366.001 - 11 Flinders ST, EDEN, NSW 2551 Storing 010.2020.00000391.001 - 9 Philip ST, WOLUMLA, NSW 2550 Storing 010.2020.00000358.001 - 341 Coolagolite RD, COOLAGOLITE, NSW 2550 Storing 010.2020.00000353.001 - 419 Upper Brogo RD, VERONA, NSW 2550 Storing 010.2020.00000370.001 - 11 Lookout LA, BEMBOKA, NSW 2550 Storing 010.2020.00000368.001 - 15 Woollybutt WY, TATHRA, NSW 2550 Storing 010.2020.00000356.001 - 334 Goughs RD, YOWRIE, NSW 2550 Storing 010.2020.00000364.001 - 74 South Tilba RD, BERMAGUI, NSW 2546 Storing 010.2005.00000888.003 - 129 Hergenhans RD, MURRAH, NSW 2546 Storing 010.2018.00000410.002 - 49 Omaru LA, LOCHIEL, NSW 2549 Storing 010.2020.00000377.001 - 997 Wandella RD, WANDELLA, NSW 2550 Storing 010.2020.00000367.001 - Mac Hill PL, BALD HILLS, NSW 2549 Storing 010.2020.00000365.001 - 3 Glen Mia DR, BEGA, NSW 2550 Storing 010.2020.00000359.001 - 23 Gillespies RD, YOWRIE, NSW 2550 Storing 010.2017.00000433.002 - 298 Goughs RD, YOWRIE, NSW 2550 Storing 010.2020.00000375.001 - 77 Terlichs RD, VERONA, NSW 2550 Storing 010.2020.00000374.001 - 624 Yowrie RD, YOWRIE, NSW 2550 Storing 010.2020.00000371.001 - 168 Murrabrine Forest RD, YOWRIE, NSW 2550 Storing 010.2020.00000360.001 - 781 New Buildings RD, WYNDHAM, NSW 2550 Storing 010.2016.00000290.002 - 18-24 Stanley ST, EDEN, NSW 2551 Storing 010.2020.00000362.001 - 3 Corkhill PL, BEGA, NSW 2550 Storing 010.2020.00000361.001 - 3 Avernus ST, COBARGO, NSW 2550 Storing 010.2018.00000132.002 - 195 Imlay ST, EDEN, NSW 2551 Storing 010.2020.00000357.001 - 184 Mirador DR, MIRADOR, NSW 2548 Storing 010.2020.00000175.002 - 14 Casey Jayne CT, TURA BEACH, NSW 2548 Collecting ATDIS feed data for ballina... D, [2020-10-01T14:02:40.954003 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=3 D, [2020-10-01T14:02:50.069496 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=4 D, [2020-10-01T14:02:57.686752 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=5 D, [2020-10-01T14:03:05.891096 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=6 D, [2020-10-01T14:03:13.334737 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=7 D, [2020-10-01T14:03:22.627616 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=8 D, [2020-10-01T14:03:30.507485 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=9 D, [2020-10-01T14:03:37.033686 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=10 D, [2020-10-01T14:03:43.589584 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=11 D, [2020-10-01T14:03:51.935233 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=12 D, [2020-10-01T14:03:57.301122 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=13 D, [2020-10-01T14:04:03.463731 #16] DEBUG -- : Fetching http://da.ballina.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=14 W, [2020-10-01T14:04:03.993403 #16] WARN -- : No/invalid pagination, assuming no records/aborting Storing 010.2020.00000624.001 - 84 Hinterland WY, KINVARA, NSW 2478 Storing 010.2020.00000608.001 - 10 Henderson PL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000602.001 - 186 Coolgardie RD, COOLGARDIE, NSW 2478 Storing 010.2020.00000593.001 - 1/4 Olivia CCT, LENNOX HEAD, NSW 2478 Storing 010.2020.00000589.001 - 11 Foley AVE, CUMBALUM, NSW 2478 Storing 010.2019.00000757.003 - 2 Triton St, LENNOX HEAD, NSW 2478 Storing 010.2020.00000588.001 - 2 Wejuba GDNS, BALLINA, NSW 2478 Storing 010.2020.00000587.001 - 19 Ridgeview CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000585.001 - 54 Allens PDE, LENNOX HEAD, NSW 2478 Storing 010.2020.00000572.001 - 10 Greenview PL, SKENNARS HEAD, NSW 2478 Storing 010.2020.00000554.001 - 33 Valley DR, ALSTONVILLE, NSW 2477 Storing 010.2020.00000637.001 - 35 Leach CR, WEST BALLINA, NSW 2478 Storing 010.2020.00000395.002 - 440 South Ballina Beach RD, SOUTH BALLINA, NSW 2478 Storing 010.2020.00000437.002 - 2 Seaswell CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000636.001 - 32 Elkhorn PDE, BALLINA, NSW 2478 Storing 010.2020.00000649.001 - 30 Kellie-Ann CR, LENNOX HEAD, NSW 2478 Storing 018.2020.00005071.001 - 217 Leadbeatters LA, ALSTONVILLE, NSW 2477 Storing 010.2019.00000391.002 - 1/37 Leach CR, WEST BALLINA, NSW 2478 Storing 010.2020.00000635.001 - 7 Suzanne ST, EAST BALLINA, NSW 2478 Storing 010.2020.00000648.001 - 137 Phoenix DR, TINTENBAR, NSW 2478 Storing 010.2020.00000628.001 - 1261 Teven RD, ALSTONVILLE, NSW 2477 Storing 010.2020.00000647.001 - 125 Links AVE, EAST BALLINA, NSW 2478 Storing 010.2020.00000634.001 - 5 Leach CR, WEST BALLINA, NSW 2478 Storing 010.2020.00000646.001 - 4 Ryan ST, CUMBALUM, NSW 2478 Storing 010.2020.00000633.001 - 56 Stoneyhurst DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000645.001 - 6B Brighton ST, EAST BALLINA, NSW 2478 Storing 018.2020.00005068.001 - 45 Fox Valley WY, LENNOX HEAD, NSW 2478 Storing 010.2020.00000631.001 - 12 Crane ST, BALLINA, NSW 2478 Storing 018.2020.00005069.001 - 67 Lindsay AVE, CUMBALUM, NSW 2478 Storing 010.2020.00000630.001 - 280 River DR, EAST WARDELL, NSW 2477 Storing 010.2020.00000629.001 - 8 Newport ST, EAST BALLINA, NSW 2478 Storing 010.2020.00000610.001 - 48 Cooke AVE, ALSTONVILLE, NSW 2477 Storing 010.2015.00000430.002 - 54 Survey ST, LENNOX HEAD, NSW 2478 Storing 010.2020.00000643.001 - 29 Dress Circle DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000605.001 - 52 Norton ST, BALLINA, NSW 2478 Storing 010.2020.00000356.002 - 19 Ridgeview CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000642.001 - 174 Lindendale RD, WOLLONGBAR, NSW 2477 Storing 010.2020.00000640.001 - 126 Bagotville RD, MEERSCHAUM VALE, NSW 2477 Storing 010.2020.00000644.001 - 32 Burnet ST, BALLINA, NSW 2478 Storing 010.2020.00000115.002 - 38 Cerreto CCT, WOLLONGBAR, NSW 2477 Storing 010.2020.00000559.001 - 12 Anderson ST, EAST BALLINA, NSW 2478 Storing 018.2020.00005070.001 - 1037 Tamarind DR, TINTENBAR, NSW 2478 Storing 010.2020.00000641.001 - 9 Houghlahans Creek RD, TEVEN, NSW 2478 Storing 010.2020.00000582.001 - 400 Old Byron Bay RD, NEWRYBAR, NSW 2479 Storing 010.2020.00000558.001 - 233 Hutley DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000626.001 - 46 Barrett DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000577.001 - 1 Monivae PL, SKENNARS HEAD, NSW 2478 Storing 010.2020.00000619.001 - 96 Martin ST, BALLINA, NSW 2478 Storing 010.2020.00000622.001 - 19 Ridgeview CR, LENNOX HEAD, NSW 2478 Storing 010.2019.00000598.002 - Hutley DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000639.001 - 30 Grant ST, BALLINA, NSW 2478 Storing 010.2020.00000617.001 - 25 Rous Mill RD, ROUS MILL, NSW 2477 Storing 010.2020.00000623.001 - 11 Sunray CR, LENNOX HEAD, NSW 2478 Storing 018.2020.00005067.001 - 3 Seamist PL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000620.001 - 8/10 Pinnacle ROW, LENNOX HEAD, NSW 2478 Storing 010.2020.00000615.001 - 24 Teven RD, ALSTONVILLE, NSW 2477 Storing 010.2017.00000273.002 - 56 Gibbon ST, LENNOX HEAD, NSW 2478 Storing 010.2003.00000248.002 - 719 Houghlahans Creek RD, PEARCES CREEK, NSW 2477 Storing 010.2020.00000616.001 - 1/193 Ballina RD, ALSTONVILLE, NSW 2477 Storing 010.2020.00000614.001 - 18 Dent CR, CUMBALUM, NSW 2478 Storing 010.2020.00000613.001 - 3 Selkirk PL, WOLLONGBAR, NSW 2477 Storing 010.2020.00000557.001 - 25 Cummings CR, CUMBALUM, NSW 2478 Storing 010.2020.00000568.001 - Midgen Flat RD, NEWRYBAR, NSW 2479 Storing 010.2020.00000611.001 - 20 Dent CR, CUMBALUM, NSW 2478 Storing 010.2020.00000324.002 - 56 Adele ST, ALSTONVILLE, NSW 2477 Storing 010.2020.00000335.002 - 125 Montwood DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000632.001 - Teven RD, WEST BALLINA, NSW 2478 Storing 010.2020.00000612.001 - 16 Bantry St, BALLINA, NSW 2478 Storing 010.2020.00000603.001 - 17 Sunray CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000565.001 - Whytes LA, PIMLICO, NSW 2478 Storing 010.2020.00000609.001 - 2 Henderson DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000586.001 - 16 Fern ST, LENNOX HEAD, NSW 2478 Storing 010.2020.00000604.001 - 111 Mellis CCT, ALSTONVILLE, NSW 2477 Storing 010.2020.00000627.001 - 36 Alston AVE, ALSTONVILLE, NSW 2477 Storing 010.2020.00000606.001 - 9 Pacific TCE, EAST BALLINA, NSW 2478 Storing 010.2020.00000625.001 - 127-129 River ST, BALLINA, NSW 2478 Storing 010.2020.00000562.001 - Montwood DR, LENNOX HEAD, NSW 2478 Storing 018.2020.00005062.001 - 133 Hutley DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000600.001 - 14 Bahama AVE, ALSTONVILLE, NSW 2477 Storing 010.2020.00000621.001 - 87 North Creek RD, LENNOX HEAD, NSW 2478 Storing 010.2020.00000618.001 - 12 Anchorage AVE, LENNOX HEAD, NSW 2478 Storing 010.2020.00000601.001 - 89 Fredericks LA, TINTENBAR, NSW 2478 Storing 010.2020.00000594.001 - 14 Riberry GR, WOLLONGBAR, NSW 2477 Storing 010.2020.00000575.001 - 91-93 River ST, BALLINA, NSW 2478 Storing 010.2020.00000607.001 - 30 Smith DR, WEST BALLINA, NSW 2478 Storing 010.2020.00000598.001 - 6 Coral ST, ALSTONVILLE, NSW 2477 Storing 010.2020.00000597.001 - 14 Clovelly PL, WOLLONGBAR, NSW 2477 Storing 010.2020.00000595.001 - 17 River ST, EAST WARDELL, NSW 2477 Storing 010.2020.00000592.001 - 31 Emigrant Creek LA, WEST BALLINA, NSW 2478 Storing 010.2020.00000583.001 - 38 Sunray CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000590.001 - 51A Blue Seas PDE, LENNOX HEAD, NSW 2478 Storing 010.2020.00000571.001 - 1/11 Tamar ST, BALLINA, NSW 2478 Storing 010.2020.00000581.001 - 62 Stoneyhurst DR, LENNOX HEAD, NSW 2478 Storing 010.2018.00000441.002 - 23 Condon DR, EAST BALLINA, NSW 2478 Storing 018.2020.00005066.001 - 3 Seamist PL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000584.001 - 11 Martins LA, KNOCKROW, NSW 2479 Storing 010.2020.00000573.001 - 44 Bolwarra CCT, WOLLONGBAR, NSW 2477 Storing 010.2020.00000566.001 - Ballina Heights DR, CUMBALUM, NSW 2478 Storing 018.2020.00005061.001 - 45 Jameson AVE, EAST BALLINA, NSW 2478 Storing 010.2020.00000254.002 - 154 Sneesbys LA, EAST WARDELL, NSW 2477 Storing 010.2020.00000570.001 - 33 Amber DR, LENNOX HEAD, NSW 2478 Storing 018.2020.00005064.001 - 31 Sandbreak CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000580.001 - 19 Rancher CT, WOLLONGBAR, NSW 2477 Storing 010.2020.00000599.001 - 17 Ibis PL, LENNOX HEAD, NSW 2478 Storing 010.2020.00000579.001 - 3 Sunway PL, BALLINA, NSW 2478 Storing 010.2020.00000596.001 - 14 Southern Cross DR, BALLINA, NSW 2478 Storing 010.2020.00000578.001 - 28 Ridgeview CR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000576.001 - 338 Fernleigh RD, FERNLEIGH, NSW 2479 Storing 010.2010.00000457.002 - 7 Clavan ST, BALLINA, NSW 2478 Storing 010.2020.00000569.001 - 6-7 Easton PL, EAST BALLINA, NSW 2478 Storing 010.2020.00000574.001 - 2 Shelly Beach RD, EAST BALLINA, NSW 2478 Storing 010.2018.00000725.002 - 166 Nashua RD, FERNLEIGH, NSW 2479 Storing 010.2020.00000591.001 - 25-39 Boeing AVE, BALLINA, NSW 2478 Storing 010.2020.00000563.001 - 3 Kardella AVE, EAST BALLINA, NSW 2478 Storing 010.2018.00000746.002 - 44 Bangalow RD, BALLINA, NSW 2478 Storing 010.2020.00000564.001 - 30 Amber DR, LENNOX HEAD, NSW 2478 Storing 010.2020.00000567.001 - 20-22 Summerhill CR, CUMBALUM, NSW 2478 Storing 010.2020.00000556.001 - 174 McLeans Ridges RD, MCLEANS RIDGES, NSW 2480 Storing 010.2020.00000555.001 - 48 Glenross DR, KINVARA, NSW 2478 Storing 010.2020.00000553.001 - 73 Plateau DR, WOLLONGBAR, NSW 2477 Storing 010.2020.00000561.001 - 49 Snapper DR, LENNOX HEAD, NSW 2478 Storing 018.2020.00005063.001 - 23 Lawrence RI, CUMBALUM, NSW 2478 Storing 010.2020.00000560.001 - 2 Page CT, LENNOX HEAD, NSW 2478 Collecting ATDIS feed data for bathurst... D, [2020-10-01T14:04:48.361883 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=3 D, [2020-10-01T14:05:05.020925 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=4 D, [2020-10-01T14:05:20.720290 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=5 D, [2020-10-01T14:05:35.146124 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=6 D, [2020-10-01T14:05:47.776693 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=7 D, [2020-10-01T14:05:58.817407 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=8 D, [2020-10-01T14:06:12.772640 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=9 D, [2020-10-01T14:06:28.088152 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=10 D, [2020-10-01T14:06:36.258067 #16] DEBUG -- : Fetching Storing 010.2020.00000331.001 - 4/2 Vale RD, SOUTH BATHURST, NSW 2795 Storing 018.2020.00000171.001 - 20 Connolly DR, KELSO, NSW 2795 Storing 018.2020.00000173.001 - 9 Vista PL, WHITE ROCK, NSW 2795 Storing 018.2020.00000169.001 - 12 Bolton ST, KELSO, NSW 2795 Storing 010.2020.00000349.001 - 16 Adrienne ST, RAGLAN, NSW 2795 Storing 010.2020.00000348.001 - 7 Keppel ST, BATHURST, NSW 2795 Storing 018.2020.00000170.001 - 117 Hughes ST, KELSO, NSW 2795 Storing 010.2020.00000352.001 - 9 Vista PL, WHITE ROCK, NSW 2795 Storing 010.2020.00000326.001 - 71 Stanley ST, BATHURST, NSW 2795 Storing 010.2020.00000305.001 - 103 Durham ST, BATHURST, NSW 2795 Storing 010.2020.00000337.001 - 47 Darwin DR, LLANARTH, NSW 2795 Storing 010.2010.00000847.003 - 15 Boundary RD, ROBIN HILL, NSW 2795 Storing 010.2020.00000310.001 - 12 Leo Grant DR, KELSO, NSW 2795 Storing 018.2020.00000145.001 - 17 Bolton ST, KELSO, NSW 2795 Storing 010.2020.00000334.001 - 49 Wellington ST, PEEL, NSW 2795 Storing 010.2020.00000338.001 - 13 Oates PL, EGLINTON, NSW 2795 Storing 010.2020.00000339.001 - 39 Wellington ST, EGLINTON, NSW 2795 Storing 010.2020.00000307.001 - 62 Ashworth DR, KELSO, NSW 2795 Storing 010.2020.00000345.001 - 22 Billywillinga RD, BILLYWILLINGA, NSW 2795 Storing 010.2020.00000346.001 - 29 Prince ST, PERTHVILLE, NSW 2795 Storing 010.2018.00000140.002 - Russell ST, BATHURST, NSW 2795 Storing 010.2020.00000341.001 - 228 Russell ST, BATHURST, NSW 2795 Storing 010.2020.00000347.001 - 225 Howick ST, BATHURST, NSW 2795 Storing 010.2017.00000020.002 - 25 Stockland DR, KELSO, NSW 2795 Storing 018.2020.00000168.001 - 16 Ignatius PL, KELSO, NSW 2795 Storing 018.2020.00000167.001 - 13 Ecrates PL, KELSO, NSW 2795 Storing 010.2020.00000320.001 - 176 Bentinck ST, BATHURST, NSW 2795 Storing 010.2020.00000340.001 - 308 Stewart ST, BATHURST, NSW 2795 Storing 010.2020.00000342.001 - 23 George Thomas CL, THE LAGOON, NSW 2795 Storing 010.2018.00000193.002 - 3 Clairvaux LA, KELSO, NSW 2795 Storing 010.2017.00000283.002 - 3249 O'Connell RD, BREWONGLE, NSW 2795 Storing 018.2020.00000166.001 - 2 Newlands CR, KELSO, NSW 2795 Storing 018.2020.00000162.001 - 64 Arabella PL, ROBIN HILL, NSW 2795 Storing 018.2020.00000164.001 - 9 Opal CT, KELSO, NSW 2795 Storing 018.2020.00000161.001 - 8 Matthews ST, WINDRADYNE, NSW 2795 Storing 018.2020.00000163.001 - 2 Robindale CT, ROBIN HILL, NSW 2795 Storing 018.2020.00000165.001 - 8 Tandora ST, KELSO, NSW 2795 Storing 010.2020.00000272.002 - 5 Delaware CR, ROBIN HILL, NSW 2795 Storing 010.2020.00000335.001 - 29 Durham ST, BATHURST, NSW 2795 Storing 018.2020.00000160.001 - 89 Graham DR, KELSO, NSW 2795 Storing 010.2020.00000327.001 - 16 Ophir ST, BATHURST, NSW 2795 Storing 010.2020.00000330.001 - 26A Gilmour ST, KELSO, NSW 2795 Storing 010.2020.00000336.001 - 7 Keppel ST, BATHURST, NSW 2795 Storing 010.2020.00000313.001 - 428 Conrod STGT, MOUNT PANORAMA, NSW 2795 Storing 018.2020.00000159.001 - 70 Eglinton RD, LLANARTH, NSW 2795 Storing 010.2020.00000328.001 - 8 Matthews ST, WINDRADYNE, NSW 2795 Storing 018.2020.00000158.001 - 34 Newlands CR, KELSO, NSW 2795 Storing 010.2020.00000308.001 - 45 Windemere RD, ROBIN HILL, NSW 2795 Storing 018.2020.00000157.001 - 31 Kurumben PL, WEST BATHURST, NSW 2795 Storing 010.2020.00000332.001 - 447 Limekilns RD, KELSO, NSW 2795 Storing 010.2020.00000329.001 - 15 Darling ST, EGLINTON, NSW 2795 Storing 010.2020.00000325.001 - 32 Beath ST, WATTLE FLAT, NSW 2795 Storing 010.2020.00000318.001 - 1 Rankin ST, BATHURST, NSW 2795 Storing 018.2020.00000147.001 - 29 Ignatius PL, KELSO, NSW 2795 Storing 010.2020.00000311.001 - 43 Lew AVE, EGLINTON, NSW 2795 Storing 010.2020.00000324.001 - 2529 Sofala RD, WIAGDON, NSW 2795 Storing 010.2020.00000321.001 - 821 Ophir RD, ROCK FOREST, NSW 2795 Storing 010.2020.00000323.001 - 43 Graham DR, KELSO, NSW 2795 Storing 010.2020.00000319.001 - 89 Graham DR, KELSO, NSW 2795 Storing 018.2020.00000154.001 - 70 Sunbright RD, KELSO, NSW 2795 Storing 018.2020.00000155.001 - 150 Stewart ST, EVANS PLAINS, NSW 2795 Storing 010.2019.00000387.003 - 11 Peard CL, EGLINTON, NSW 2795 Storing 010.2020.00000322.001 - 95 Lee ST, KELSO, NSW 2795 Storing 018.2020.00000152.001 - 18 Meagher ST, LLANARTH, NSW 2795 Storing 018.2020.00000153.001 - 308 Laffing Waters LA, LAFFING WATERS, NSW 2795 Storing 018.2020.00000150.001 - 50 Parer RD, ABERCROMBIE, NSW 2795 Storing 018.2020.00000149.001 - 14 Fairleigh PL, KELSO, NSW 2795 Storing 010.2020.00000317.001 - 250 Fitzgeralds Valley RD, FITZGERALDS VALLEY, NSW 2795 Storing 018.2020.00000151.001 - 21 Willow DR, KELSO, NSW 2795 Storing 010.2020.00000312.001 - 21 Willow DR, KELSO, NSW 2795 Storing 010.2020.00000316.001 - 68 Christie ST, RAGLAN, NSW 2795 Storing 010.2020.00000314.001 - 1336 Limekilns RD, CLEAR CREEK, NSW 2795 Storing 018.2020.00000148.001 - 46 Sunbright RD, KELSO, NSW 2795 Storing 010.2020.00000309.001 - 7 Nancarrow PL, KELSO, NSW 2795 Storing 018.2020.00000144.001 - 22 Bolton ST, KELSO, NSW 2795 Storing 018.2020.00000143.001 - 60 Mendel DR, KELSO, NSW 2795 Storing 010.2020.00000306.001 - 265 Durham ST, WEST BATHURST, NSW 2795 Collecting ATDIS feed data for berrigan... berrigan: ERROR: 500 Internal Server Error E, [2020-10-01T14:06:38.300003 #16] ERROR -- : 500 Internal Server Error /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/abstract_response.rb:249:in `exception_with_response' D, [2020-10-01T14:06:38.300172 #16] DEBUG -- : /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/abstract_response.rb:249:in `exception_with_response' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/abstract_response.rb:129:in `return!' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/abstract_response.rb:129:in `return!' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:836:in `process_result' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:836:in `process_result' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:743:in `block in transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:743:in `block in transmit' /app/vendor/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:910:in `start' /app/vendor/ruby-2.5.1/lib/ruby/2.5.0/net/http.rb:910:in `start' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:727:in `transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:727:in `transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:163:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:163:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:63:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:63:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/resource.rb:51:in `get' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/resource.rb:51:in `get' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:79:in `read_url_raw' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:79:in `read_url_raw' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:83:in `read_url' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:83:in `read_url' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/feed.rb:72:in `applications' /app/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/feed.rb:72:in `applications' /app/lib/atdisplanningalertsfeed.rb:19:in `fetch' /app/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' scraper.rb:57:in `block in <main>' scraper.rb:51:in `each' /app/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:51:in `<main>' scraper.rb:57:in `block in <main>' scraper.rb:51:in `each' scraper.rb:51:in `<main>' Collecting ATDIS feed data for dubbo... D, [2020-10-01T14:06:56.183767 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=3 D, [2020-10-01T14:07:00.272172 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=4 D, [2020-10-01T14:07:04.397842 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=5 D, [2020-10-01T14:07:08.456046 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=6 D, [2020-10-01T14:07:12.789616 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=7 D, [2020-10-01T14:07:16.893556 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=8 D, [2020-10-01T14:07:20.820865 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=9 D, [2020-10-01T14:07:24.589539 #16] DEBUG -- : Fetching https://planning.dubbo.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=10 D, [2020-10-01T14:07:27.240737 #16] DEBUG -- : Fetching Storing 010.2020.00000496.001 - 29L Obley RD, DUBBO, NSW 2830 Storing 010.2020.00000502.001 - Keswick PKWY, DUBBO, NSW 2830 Storing 010.2020.00000503.001 - 24 Old Homestead DR, DUBBO, NSW 2830 Storing 010.2020.00000505.001 - 2 Keswick PKWY, DUBBO, NSW 2830 Storing 010.2020.00000471.001 - 1R Richardson RD, DUBBO, NSW 2830 Storing 010.2020.00000504.001 - 11 Dalton ST, DUBBO, NSW 2830 Storing 010.2020.00000501.001 - 74R North Minore RD, DUBBO, NSW 2830 Storing 010.2020.00000500.001 - 46 Cypress Point DR, DUBBO, NSW 2830 Storing 010.2020.00000446.001 - 10 Homestead DR, DUBBO, NSW 2830 Storing 016.2020.00000011.003 - 141 Sheraton RD, DUBBO, NSW 2830 Storing 016.2020.00000049.001 - 24 Leonard ST, DUBBO, NSW 2830 Storing 016.2020.00000047.001 - 4 Sherrard CRES, DUBBO, NSW 2830 Storing 010.2020.00000493.001 - 99 Percy ST, WELLINGTON, NSW 2820 Storing 010.2020.00000498.001 - 10 Geurie ST, GEURIE, NSW 2818 Storing 010.2020.00000497.001 - 15 Wollomombi PDE, DUBBO, NSW 2830 Storing 010.2020.00000474.001 - 6 Narrabeen PL, DUBBO, NSW 2830 Storing 010.2020.00000464.001 - 12R Veechs RD, WONGARBON, NSW 2831 Storing 010.2020.00000439.001 - 9 Tallwoods GR, DUBBO, NSW 2830 Storing 016.2020.00000048.001 - 21 Peel PL, DUBBO, NSW 2830 Storing 016.2020.00000046.001 - 45 Azure AVE, DUBBO, NSW 2830 Storing 016.2020.00000040.002 - 43 Azure AVE, DUBBO, NSW 2830 Storing 010.2020.00000492.001 - 2 Capital DR, DUBBO, NSW 2830 Storing 010.2020.00000481.001 - 156 Brisbane ST, DUBBO, NSW 2830 Storing 010.2020.00000477.001 - 21 Buninyong RD, DUBBO, NSW 2830 Storing 010.2020.00000463.001 - 10 Bernborough PL, DUBBO, NSW 2830 Storing 010.2020.00000475.001 - 91 Champagne DR, DUBBO, NSW 2830 Storing 010.2020.00000469.001 - 5 Catherine DR, DUBBO, NSW 2830 Storing 010.2020.00000483.001 - 75 Jubilee ST, DUBBO, NSW 2830 Storing 016.2020.00000042.001 - 3 Frith ST, DUBBO, NSW 2830 Storing 010.2020.00000485.001 - 32 Melaleuca DR, MONTEFIORES, NSW 2820 Storing 016.2020.00000043.001 - 70 Argyle AVE, DUBBO, NSW 2830 Storing 016.2020.00000044.001 - 35 Bradford CRCT, DUBBO, NSW 2830 Storing 010.2020.00000459.001 - 16L Ambrose Lake RD, TERRAMUNGAMINE, NSW 2830 Storing 016.2020.00000045.001 - 35 Boyd AVE, DUBBO, NSW 2830 Storing 010.2020.00000461.001 - 29 Amadeus AVE, DUBBO, NSW 2830 Storing 010.2020.00000480.001 - 10 Apsley ST, WELLINGTON, NSW 2820 Storing 010.2020.00000445.001 - 22R Rifle Range RD, DUBBO, NSW 2830 Storing 010.2020.00000491.001 - 18 Severn CIR, DUBBO, NSW 2830 Storing 010.2020.00000490.001 - 43L Minore RD, DUBBO, NSW 2830 Storing 010.2020.00000489.001 - 28L Lonsdale RD, DUBBO, NSW 2830 Storing 010.2020.00000495.001 - 8 Campese CT, DUBBO, NSW 2830 Storing 010.2020.00000494.001 - 45 Sterling ST, DUBBO, NSW 2830 Storing 010.2020.00000455.001 - 8 Jubilee ST, DUBBO, NSW 2830 Storing 010.2020.00000486.001 - 128 Brisbane ST, DUBBO, NSW 2830 Storing 010.2020.00000442.001 - 26 Ron Gordon PL, DUBBO, NSW 2830 Storing 010.2020.00000470.001 - 17 Braun AVE, DUBBO, NSW 2830 Storing 010.2020.00000479.001 - 42 Umangla ST, WONGARBON, NSW 2831 Storing 010.2020.00000487.001 - 174 Darling ST, DUBBO, NSW 2830 Storing 010.2020.00000299.002 - 95 Champagne DR, DUBBO, NSW 2830 Storing 010.2020.00000450.001 - 120L Narromine RD, DUBBO, NSW 2830 Storing 016.2020.00000041.001 - 59 Stream AVE, DUBBO, NSW 2830 Storing 010.2020.00000453.001 - 11L Woodstock RD, DUBBO, NSW 2830 Storing 010.2020.00000433.001 - 1 Briarwood CL, DUBBO, NSW 2830 Storing 010.2020.00000484.001 - 17 Torryburn WY, DUBBO, NSW 2830 Storing 010.2020.00000478.001 - 9 Nagle DR, DUBBO, NSW 2830 Storing 010.2020.00000472.001 - 40 Bradford CRCT, DUBBO, NSW 2830 Storing 010.2020.00000482.001 - 68-70 Lime ST, GEURIE, NSW 2818 Storing 010.2020.00000466.001 - 27A Bradford CRCT, DUBBO, NSW 2830 Storing 010.2018.00000392.002 - 1 Burraway ST, BROCKLEHURST, NSW 2830 Storing 010.2020.00000476.001 - Narromine ST, MINORE, NSW 2830 Storing 010.2020.00000473.001 - 424 Wheelers LA, DUBBO, NSW 2830 Storing 010.2020.00000468.001 - 182 Wingewarra ST, DUBBO, NSW 2830 Storing 010.2020.00000456.001 - 35 Peel PL, DUBBO, NSW 2830 Storing 010.2020.00000462.001 - 105 Bourke ST, DUBBO, NSW 2830 Storing 010.2020.00000457.001 - 24 Depot RD, DUBBO, NSW 2830 Storing 010.2020.00000467.001 - 1R Dunlop RD, DUBBO, NSW 2830 Storing 010.2020.00000465.001 - 88 Champagne DR, DUBBO, NSW 2830 Storing 010.2019.00000264.002 - 19 Siren ST, DUBBO, NSW 2830 Storing 010.2020.00000367.001 - 409 Yarragal RD, YARRAGAL, NSW 2820 Storing 010.2020.00000460.001 - 22 Arthur Summons ST, DUBBO, NSW 2830 Storing 010.2018.00000637.003 - 20R Peak Hill RD, DUBBO, NSW 2830 Storing 010.2020.00000454.001 - 8 Abelia CT, DUBBO, NSW 2830 Storing 010.2020.00000458.001 - 7 Tallwoods GR, DUBBO, NSW 2830 Storing 010.2020.00000435.001 - 44 Azure AVE, DUBBO, NSW 2830 Storing 010.2020.00000443.001 - 93 Palmer ST, DUBBO, NSW 2830 Storing 010.2020.00000452.001 - 34 Peel PL, DUBBO, NSW 2830 Storing 010.2020.00000449.001 - 18 Victoria ST, DUBBO, NSW 2830 Storing 010.2020.00000448.001 - 14 Victoria ST, DUBBO, NSW 2830 Storing 010.2020.00000444.001 - 8 King ST, MONTEFIORES, NSW 2820 Storing 010.2020.00000451.001 - 9 Thomas Smith PL, DUBBO, NSW 2830 Storing 010.2017.00000460.002 - Tenandra ST, WONGARBON, NSW 2831 Storing 010.2020.00000438.001 - 42 Azure AVE, DUBBO, NSW 2830 Storing 010.2020.00000447.001 - 21L Angle Park RD, DUBBO, NSW 2830 Storing 010.2020.00000434.001 - 38 Azure AVE, DUBBO, NSW 2830 Storing 010.2020.00000436.001 - 63 Fitzroy ST, DUBBO, NSW 2830 Storing 010.2020.00000441.001 - 69 Whiteley ST, WELLINGTON, NSW 2820 Storing 016.2020.00000040.001 - 43 Azure AVE, DUBBO, NSW 2830 Storing 010.2020.00000440.001 - 128B Wingewarra ST, DUBBO, NSW 2830 Storing 010.2020.00000431.001 - 23A Bradford CRCT, DUBBO, NSW 2830 Storing 010.2020.00000437.001 - 133 Bultje ST, DUBBO, NSW 2830 Storing 010.2020.00000432.001 - 23B Bradford CRCT, DUBBO, NSW 2830 Collecting ATDIS feed data for kiama... D, [2020-10-01T14:07:47.947421 #16] DEBUG -- : Fetching https://da.kiama.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=3 D, [2020-10-01T14:07:53.970116 #16] DEBUG -- : Fetching Storing 010.2016.00000141.002 - 22 Cameron CR, KIAMA DOWNS, NSW 2533 Storing 010.2020.00000168.001 - 129 North Kiama DR, KIAMA DOWNS, NSW 2533 Storing 010.2019.00000217.002 - 6 Parkes ST, GERRINGONG, NSW 2534 Storing 010.2020.00000165.001 - 215 Princes HWY, KIAMA HEIGHTS, NSW 2533 Storing 010.2017.00000154.002 - Havilah PL, KIAMA, NSW 2533 Storing 010.2020.00000161.001 - 159 Princes HWY, KIAMA HEIGHTS, NSW 2533 Storing 010.2020.00000163.001 - 22 Charles AV, MINNAMURRA, NSW 2533 Storing 010.2020.00000160.001 - 40 Tingira CR, KIAMA, NSW 2533 Storing 010.2020.00000095.002 - 1/42 Collins ST, KIAMA, NSW 2533 Storing 010.2020.00000159.001 - 92 Merrick CCT, KIAMA, NSW 2533 Storing 010.2020.00000158.001 - 42 Kalang RD, KIAMA HEIGHTS, NSW 2533 Storing 010.2020.00000157.001 - 5 Cole ST, KIAMA, NSW 2533 Storing 010.2020.00000156.001 - 116 Headland DR, GERROA, NSW 2534 Storing 010.2020.00000151.001 - 9 Caliope ST, KIAMA, NSW 2533 Storing 010.2020.00000152.001 - 59 Princes HWY, KIAMA HEIGHTS, NSW 2533 Storing 010.2020.00000148.001 - 261 Jamberoo Mountain RD, JAMBEROO, NSW 2533 Storing 010.2020.00000145.001 - 14 Gwinganna AV, KIAMA, NSW 2533 Storing 010.2020.00000150.001 - 26 Eureka AV, KIAMA DOWNS, NSW 2533 Storing 010.2017.00000347.003 - 23 Farmer ST, KIAMA, NSW 2533 Storing 010.2020.00000155.001 - 8 Darien AV, BOMBO, NSW 2533 Storing 010.2020.00000153.001 - 19 Flinders AV, KIAMA DOWNS, NSW 2533 Storing 010.2020.00000149.001 - 16 Love ST, KIAMA, NSW 2533 Storing 010.2020.00000147.001 - 77 Merrick CCT, KIAMA, NSW 2533 Storing 010.2012.00000273.005 - Blackwood ST, GERRINGONG, NSW 2534 Storing 010.2020.00000143.001 - 1/16 Hothersal ST, KIAMA, NSW 2533 Storing 010.2020.00000146.001 - 47 Hollings DR, KIAMA DOWNS, NSW 2533 Storing 010.2020.00000144.001 - 28 Eureka AV, KIAMA DOWNS, NSW 2533 Storing 010.2020.00000141.001 - 26 Tombonda DR, KIAMA, NSW 2533 Storing 010.2020.00000142.001 - 25 Geering ST, GERRINGONG, NSW 2534 Collecting ATDIS feed data for upper_hunter... W, [2020-10-01T14:07:55.691103 #16] WARN -- : No/invalid pagination, assuming no records/aborting Collecting ATDIS feed data for armidale... Storing 277017 - 3 Karina Close, ARMIDALE, NSW 2350 Collecting ATDIS feed data for ashfield... Collecting ATDIS feed data for leeton... W, [2020-10-01T14:08:27.522844 #16] WARN -- : No/invalid pagination, assuming no records/aborting Collecting ATDIS feed data for muswellbrook... D, [2020-10-01T14:08:33.663308 #16] DEBUG -- : Fetching https://datracker.muswellbrook.nsw.gov.au/atdis/1.0/applications.json?lodgement_date_end=2020-10-01&lodgement_date_start=2020-09-01&page=3 W, [2020-10-01T14:08:34.350488 #16] WARN -- : No/invalid pagination, assuming no records/aborting Storing 005.2020.00000110.001 - Foley LA, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000109.001 - Denman RD, MUSWELLBROOK, NSW 2333 Storing 017.2020.00000020.001 - 72-78 Brook ST, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000108.001 - 5-7 Industrial CL, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000107.001 - 18 Wilson ST, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000106.001 - 16 Angus Peebles CL, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000102.001 - 10 Ogilvie ST, DENMAN, NSW 2328 Storing 005.2020.00000104.001 - Maitland ST, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000103.001 - 6 Honeyeater CL, DENMAN, NSW 2328 Storing 005.2020.00000101.001 - 2 Yarraman ROW, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000105.001 - 15 Edinglassie DR, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000099.001 - 59 Carl ST, MUSWELLBROOK, NSW 2333 Storing 005.2020.00000100.001 - 11 Silver Eye RD, MUSWELLBROOK, NSW 2333 Collecting ATDIS feed data for walgett... W, [2020-10-01T14:08:35.335747 #16] WARN -- : No/invalid pagination, assuming no records/aborting Collecting ATDIS feed data for newcastle... newcastle: ERROR: Timed out connecting to server /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:749:in `rescue in transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:651:in `transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:163:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:63:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/resource.rb:51:in `get' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:79:in `read_url_raw' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:83:in `read_url' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/feed.rb:72:in `applications' /app/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:57:in `block in <main>' scraper.rb:51:in `each' scraper.rb:51:in `<main>' scraper.rb:69:in `<main>': There were earlier errors. See output for details (RuntimeError)

Data

Downloaded 6955 times by openaustralia DevAppProject eduardomontero mattykn vietbaracon

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.97 MB) Use the API

rows 10 / 6200

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to authority_label
010.2019.00000164.001
119 Lochview Farm RD, LOCHIEL, NSW 2549
Installation of above ground swimming pool
2019-06-06
2019-05-07
bega_valley
010.2012.00000402.002
5 Bells PL, DOCTOR GEORGE MOUNTAIN, NSW 2550
Modification Erection of a dwelling & detached garage/studio & carport
2019-06-06
2019-05-07
2019-05-23
2019-06-06
bega_valley
018.2019.00005032.001
10 Willie Wagtail PL, ALSTONVILLE, NSW 2477
PCA CDC - New Dwelling - Single Residence
2019-06-06
2019-05-07
ballina
010.2019.00000283.001
2/188-202 Southern Cross DR, BALLINA, NSW 2478
Change of Use and Alterations to Commercial Building Change of use from a high technology industry to general industry involving the assembly, manufacture and fitting of prosthetic limbs and alterations to the building
2019-06-06
2019-05-07
ballina
010.2019.00000284.001
84 Meridian DR, COOLGARDIE, NSW 2478
New Dwelling - Single Residence Construction of a two storey dwelling house and associated studio as depicted in plans by Dean Poile Designs for 64 Meridian Drive Coolgardie dated 2 May 2019
2019-06-06
2019-05-07
ballina
010.2019.00000139.001
141 Keppel ST, BATHURST, NSW 2795
Stage 1 - Five lot torens title subdivision (four residential lots and one development lot) Stage 2 - Seven lot strata title subdivision
2019-06-06
2019-05-07
bathurst
018.2019.00000051.001
10 Newlands CR, KELSO, NSW 2795
Construction of a Single storey dwelling with attached garage
2019-06-06
2019-05-07
bathurst
010.2019.00000138.001
82 Graham DR, KELSO, NSW 2795
Dual Occupancy and two lot residential subdivision
2019-06-06
2019-05-07
bathurst
010.2019.00000078.001
21 Boyd ST, MINNAMURRA, NSW 2533
dwelling & air conditioning unit
2019-06-06
2019-05-07
kiama
010.2019.00000080.001
133 Headland DR, GERROA, NSW 2534
detached storage shed
2019-06-06
2019-05-07
2019-05-23
2019-06-06
kiama

Statistics

Average successful run time: 6 minutes

Total run time: 3 days

Total cpu time used: 21 minutes

Total disk space used: 5.01 MB

History

  • Auto ran revision 6f9ca88d and failed .
    394 records added, 368 records removed in the database
  • Auto ran revision 6f9ca88d and failed .
    380 records added, 370 records removed in the database
  • Auto ran revision 6f9ca88d and failed .
    371 records added, 339 records removed in the database
  • Auto ran revision 6f9ca88d and failed .
    339 records added, 331 records removed in the database
  • Auto ran revision 6f9ca88d and failed .
    342 records added, 342 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

multiple_atdis / scraper.rb