ianheggie-oaf / multiple_atdis

Scrapes multiple authorities that have ATDIS feed

README.md

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

To run the tests

bundle exec rake

To run style and coding checks

bundle exec rubocop

Contributors mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Collecting ATDIS feed data for armidale... Storing 337300 - 141 Taylor Street, ARMIDALE, NSW 2350 Storing 337298 - 124 Faulkner Street, ARMIDALE, NSW 2350 Storing 337296 - 3 Scouler Street, HILLGROVE, NSW 2350 Storing 337293 - 105-107 Sandon Street, GUYRA, NSW 2365 Storing 337289 - 248 Rockvale Road, ARMIDALE, NSW 2350 Collecting ATDIS feed data for bathurst... D, [2025-01-27T20:41:45.052447 #18] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2025-01-27&lodgement_date_start=2024-12-28&page=3 W, [2025-01-27T20:41:45.824177 #18] WARN -- : No/invalid pagination, assuming no records/aborting Collecting ATDIS feed data for kiama... kiama: ERROR: Timed out connecting to server /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:749:in `rescue in transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:651:in `transmit' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:163:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/request.rb:63:in `execute' /app/vendor/bundle/ruby/2.5.0/gems/rest-client-2.1.0/lib/restclient/resource.rb:51:in `get' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:79:in `read_url_raw' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/model.rb:83:in `read_url' /app/vendor/bundle/ruby/2.5.0/gems/atdis-0.5.0/lib/atdis/feed.rb:72:in `applications' /app/lib/atdisplanningalertsfeed.rb:45:in `fetch_all_pages' /app/lib/atdisplanningalertsfeed.rb:19:in `fetch' scraper.rb:27:in `block in <main>' scraper.rb:21:in `each' scraper.rb:21:in `<main>' scraper.rb:39:in `<main>': There were earlier errors. See output for details (RuntimeError)

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (6 KB) Use the API

rows 5 / 5

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to authority_label
337300
141 Taylor Street, ARMIDALE, NSW 2350
Shed
https://epathway.newengland.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/ExternalRequestBroker.aspx?Module=EGELAP&Type=TRACK&Class=EPA&ApplicationId=337300
2025-01-27
2025-01-12
2025-01-22
2025-02-05
armidale
337298
124 Faulkner Street, ARMIDALE, NSW 2350
Alterations and Additions to the Folk Museum
https://epathway.newengland.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/ExternalRequestBroker.aspx?Module=EGELAP&Type=TRACK&Class=EPA&ApplicationId=337298
2025-01-27
2025-01-10
armidale
337296
3 Scouler Street, HILLGROVE, NSW 2350
Alterations to Hillgrove Museum to provide a new bathroom and kitchenette and provide disability access to the building.
https://epathway.newengland.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/ExternalRequestBroker.aspx?Module=EGELAP&Type=TRACK&Class=EPA&ApplicationId=337296
2025-01-27
2025-01-10
2025-01-20
2025-02-03
armidale
337293
105-107 Sandon Street, GUYRA, NSW 2365
Shed
https://epathway.newengland.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/ExternalRequestBroker.aspx?Module=EGELAP&Type=TRACK&Class=EPA&ApplicationId=337293
2025-01-27
2025-01-08
2025-01-20
2025-02-03
armidale
337289
248 Rockvale Road, ARMIDALE, NSW 2350
Erection of a detached dual occupancy and new shed.
https://epathway.newengland.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/ExternalRequestBroker.aspx?Module=EGELAP&Type=TRACK&Class=EPA&ApplicationId=337289
2025-01-27
2024-12-31
2025-01-08
2025-01-22
armidale

Statistics

Total run time: 1 minute

Total cpu time used: less than 5 seconds

Total disk space used: 955 KB

History

  • Manually ran revision 5afa7a3a and failed .
    5 records added in the database
  • Created on morph.io

Scraper code

Ruby

multiple_atdis / scraper.rb