planningalerts-scrapers / multiple_horizon

Scrapes multiple authorities with the horizon system


This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

To run the tests

bundle exec rake

To run style and coding checks

bundle exec rubocop

What an Horizon/SolOrient website looks like

Screenshots

(To update this screenshot run bundle exec rake screenshots)

Contributors mlandauer dependabot[bot]

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Scraping authorities: cowra, liverpool_plains, uralla, walcha, weddin, maitland Collecting feed data for cowra... Collecting feed data for liverpool_plains... Collecting feed data for uralla... Collecting feed data for walcha... Collecting feed data for weddin... Collecting feed data for maitland...

Data

Downloaded 3364 times by openaustralia DevAppProject jesse-a-reilly dcalde mattykn vietbaracon

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.2 MB) Use the API

rows 10 / 3274

council_reference address description info_url date_scraped date_received authority_label
DA 2019/73.1
108 Darling Street COWRA 2794 NSW
New Building Residential Garage/Shed
2019-07-31
2019-07-30
cowra
CC 2019/42.1
1 Victor Street COWRA 2794 NSW
Dwelling additions
2019-07-31
2019-07-26
cowra
DA 2016/11.3
112E Airport Road COWRA 2794 NSW
New Building Assemble Bulding and New Building Factory, Workshop or La
2019-07-31
2019-07-25
cowra
DA 2019/72.1
Fishburn Street COWRA 2794 NSW
Subdivision
2019-07-31
2019-07-25
cowra
FS 2017/139.1
2 Mulyan Street COWRA 2794 NSW
Addition to existing car wash and 2 new shad structures
2019-07-31
2019-07-25
cowra
CC 2017/20.2
15 Parkes Street WOODSTOCK 2793 NSW
Modification - relocate caport
2019-07-31
2019-07-24
cowra
DA 2019/71.1
11 Waratah Street COWRA 2794 NSW
Extension to industrial shed
2019-07-31
2019-07-23
cowra
CC 2019/71.1
11 Waratah Street COWRA 2794 NSW
Extension to industrial shed
2019-07-31
2019-07-22
cowra
DA 2019/70.1
Carleton Street COWRA 2794 NSW
Demolition of an existing shed and construction of a new shed
2019-07-31
2019-07-22
cowra
CC 2019/70.1
Carleton Street COWRA 2794 NSW
Demolition of an existing shed and construction of a new shed
2019-07-31
2019-07-22
cowra

Statistics

Average successful run time: 2 minutes

Total run time: about 23 hours

Total cpu time used: 12 minutes

Total disk space used: 1.73 MB

History

  • Auto ran revision 60454ed8 and completed successfully .
    nothing changed in the database
  • Auto ran revision 60454ed8 and failed .
    234 records added, 233 records removed in the database
  • Auto ran revision 60454ed8 and failed .
    233 records added, 233 records removed in the database
  • Auto ran revision 60454ed8 and failed .
    225 records added, 212 records removed in the database
  • Auto ran revision 60454ed8 and failed .
    223 records added, 223 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

multiple_horizon / scraper.rb