LoveMyData / liverpool_plains

README.md

Liverpool Plains Shire Council scraper

System - myhorizon from solorient
Cookie tracking - Yes
Clearly defined data within a row - Yes
XML - Yes

Setup MORPH_PERIOD for data recovery, available options are

thisweek (default)
thismonth
lastmonth
year (eg: 2017, minimum is 1998)

Enjoy

Contributors

Last run failed about 12 hours ago with status code 999.

Console output of last run

Data

Downloaded 1578 times by

data

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.5 MB) Use the API

rows 10 / 5652

id	council_reference	address	description	info_url	comment_url	date_scraped	date_received
id 1	council_reference DA 1998/50.1	address 144 South Street QUIRINDI 2343 NSW	description Residential dwelling	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-12-17
id 2	council_reference DA 1998/49.1	address 22 Underwood Street QUIRINDI 2343 NSW	description Dwelling extension	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-12-16
id 3	council_reference DA 1998/48.1	address 1681 Kamilaroi Highway QUIRINDI 2343 NSW	description Extension to Shed	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-25
id 4	council_reference DA 1998/47.1	address 120 Railway Avenue QUIRINDI 2343 NSW	description Dwelling extension	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-25
id 5	council_reference DA 1998/46.1	address 6 Pollocks Lane QUIRINDI 2343 NSW	description Dwelling extension	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-25
id 6	council_reference DA 1998/45.1	address 5 Mooki Road CAROONA 2343 NSW	description Steel Shed	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-23
id 7	council_reference DA 1998/44.1	address 60 Hill Street QUIRINDI 2343 NSW	description Carport	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-17
id 8	council_reference DA 1998/43.1	address 132 Loder Street QUIRINDI 2343 NSW	description Dwelling	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-17
id 9	council_reference DA 1998/42.1	address 461 Norvills Road WALLABADAH 2343 NSW	description Telecommunication facility	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-10
id 10	council_reference DA 1998/41.1	address 9538 New England Highway WALLABADAH 2343 NSW	description Telecommunications facility	info_url http://myhorizon.solorient.com.au/Horizon/logonGuest.aw?domain=horizondap_lpsc	comment_url mailto:lpsc@lpsc.nsw.gov.au	date_scraped 2017-10-19	date_received 1998-11-10

Statistics

Average successful run time: 1 minute

Total run time: 1 day

Total cpu time used: 27 minutes

Total disk space used: 1.54 MB

History

Auto ran and failed about 12 hours ago.

run time 0 s
Auto ran and failed a day ago.

run time 0 s
Auto ran and failed 3 days ago.

run time 0 s
Auto ran and failed 3 days ago.

run time 0 s
Auto ran and failed 4 days ago.

run time 0 s
...
Created on morph.io 7 years ago

Show complete history

Scraper code

Ruby

liverpool_plains / scraper.rb

git clone URL