planningalerts-scrapers / spear

Streamlined Planning through Electronic Applications and Referrals Victoria

Contributors

Last run failed 2020-09-28 with status code 1.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Compiling Ruby [1G-----> Using Ruby version: ruby-2.5.1 [1G-----> Installing dependencies using bundler version 1.15.2 [1G Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment [1G Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. [1G Fetching gem metadata from https://rubygems.org/...... [1G Fetching version metadata from https://rubygems.org/. [1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git [1G Using bundler 1.15.2 [1G Fetching connection_pool 2.2.2 [1G Fetching unf_ext 0.0.7.6 [1G Fetching httpclient 2.8.3 [1G Installing connection_pool 2.2.2 [1G Installing unf_ext 0.0.7.6 with native extensions [1G Fetching mime-types-data 3.2019.1009 [1G Installing httpclient 2.8.3 [1G Installing mime-types-data 3.2019.1009 [1G Fetching net-http-digest_auth 1.4.1 [1G Installing net-http-digest_auth 1.4.1 [1G Fetching mini_portile2 2.4.0 [1G Fetching ntlm-http 0.1.1 [1G Installing ntlm-http 0.1.1 [1G Installing mini_portile2 2.4.0 [1G Fetching webrobots 0.1.2 [1G Installing webrobots 0.1.2 [1G Fetching sqlite3 1.4.2 [1G Fetching net-http-persistent 3.1.0 [1G Installing net-http-persistent 3.1.0 [1G Installing sqlite3 1.4.2 with native extensions [1G Fetching mime-types 3.3.1 [1G Installing mime-types 3.3.1 [1G Fetching nokogiri 1.10.7 [1G Installing nokogiri 1.10.7 with native extensions [1G Fetching unf 0.1.4 [1G Installing unf 0.1.4 [1G Fetching domain_name 0.5.20190701 [1G Installing domain_name 0.5.20190701 [1G Fetching http-cookie 1.0.3 [1G Installing http-cookie 1.0.3 [1G Fetching sqlite_magic 0.0.6 [1G Installing sqlite_magic 0.0.6 [1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176) [1G Fetching mechanize 2.7.6 [1G Installing mechanize 2.7.6 [1G Bundle complete! 2 Gemfile dependencies, 19 gems now installed. [1G Gems in the groups development and test were not installed. [1G Bundled gems are installed into ./vendor/bundle. [1G Bundle completed (25.34s) [1G Cleaning up the bundler cache. [1G Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. [1G-----> Detecting rake tasks [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... scraper.rb:100:in `<main>': undefined method `field_with' for nil:NilClass (NoMethodError)

Data

Downloaded 41623 times by

data

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (41.6 MB) Use the API

rows 10 / 99475

date_scraped	description	info_url	date_received	council_reference	address	comment_url
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S026214J&cacheApplicationListContext=true	2012-08-13	S026214J	33 DEACON AVENUE, BRIGHT VIC 3741	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17	2 lot resubdivision	https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S024425C&cacheApplicationListContext=true	2012-06-20	S024425C	15 DELANY AVENUE, BRIGHT VIC 3741	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S025191P&cacheApplicationListContext=true	2012-07-16	S025191P	MILL ROAD MOUNT BEAUTY 3699	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S010773J&cacheApplicationListContext=true	2010-12-02	S010773J	2/29A BAKERS GULLY ROAD, BRIGHT VIC 3741	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S018317C&cacheApplicationListContext=true	2011-11-07	S018317C	73 STATION STREET, POREPUNKAH VIC 3740	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S013074J&cacheApplicationListContext=true	2011-04-08	S013074J	51 DELANY AVENUE, BRIGHT VIC 3741	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S016718J&cacheApplicationListContext=true	2011-09-08	S016718J	7 THE TRACK, BRIGHT VIC 3741	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S023304S&cacheApplicationListContext=true	2012-05-14	S023304S	6 DIGGINGS ROAD, TAWONGA VIC 3697	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17		https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S021086C&cacheApplicationListContext=true	2012-02-24	S021086C	137 SCHOOL ROAD, WANDILIGONG VIC 3744	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17	Two lot resubdivision	https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S023932B&cacheApplicationListContext=true	2012-06-05	S023932B	8 TYNTYNDER LANE, BRIGHT VIC 3741	https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml

Statistics

Average successful run time: 14 minutes

Total run time: 22 days

Total cpu time used: about 6 hours

Total disk space used: 41.6 MB

History

Auto ran revision 5e3a3442 and failed 2020-09-28.

run time 23 minutes

nothing changed in the database
Auto ran revision 5e3a3442 and failed 2020-09-27.

run time 28 minutes

nothing changed in the database
Auto ran revision 5e3a3442 and failed 2020-09-26.

run time 58 s

nothing changed in the database
Auto ran revision 5e3a3442 and failed 2020-09-25.

run time 60 s

nothing changed in the database
Auto ran revision 5e3a3442 and completed successfully 2020-09-24.

run time about 1 hour

2682 records added, 2605 records removed in the database
...
Forked from ScraperWiki 2014-01-22

Show complete history

Scraper code

Ruby

spear / scraper.rb

git clone URL