planningalerts-scrapers / spear

Streamlined Planning through Electronic Applications and Referrals Victoria

Forked from ScraperWiki

Contributors CloCkWeRX henare mlandauer archoo

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.5.1 -----> Installing dependencies using bundler version 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.............  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching unf_ext 0.0.7.6  Fetching httpclient 2.8.3  Installing connection_pool 2.2.2  Fetching mime-types-data 3.2019.1009  Installing unf_ext 0.0.7.6 with native extensions  Installing httpclient 2.8.3  Installing mime-types-data 3.2019.1009  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching mini_portile2 2.4.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Installing mini_portile2 2.4.0  Fetching webrobots 0.1.2  Installing webrobots 0.1.2  Fetching sqlite3 1.4.2  Fetching net-http-persistent 3.1.0  Installing sqlite3 1.4.2 with native extensions  Installing net-http-persistent 3.1.0  Fetching mime-types 3.3.1  Installing mime-types 3.3.1  Fetching nokogiri 1.10.7  Installing nokogiri 1.10.7 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20190701  Installing domain_name 0.5.20190701  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.6  Installing mechanize 2.7.6  Bundle complete! 2 Gemfile dependencies, 19 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (28.19s)  Cleaning up the bundler cache.  Warning: the running version of Bundler (1.15.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Scraping Alpine Shire Council - Bright Office... Scraping Ararat Rural City Council... Scraping Ballarat City Council... Scraping Banyule City Council... Scraping Bass Coast Shire Council... Scraping Baw Baw Shire Council... Scraping Bayside City Council... Scraping Benalla Rural City Council... Scraping Boroondara City Council... Scraping Borough of Queenscliffe... Scraping Brimbank City Council... Scraping Buloke Shire Council... Scraping Cardinia Shire Council... Scraping Casey City Council... Scraping Central Goldfields Shire Council... Scraping City of Greater Dandenong... Scraping City of Greater Geelong... Scraping City of Wodonga... Scraping Colac Otway Shire... Scraping Corangamite Shire Council... Scraping Darebin City Council... Scraping East Gippsland Shire Council... Scraping Frankston City Council... Scraping Gannawarra Shire Council... Scraping Glen Eira City Council... Scraping Glenelg Shire Council... Scraping Golden Plains Shire Council... Scraping Greater Bendigo City Council... Scraping Greater Shepparton City Council... Scraping Hepburn Shire Council... Scraping Hindmarsh Shire Council... Scraping Hobsons Bay City Council... Scraping Horsham Rural City Council... Scraping Hume City Council... Scraping Indigo Shire Council... Scraping Kingston City Council... Scraping Knox City Council... Scraping Latrobe City Council... Scraping Loddon Shire Council... Scraping Macedon Ranges Shire Council... Scraping Manningham City Council... Scraping Mansfield Shire Council... Scraping Maribyrnong City Council... Scraping Maroondah City Council... Scraping Melbourne City Council... Scraping Melton City Council... Scraping Mildura Rural City Council... Scraping Minister for Planning - Others... Scraping Minister for Planning - Social Housing... Scraping Mitchell Shire Council... Scraping Moira Shire Council... Scraping Monash City Council... Scraping Moonee Valley City Council... Scraping Moorabool Shire Council... Scraping Moreland City Council... Scraping Mornington Peninsula Shire Council... Scraping Mount Alexander Shire Council... Scraping Moyne Shire Council... Scraping Murrindindi Shire Council... Scraping Nillumbik Shire Council... Scraping Northern Grampians Shire Council... Scraping Port Phillip City Council... Scraping Pyrenees Shire Council... Scraping Rural City of Wangaratta... Scraping Shire of Campaspe... Scraping South Gippsland Shire Council... Scraping Southern Grampians Shire Council... Scraping SPEAR Test Council... Scraping Stonnington City Council... Scraping Strathbogie Shire Council... Scraping Surf Coast Shire Council... Scraping Swan Hill Rural City Council... Scraping Towong Shire Council... Scraping Warrnambool City Council... Scraping Wellington Shire Council... Scraping West Wimmera Shire Council... Scraping Whitehorse City Council... Scraping Whittlesea City Council... Scraping Wyndham City Council... Scraping Yarra City Council... Scraping Yarra Ranges Shire Council... Scraping Yarriambiack Shire Council...

Data

Downloaded 22810 times by slav123 openaustralia LoveMyData DevAppProject jesse-a-reilly GitHubMLee TheZepto srsmonty cankei TornMarketing Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (38.8 MB) Use the API

rows 10 / 93822

date_scraped description info_url date_received council_reference address comment_url
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S026214J&cacheApplicationListContext=true
2012-08-13
S026214J
33 DEACON AVENUE, BRIGHT VIC 3741
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
2 lot resubdivision
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S024425C&cacheApplicationListContext=true
2012-06-20
S024425C
15 DELANY AVENUE, BRIGHT VIC 3741
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S025191P&cacheApplicationListContext=true
2012-07-16
S025191P
MILL ROAD MOUNT BEAUTY 3699
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S010773J&cacheApplicationListContext=true
2010-12-02
S010773J
2/29A BAKERS GULLY ROAD, BRIGHT VIC 3741
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S018317C&cacheApplicationListContext=true
2011-11-07
S018317C
73 STATION STREET, POREPUNKAH VIC 3740
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S013074J&cacheApplicationListContext=true
2011-04-08
S013074J
51 DELANY AVENUE, BRIGHT VIC 3741
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S016718J&cacheApplicationListContext=true
2011-09-08
S016718J
7 THE TRACK, BRIGHT VIC 3741
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S023304S&cacheApplicationListContext=true
2012-05-14
S023304S
6 DIGGINGS ROAD, TAWONGA VIC 3697
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S021086C&cacheApplicationListContext=true
2012-02-24
S021086C
137 SCHOOL ROAD, WANDILIGONG VIC 3744
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml
2012-09-17
Two lot resubdivision
https://www.spear.land.vic.gov.au/spear/applicationDetails/RetrievePublicApplication.do?spearNum=S023932B&cacheApplicationListContext=true
2012-06-05
S023932B
8 TYNTYNDER LANE, BRIGHT VIC 3741
https://www.spear.land.vic.gov.au/spear/pages/public-and-other-users/objectors.shtml

Statistics

Average successful run time: 12 minutes

Total run time: 18 days

Total cpu time used: about 4 hours

Total disk space used: 38.8 MB

History

  • Auto ran revision 5e3a3442 and completed successfully .
    2665 records added, 2651 records removed in the database
  • Auto ran revision 5e3a3442 and completed successfully .
    2665 records added, 2625 records removed in the database
  • Auto ran revision 5e3a3442 and completed successfully .
    2665 records added, 2605 records removed in the database
  • Auto ran revision 5e3a3442 and completed successfully .
    2666 records added, 2663 records removed in the database
  • Auto ran revision 5e3a3442 and failed .
    nothing changed in the database
  • ...
  • Forked from ScraperWiki

Show complete history

Scraper code

Ruby

spear / scraper.rb