LoveMyData / mornington_peninsula

Mornington Peninsula Shire - Development Applications

Scrapes www.mornpen.vic.gov.au

Home - Mornington Peninsula


Mornington Peninsula Shire Council scraper

Setup MORPH_PROXY (host:port) to make proxy request for web pages

Contributors LoveMyData henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.4.1 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching unf_ext 0.0.7.4  Fetching httpclient 2.8.3  Fetching mime-types-data 3.2016.0521  Installing httpclient 2.8.3  Installing mime-types-data 3.2016.0521  Installing unf_ext 0.0.7.4 with native extensions  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching net-http-persistent 2.9.4  Fetching mini_portile2 2.3.0  Installing net-http-persistent 2.9.4  Installing mini_portile2 2.3.0  Fetching ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing ntlm-http 0.1.1  Installing webrobots 0.1.2  Fetching sqlite3 1.3.13  Fetching mime-types 3.1  Installing mime-types 3.1  Installing sqlite3 1.3.13 with native extensions  Fetching nokogiri 1.8.1  Installing nokogiri 1.8.1 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20170404  Installing domain_name 0.5.20170404  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.5  Installing mechanize 2.7.5  Bundle complete! 2 Gemfile dependencies, 18 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (26.48s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Checking page 1 of 15 Skipping already saved record P16/0548.01 Skipping already saved record P18/021 Skipping already saved record P1/1840.37 Skipping already saved record P18/0262 Skipping already saved record P18/0148 Skipping already saved record P12/1956.02 Skipping already saved record P17/238 Skipping already saved record P18/0506 Skipping already saved record P17/0354.01 Checking page 2 of 15 Skipping already saved record P18/0230 Something not right here: {"council_reference"=>"P17/2560 - (VCAT Advertising)", "description"=>nil, "address"=>"646 Nepean Highway, Mount Martha, VIC", "info_url"=>"https://www.mornpen.vic.gov.au/Building-Planning/Planning/Advertised-Planning-Applications/P172560-646-Nepean-Highway-Mount-Martha-VCAT-Ref.-P8452018", "comment_url"=>"mailto:planning.submission@mornpen.vic.gov.au", "date_scraped"=>"2018-06-18"} Skipping already saved record P17/2349 Skipping already saved record P18/0130 Skipping already saved record P18/0427 Skipping already saved record P18/067 Skipping already saved record P18/0176 Skipping already saved record P18/029 Skipping already saved record P18/0621 Checking page 3 of 15 Skipping already saved record P18/0629 Skipping already saved record P18/0632 Skipping already saved record P18/063 Skipping already saved record P18/0634 Skipping already saved record P18/0719 Skipping already saved record P18/073 Skipping already saved record P09/0815.03 Skipping already saved record P17/1691 Skipping already saved record P17/2452 Checking page 4 of 15 Skipping already saved record P17/2546 Skipping already saved record P18/0132 Skipping already saved record P18/0180 Skipping already saved record P18/0257 Skipping already saved record P18/0527 Skipping already saved record P18/0530 Skipping already saved record P18/057 Skipping already saved record P18/0627 Skipping already saved record P17/079 Checking page 5 of 15 Skipping already saved record P18/038 Skipping already saved record P18/0272 Skipping already saved record P18/028 Skipping already saved record P18/0356 Skipping already saved record P18/0375 Skipping already saved record P18/0401 Skipping already saved record P18/056 Skipping already saved record P18/0817 Skipping already saved record P16/1307.01 Checking page 6 of 15 Skipping already saved record P18/0249 Skipping already saved record P18/0402 Skipping already saved record P18/0730 Skipping already saved record P14/0267.05 Skipping already saved record P16/061.03 Skipping already saved record P16/2165.05 Skipping already saved record P18/062 Skipping already saved record P18/074 Skipping already saved record P18/0215 Checking page 7 of 15 Skipping already saved record P18/0497 Skipping already saved record P18/0525 Skipping already saved record P18/0578 Something not right here: {"council_reference"=>"P17/10", "description"=>nil, "address"=>"219 Main Street, Mornington, VIC", "info_url"=>"https://www.mornpen.vic.gov.au/Building-Planning/Planning/Advertised-Planning-Applications/P171100-219-257-Main-Street-Mornington-VCAT-Ref.-P9382018", "comment_url"=>"mailto:planning.submission@mornpen.vic.gov.au", "date_scraped"=>"2018-06-18"} Skipping already saved record P18/0280 Skipping already saved record P18/0429 Skipping already saved record P18/0517 Skipping already saved record P18/0641 Skipping already saved record P18/069 Checking page 8 of 15 Skipping already saved record P16/106.03 Skipping already saved record P17/ 1737 Skipping already saved record P18/0297 Skipping already saved record P18/0315 Skipping already saved record P16/21.01 Skipping already saved record P16/212.01 Skipping already saved record P16/213.01 Skipping already saved record P16/2454.01 Skipping already saved record P17/109 Checking page 9 of 15 Skipping already saved record P18/020 Skipping already saved record P18/0482 Skipping already saved record P18/0548 Skipping already saved record P18/0710 Skipping already saved record P18/0540 Skipping already saved record P18/0327 Skipping already saved record P09/2629.05 Skipping already saved record P13/213.03 Skipping already saved record P15/147.04 Checking page 10 of 15 Skipping already saved record P17/0841.01 Skipping already saved record P18/0246 Skipping already saved record P18/0285 Skipping already saved record P18/0289 Skipping already saved record P18/0654 Skipping already saved record P18/1051 Skipping already saved record P16/2184.01 Skipping already saved record P18/0259 Skipping already saved record P18/0649 Checking page 11 of 15 Skipping already saved record P18/0731 Skipping already saved record P18/0242 Skipping already saved record P18/0673 Skipping already saved record P18/0691 Skipping already saved record P18/0374 Skipping already saved record P18/0512 Skipping already saved record P18/0549 Skipping already saved record P18/0680 Skipping already saved record P18/0653 Checking page 12 of 15 Skipping already saved record P17/2039.01 Skipping already saved record P17/2413 Skipping already saved record P18/0192 Skipping already saved record P18/0305 Skipping already saved record P18/0459 Skipping already saved record P18/0504 Skipping already saved record P18/0563 Skipping already saved record P18/0616 Skipping already saved record P18/067 Checking page 13 of 15 Skipping already saved record P15/1812.03 Skipping already saved record P16/1945.01 Skipping already saved record P17/2520 Skipping already saved record P18/054 Skipping already saved record P18/028 Skipping already saved record P18/0392 Skipping already saved record P18/0414 Skipping already saved record P13/0637.07 Skipping already saved record P13/145.04 Checking page 14 of 15 Skipping already saved record P15/0216.03 Skipping already saved record P15/0684.03 Skipping already saved record P18/042 Skipping already saved record P18/0486 Skipping already saved record P18/064 Skipping already saved record P18/0968 Skipping already saved record P18/092 Skipping already saved record P18/093 Skipping already saved record P18/1019 Checking page 15 of 15 Skipping already saved record P17/172 Skipping already saved record P18/0162 Skipping already saved record P18/0486 Skipping already saved record P18/0397

Data

Downloaded 546 times by LoveMyData

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.48 MB) Use the API

rows 10 / 2657

council_reference description address info_url comment_url date_scraped
P14/0484.04
Development of a dwelling and associated works in accordance with the endorsed plans
41 Hull Road Mount Martha, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1234
Develop a single storey dwelling and associated works generally in accordance with the submitted plans
35 Leonard Street Tootgarook, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1427
Develop a single dwelling and associated works generally in accordance with the submitted plans
17 Macks Place Safety Beach, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1583
Develop dwelling additions (second storey) and alterations, remove vegetation and associated works generally in accordance with the submitted plans
9 Woodland Avenue Mount Eliza, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1373
Alterations and additions to the existing dwelling, vegetation removal and associated works generally in accordance with the submitted plans
13 Tourello Road Mout Eliza, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/0565
Development of a dwelling and associated works generally in accordance with the submitted plans
3 Rendlesham Avenue Mount Eliza, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1518
Development of two carports and associated works generally in accordance with the submitted plans
28 Seaview Road Tyabb, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1520
Development of outbuilding and associated works generally in accordance with the submitted plans
104 Grandview Terrace Mount Martha, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/0832
Change of use to animal keeping (dog day care) and associated works generally in accordance with the submitted plans
35 Mckirdys Road Tyabb, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27
P16/1291
Develop a carport and associated works generally in accordance with the submitted plans
1/68 James Street Hastings, VIC
mailto:planning.submission@mornpen.vic.gov.au
2016-09-27

Statistics

Average successful run time: 6 minutes

Total run time: 2 days

Total cpu time used: about 1 hour

Total disk space used: 1.51 MB

History

  • Auto ran revision b9173281 and completed successfully .
    nothing changed in the database
    161 pages scraped
  • Auto ran revision b9173281 and completed successfully .
    2 records added in the database
  • Auto ran revision b9173281 and completed successfully .
    nothing changed in the database
    163 pages scraped
  • Auto ran revision b9173281 and completed successfully .
    9 records added in the database
    163 pages scraped
  • Auto ran revision b9173281 and completed successfully .
    nothing changed in the database
    150 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

mornington_peninsula / scraper.rb