planningalerts-scrapers / onkaparinga

Onkaparinga SA Development Applications

Scrapes pathway.onkaparinga.sa.gov.au

IIS7


City Of Onkaparinga Council Scraper

Note - This scraper will only scrape the primary DA and any subsequent DAs will not be scrape. eg. 683/2017/2 will not be scraped but 683/2017 will

  • Server - .NET - ePathway
  • Cookie tracking - Yes
  • Pagination - No
  • Javascript - Yes
  • Clearly defined data within a row - No but acceptable
  • Force scan one record at a time - Yes
  • User-Agent - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • year value (default is this year)

Enjoy

Forked from ScraperWiki

Contributors emikulic henare LoveMyData equivalentideas

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching unf_ext 0.0.7.1  Fetching httpclient 2.6.0.1  Fetching mime-types 2.6.1  Installing httpclient 2.6.0.1  Installing mime-types 2.6.1  Fetching net-http-digest_auth 1.4  Installing net-http-digest_auth 1.4  Fetching net-http-persistent 2.9.4  Fetching mini_portile 0.6.2  Installing mini_portile 0.6.2  Installing unf_ext 0.0.7.1 with native extensions  Fetching ntlm-http 0.1.1  Installing net-http-persistent 2.9.4  Fetching webrobots 0.1.1  Installing ntlm-http 0.1.1  Installing webrobots 0.1.1  Fetching sqlite3 1.3.10  Fetching nokogiri 1.6.6.2  Installing sqlite3 1.3.10 with native extensions  Installing nokogiri 1.6.6.2 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.24  Installing domain_name 0.5.24  Fetching http-cookie 1.0.2  Installing http-cookie 1.0.2  Fetching sqlite_magic 0.0.5  Installing sqlite_magic 0.0.5  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.3  Installing mechanize 2.7.3  Bundle complete! 2 Gemfile dependencies, 17 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (29.54s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data in year `2018`, changable via MORPH_PERIOD environment Skipping already saved record 896/2018 Saving record 897/2018, 35 Brenton Street, MORPHETT VALE SA 5162 Saving record 898/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 899/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 900/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 901/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 902/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 903/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 904/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 905/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 906/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 907/2018, Allot 1100 Robinson Road, SEAFORD HEIGHTS SA 5169 Saving record 908/2018, 11 Shirley Court, WOODCROFT SA 5162 Saving record 909/2018, 46 Esplanade, SELLICKS BEACH SA 5174 Saving record 910/2018, 6 Fathom Crescent, SEAFORD MEADOWS SA 5169 Saving record 911/2018, 43 Clement Terrace, CHRISTIES BEACH SA 5165 Saving record 912/2018, 43 Clement Terrace, CHRISTIES BEACH SA 5165 Saving record 913/2018, 33 Eden Terrace, PORT WILLUNGA SA 5173 Saving record 914/2018, 12 Gundowringa Avenue, HAPPY VALLEY SA 5159 Saving record 915/2018, 62 Compass Drive, SEAFORD SA 5169 Saving record 916/2018, 7 Gouger Close, WOODCROFT SA 5162 Saving record 917/2018, 19 Birchenough Road, HUNTFIELD HEIGHTS SA 5163 Saving record 918/2018, 53 Homestead Drive, ABERFOYLE PARK SA 5159 Saving record 919/2018, 5 Griffiths Drive, MOANA SA 5169 Saving record 920/2018, 18 Dominic Crescent, MORPHETT VALE SA 5162 Saving record 921/2018, 13 Whistlers Avenue, ALDINGA BEACH SA 5173 Saving record 922/2018, 26 Anselm Street, CHRISTIE DOWNS SA 5164 Saving record 923/2018, 43 Clement Terrace, CHRISTIES BEACH SA 5165 Saving record 924/2018, 107 Black Road, FLAGSTAFF HILL SA 5159 Saving record 926/2018, 339 Upper Penneys Hill Road, ONKAPARINGA HILLS SA 5163 Saving record 927/2018, 90 Albany Avenue, PORT NOARLUNGA SOUTH SA 5167

Data

Downloaded 1540 times by openaustralia LoveMyData TheZepto equivalentideas oliyoung GitHubMLee

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.41 MB) Use the API

rows 10 / 4462

date_scraped description info_url address council_reference on_notice_from on_notice_to comment_url date_received
2011-09-11
Alterations & additions to the existing dwelling, with a solid wall (two storey) proposed along the western side boundary for a length of 13.l metres
80 Esplanade, ALDINGA BEACH SA 5173, SA
145/1284/2011
2011-09-07
2011-09-21
2011-09-11
Mobile phone tower measuring 24.95m in height with associated equipment housing and fencing
Pce 101 DP 59506 Golf Course Drive, WOODCROFT SA 5162, SA
145/2108/2011
2011-08-31
2011-09-14
2011-09-11
Egg storage coolroom and collection handling room in association with existing poultry farm and vineyard (retrospective)
76 Pennys Road, MCLAREN FLAT SA 5171, SA
145/1874/2011
2011-09-05
2011-09-19
2011-09-11
Storage of up to two vehicles at anyone time, including an 8m long x 2m wide x 2.7m high, 5 tonne truck and a 6m long x 2.2m wide x 2.6m high tipper, 8.5 tonne in weight (retrospective)
336-344 Pimpala Road, WOODCROFT, SA
145/1081/2011
2011
2011
2011-09-11
Convert existing dwelling to integrated mixed use (Shara Sanctuary & Retreat) incorporating office, reception, shop, library, day spa, retreat accomodation (up to 7 guests) & storage in association with the existing vineyard- (re-notification due to administrative error)
84 Oakley Road, MCLAREN FLAT, SA
145/1667/2011
2011-09-02
2011-09-16
2011-09-15
To construct moss rock and concrete sleeper retaining walls to a maximum height of approx 2.2m in conjunction with a two storey detached dwelling with associated earthworks
24 Singer Drive, HAPPY VALLEY SA 5159, SA
145/1930/2011
2011-09-14
2011-09-28
2011-09-16
Split level single storey detached dwelling with garage and associated retaining walls up to 1.2m in height (non-complying)
23 Kathrina Street, O'HALLORAN HILL SA 5158, SA
145/1010/2011
2011-09-16
2011-09-30
2011-09-22
Dwelling additions comprising poolroom, ensuite, balconies, front porch, swimming pool and an extension incorporating walls 5.8m in height and with an overall height of 6.4m - Non-Complying
233 Cherry Gardens Road, CHERRY GARDENS SA 5157, SA
145/203/2011
2011-09-26
2011-10-11
2011-09-22
Change of use from consulting room and dwelling to consulting room (non-complying)
267 Main South Road, MORPHETT VALE SA 5162, SA
145/1601/2011
2011-09-23
2011-10-10
2011-09-27
Filling of land and landscaping
108 O'Sullivan Beach Road, LONSDALE SA 5160, SA
145/1869/2011
2011-09-28
2011-10-13

Statistics

Average successful run time: 1 minute

Total run time: 17 days

Total cpu time used: 32 minutes

Total disk space used: 1.44 MB

History

  • Auto ran revision 5102c9de and completed successfully .
    30 records added in the database
    87 pages scraped
  • Auto ran revision 5102c9de and completed successfully .
    13 records added in the database
    51 pages scraped
  • Auto ran revision 5102c9de and completed successfully .
    12 records added in the database
    49 pages scraped
  • Auto ran revision 5102c9de and completed successfully .
    16 records added in the database
    59 pages scraped
  • Auto ran revision 5102c9de and completed successfully .
    nothing changed in the database
    25 pages scraped
  • ...
  • Forked from ScraperWiki

Show complete history

Scraper code

Ruby

onkaparinga / scraper.rb