planningalerts-scrapers / mackay

Mackay Regional Council Development Applications

Scrapes planning.mackay.qld.gov.au

MasterView 2.0 Application Master


Mackay Regional Council Scraper

  • Server - .NET - Application Master
  • Cookie tracking - Yes
  • Pagnation - Yes
  • Javascript - Yes
  • Clearly defined data within a row - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default)
  • thismonth
  • lastmonth

Enjoy

Forked from ScraperWiki

Contributors CloCkWeRX henare LoveMyData

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching unf_ext 0.0.7.4  Fetching httpclient 2.8.3  Fetching mime-types-data 3.2016.0521  Installing mime-types-data 3.2016.0521  Fetching net-http-digest_auth 1.4.1  Installing unf_ext 0.0.7.4 with native extensions  Installing httpclient 2.8.3  Installing net-http-digest_auth 1.4.1  Fetching net-http-persistent 2.9.4  Fetching mini_portile2 2.1.0  Installing mini_portile2 2.1.0  Installing net-http-persistent 2.9.4  Fetching ntlm-http 0.1.1  Fetching webrobots 0.1.2  Installing ntlm-http 0.1.1  Installing webrobots 0.1.2  Fetching sqlite3 1.3.13  Fetching mime-types 3.1  Installing mime-types 3.1  Installing sqlite3 1.3.13 with native extensions  Fetching nokogiri 1.6.8.1  Installing nokogiri 1.6.8.1 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20170404  Installing domain_name 0.5.20170404  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.5  Installing mechanize 2.7.5  Bundle complete! 2 Gemfile dependencies, 18 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (25.23s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data in `thisweek`, changable via MORPH_PERIOD variable Skipping already saved record ROLC-44/2018

Data

Downloaded 3075 times by slav123 openaustralia LoveMyData TheZepto henare

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (580 KB) Use the API

rows 10 / 1462

date_scraped description info_url date_received council_reference address comment_url
2011-09-10
Dual Occupancy
2011-09-01
DA - 2011 - 233
38 Schooner Avenue SHOAL POINT QLD 4750
mailto:development.services@mackay.qld.gov.au
2011-09-10
Dual Occupancy
2011-09-01
DA - 2011 - 234
59-61 Schooner Avenue SHOAL POINT QLD 4750
mailto:development.services@mackay.qld.gov.au
2011-09-10
Dual Occupancy
2011-09-01
DA - 2011 - 235
1 Galleon Circuit SHOAL POINT QLD 4750
mailto:development.services@mackay.qld.gov.au
2011-09-10
Material Change of Use - Outbuilding (Major Transport and Energy Corridor Overlay Code)
2011-09-05
DA - 2011 - 232
L 1 Bruce Highway SARINA QLD 4737
mailto:development.services@mackay.qld.gov.au
2011-09-10
Extension of the currency period - Combined Application for a Material Change of Use AND a Reconfiguration of 1 Lot to create 4 Urban Residential Lots
2011-08-04
CA - 2006 - 80
59 Coles Road ANDERGROVE QLD 4740
mailto:development.services@mackay.qld.gov.au
2011-09-10
Extension of Relevant Period - Combined Application for a Material Change of Use AND a Reconfiguration of 1 Lot to create 4 Urban Residential Lots
2011-08-04
CA - 2006 - 81
55 Coles Road ANDERGROVE QLD 4740
mailto:development.services@mackay.qld.gov.au
2011-09-10
Change of Plan of Development - Combined Application for a Material Change of Use (8 Multiple Dwelling Units) AND a Reconfiguration of 1 Lot to create 8 Community Title Residential Lots plus Common Property under the Superseded Planning Scheme
2011-08-15
CA - 2007 - 414
5 Canberra Street NORTH MACKAY QLD 4740
mailto:development.services@mackay.qld.gov.au
2011-09-10
Extension of Relevant Period - Reconfiguration of a Lot (1 into 2 Urban Residential lots) and Dual Occupancy / House on Lot less than 450m2 in area
2011-08-23
CA - 2007 - 237
172 Goldsmith Street SOUTH MACKAY QLD 4740
mailto:development.services@mackay.qld.gov.au
2011-09-10
Request to Change a Development Approval - Material Change of Use - Code Assessment - Marina International Hotel.
2011-08-08
MCUCD - 2003 - 60
40 Mulherin Drive MACKAY HARBOUR QLD 4740 34-38 Mulherin Drive MACKAY HARBOUR QLD 4740
mailto:development.services@mackay.qld.gov.au
2011-09-10
Extension of Relevant Period - Material Change of Use - Warehouse Units with Ancillary Offices
2011-08-12
MCUCD - 2007 - 219
23 Transport Avenue PAGET QLD 4740
mailto:development.services@mackay.qld.gov.au

Statistics

Average successful run time: less than a minute

Total run time: 4 days

Total cpu time used: 31 minutes

Total disk space used: 607 KB

History

  • Auto ran revision 9f9ed50a and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9f9ed50a and completed successfully .
    1 record added in the database
    1 page scraped
  • Auto ran revision 9f9ed50a and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9f9ed50a and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9f9ed50a and completed successfully .
    nothing changed in the database
    1 page scraped
  • ...
  • Forked from ScraperWiki

Show complete history

Scraper code

Ruby

mackay / scraper.rb