LoveMyData / lake_macquarie
Lake Macquarie Development Applications (ATDIS)
Lake Macquarie City Council Scraper
This council involves the followings
- Server - .NET - ApplicationMaster
- Accept Terms - Yes
- Cookie tracking - Yes
- UserAgent - No
- Request actual DA page for data - No
- Clearly defined data within a row - No, No, No, Very messy
Setup MORPH_PERIOD for data recovery, available options are
- thisweek (default)
- thismonth
- lastmonth
- custom = yyyy-mm (eg. 2017-01)
Enjoy
Contributors
Last run completed successfully
.
Console output of last run
Injecting configuration and compiling...
[1G [1G-----> Ruby app detected
[1G-----> Compiling Ruby
[1G-----> Using Ruby version: ruby-2.4.1
[1G-----> Installing dependencies using bundler 1.15.2
[1G Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
[1G Fetching gem metadata from https://rubygems.org/........
[1G Fetching version metadata from https://rubygems.org/.
[1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git
[1G Using bundler 1.15.2
[1G Fetching connection_pool 2.2.2
[1G Fetching unf_ext 0.0.7.5
[1G Fetching httpclient 2.8.3
[1G Installing connection_pool 2.2.2
[1G Fetching mime-types-data 3.2018.0812
[1G Installing unf_ext 0.0.7.5 with native extensions
[1G Installing httpclient 2.8.3
[1G Fetching net-http-digest_auth 1.4.1
[1G Installing mime-types-data 3.2018.0812
[1G Installing net-http-digest_auth 1.4.1
[1G Fetching mini_portile2 2.3.0
[1G Installing mini_portile2 2.3.0
[1G Fetching ntlm-http 0.1.1
[1G Fetching webrobots 0.1.2
[1G Installing ntlm-http 0.1.1
[1G Installing webrobots 0.1.2
[1G Fetching sqlite3 1.3.13
[1G Fetching net-http-persistent 3.0.0
[1G Installing sqlite3 1.3.13 with native extensions
[1G Installing net-http-persistent 3.0.0
[1G Fetching mime-types 3.2.2
[1G Installing mime-types 3.2.2
[1G Fetching nokogiri 1.8.5
[1G Installing nokogiri 1.8.5 with native extensions
[1G Fetching unf 0.1.4
[1G Installing unf 0.1.4
[1G Fetching domain_name 0.5.20180417
[1G Installing domain_name 0.5.20180417
[1G Fetching http-cookie 1.0.3
[1G Installing http-cookie 1.0.3
[1G Fetching sqlite_magic 0.0.6
[1G Installing sqlite_magic 0.0.6
[1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)
[1G Fetching mechanize 2.7.6
[1G Installing mechanize 2.7.6
[1G Bundle complete! 2 Gemfile dependencies, 19 gems now installed.
[1G Gems in the groups development and test were not installed.
[1G Bundled gems are installed into ./vendor/bundle.
[1G Bundle completed (26.26s)
[1G Cleaning up the bundler cache.
[1G-----> Detecting rake tasks
[1G
[1G [1G-----> Discovering process types
[1G Procfile declares types -> scraper
Injecting scraper and running...
Getting data in `thisweek`, changable via MORPH_PERIOD environment
Skipping already saved record DA-1682/2018/A
Skipping already saved record DA-613/2017/A
Skipping already saved record DA-228/2019
Skipping already saved record DA-2312/1992/B
Skipping already saved record DA-229/2019
Skipping already saved record DA-230/2019
Skipping already saved record DA-231/2019
Skipping already saved record DA-1938/2015/A
Saving record DA-232/2019 - 19 Perly Grove CAMERON PARK NSW 2285
Saving record DA-1760/2018/A - 65 Dandaraga Road BRIGHTWATERS NSW 2264
Saving record DA-233/2019 - 10 George Street HIGHFIELDS NSW 2289
Saving record DA-234/2019 - 17 Jupiter Circuit CAMERON PARK NSW 2285
Saving record DA-235/2019 - 7 Apple Gum Way MURRAYS BEACH NSW 2281
Saving record DA-751/2016/A - 3 Cormorant Close MURRAYS BEACH NSW 2281
Saving record DA-236/2019 - 281 Watkins Road WANGI WANGI NSW 2267
Data
Downloaded
85 times
by
To download data sign in with GitHub
rows 10 / 979
| council_reference | address | description | info_url | comment_url | date_scraped | date_received |
|---|---|---|---|---|---|---|
|
DA-1690/2018
|
5 Jonquil Close TORONTO NSW 2283
|
Dual Occupancy Attached 1 into 2 Lot Torrens Title Subdivison
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1691/2018
|
38 Stingaree Point Drive DORA CREEK NSW 2264
|
Alfresco and Garage
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1512/2017/B
|
43 Croft Road ELEEBANA NSW 2282
|
Proposed Garage and Retaining Wall - Section 4.55 Amended Plans
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1692/2018
|
282 Pacific Highway BELMONT NORTH NSW 2280
|
Alterations and Additions
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1693/2018
|
52 Mahers Road MARTINSVILLE NSW 2265
|
Shed
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1694/2018
|
9 Trafalgar Crescent VALENTINE NSW 2280
|
Alterations to Existing Deck, Concrete Swimming Pool and Safety Barriers, Roofed Pergola and Retaining Walls
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1695/2018
|
99 Glad Gunson Drive ELEEBANA NSW 2282
|
Deck
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1696/2018
|
24 Tora Street TORONTO NSW 2283
|
Storage Shed
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1697/2018
|
1 Ernest Street BELMONT NSW 2280
|
Garaport & Workshop/Storage Shed
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1698/2018
|
10 McIntyre Street CARDIFF NSW 2285
|
Garage
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
Statistics
Average successful run time: less than a minute
Total run time: about 1 hour
Total cpu time used: 1 minute
Total disk space used: 366 KB
History
-
Auto
ran
revision
48a6703d
and
completed successfully
.
7 records added in the database
-
Auto
ran
revision
48a6703d
and
completed successfully
.
8 records added in the database4 pages scraped
-
Auto
ran
revision
48a6703d
and
completed successfully
.
nothing changed in the database4 pages scraped
-
Auto
ran
revision
48a6703d
and
completed successfully
.
nothing changed in the database4 pages scraped
-
Auto
ran
revision
48a6703d
and
completed successfully
.
12 records added in the database4 pages scraped
- ...
- Created on morph.io