LoveMyData / lake_macquarie
Lake Macquarie Development Applications (ATDIS)
Lake Macquarie City Council Scraper
This council involves the followings
- Server - .NET - ApplicationMaster
- Accept Terms - Yes
- Cookie tracking - Yes
- UserAgent - No
- Request actual DA page for data - No
- Clearly defined data within a row - No, No, No, Very messy
Setup MORPH_PERIOD for data recovery, available options are
- thisweek (default)
- thismonth
- lastmonth
- custom = yyyy-mm (eg. 2017-01)
Enjoy
Contributors
Last run completed successfully
.
Console output of last run
Injecting configuration and compiling...
[1G [1G-----> Ruby app detected
[1G-----> Compiling Ruby
[1G-----> Using Ruby version: ruby-2.4.1
[1G-----> Installing dependencies using bundler 1.15.2
[1G Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
[1G Fetching gem metadata from https://rubygems.org/........
[1G Fetching version metadata from https://rubygems.org/.
[1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git
[1G Using bundler 1.15.2
[1G Fetching connection_pool 2.2.2
[1G Fetching unf_ext 0.0.7.5
[1G Fetching httpclient 2.8.3
[1G Installing connection_pool 2.2.2
[1G Fetching mime-types-data 3.2018.0812
[1G Installing httpclient 2.8.3
[1G Installing unf_ext 0.0.7.5 with native extensions
[1G Installing mime-types-data 3.2018.0812
[1G Fetching net-http-digest_auth 1.4.1
[1G Installing net-http-digest_auth 1.4.1
[1G Fetching mini_portile2 2.3.0
[1G Installing mini_portile2 2.3.0
[1G Fetching ntlm-http 0.1.1
[1G Fetching webrobots 0.1.2
[1G Installing ntlm-http 0.1.1
[1G Fetching sqlite3 1.3.13
[1G Installing webrobots 0.1.2
[1G Installing sqlite3 1.3.13 with native extensions
[1G Fetching net-http-persistent 3.0.0
[1G Installing net-http-persistent 3.0.0
[1G Fetching mime-types 3.2.2
[1G Installing mime-types 3.2.2
[1G Fetching nokogiri 1.8.5
[1G Installing nokogiri 1.8.5 with native extensions
[1G Fetching unf 0.1.4
[1G Installing unf 0.1.4
[1G Fetching domain_name 0.5.20180417
[1G Installing domain_name 0.5.20180417
[1G Fetching http-cookie 1.0.3
[1G Installing http-cookie 1.0.3
[1G Fetching sqlite_magic 0.0.6
[1G Installing sqlite_magic 0.0.6
[1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)
[1G Fetching mechanize 2.7.6
[1G Installing mechanize 2.7.6
[1G Bundle complete! 2 Gemfile dependencies, 19 gems now installed.
[1G Gems in the groups development and test were not installed.
[1G Bundled gems are installed into ./vendor/bundle.
[1G Bundle completed (27.85s)
[1G Cleaning up the bundler cache.
[1G-----> Detecting rake tasks
[1G
[1G [1G-----> Discovering process types
[1G Procfile declares types -> scraper
Injecting scraper and running...
Getting data in `thisweek`, changable via MORPH_PERIOD environment
Data
Downloaded
180 times
by
To download data sign in with GitHub
rows 10 / 1573
| council_reference | address | description | info_url | comment_url | date_scraped | date_received |
|---|---|---|---|---|---|---|
|
DA-1690/2018
|
5 Jonquil Close TORONTO NSW 2283
|
Dual Occupancy Attached 1 into 2 Lot Torrens Title Subdivison
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1691/2018
|
38 Stingaree Point Drive DORA CREEK NSW 2264
|
Alfresco and Garage
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1512/2017/B
|
43 Croft Road ELEEBANA NSW 2282
|
Proposed Garage and Retaining Wall - Section 4.55 Amended Plans
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1692/2018
|
282 Pacific Highway BELMONT NORTH NSW 2280
|
Alterations and Additions
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1693/2018
|
52 Mahers Road MARTINSVILLE NSW 2265
|
Shed
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1694/2018
|
9 Trafalgar Crescent VALENTINE NSW 2280
|
Alterations to Existing Deck, Concrete Swimming Pool and Safety Barriers, Roofed Pergola and Retaining Walls
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1695/2018
|
99 Glad Gunson Drive ELEEBANA NSW 2282
|
Deck
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1696/2018
|
24 Tora Street TORONTO NSW 2283
|
Storage Shed
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1697/2018
|
1 Ernest Street BELMONT NSW 2280
|
Garaport & Workshop/Storage Shed
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
|
|
DA-1698/2018
|
10 McIntyre Street CARDIFF NSW 2285
|
Garage
|
mailto:council@lakemac.nsw.gov.au
|
2018-11-28
|
2018-10-02
|
Statistics
Average successful run time: less than a minute
Total run time: about 3 hours
Total cpu time used: 3 minutes
Total disk space used: 565 KB
History
-
Auto
ran
revision
48a6703d
and
completed successfully
.
nothing changed in the database
-
Auto
ran
revision
48a6703d
and
completed successfully
.
24 records added in the database
-
Auto
ran
revision
48a6703d
and
completed successfully
.
4 records added in the database
-
Auto
ran
revision
48a6703d
and
completed successfully
.
11 records added in the database
-
Auto
ran
revision
48a6703d
and
completed successfully
.
18 records added in the database
- ...
- Created on morph.io