Contributors CloCkWeRX

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.4 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  The git source `git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git` uses the `git` protocol, which transmits data without encryption. Disable this warning with `bundle config git.allow_insecure true`, or switch to the `https` protocol to keep your data secure.  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git  Fetching concurrent-ruby 1.0.5  Fetching i18n 0.8.4  Fetching minitest 5.10.2  Installing i18n 0.8.4  Installing minitest 5.10.2  Installing concurrent-ruby 1.0.5  Fetching thread_safe 0.3.6  Fetching multi_json 1.12.1  Installing multi_json 1.12.1  Fetching unf_ext 0.0.7.4  Fetching mime-types-data 3.2016.0521  Installing mime-types-data 3.2016.0521  Fetching netrc 0.11.0  Installing thread_safe 0.3.6  Fetching rgeo 0.6.0  Installing rgeo 0.6.0 with native extensions  Installing netrc 0.11.0  Fetching httpclient 2.8.3  Installing unf_ext 0.0.7.4 with native extensions  Installing httpclient 2.8.3  Fetching sqlite3 1.3.13  Installing sqlite3 1.3.13 with native extensions  Using bundler 1.15.2  Fetching mime-types 3.1  Installing mime-types 3.1  Fetching tzinfo 1.2.3  Installing tzinfo 1.2.3  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching activesupport 5.1.1  Installing activesupport 5.1.1  Fetching domain_name 0.5.20170404  Installing domain_name 0.5.20170404  Fetching activemodel 5.1.1  Installing activemodel 5.1.1  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching rest-client 2.0.2  Installing rest-client 2.0.2  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Fetching scraperwiki 3.0.2  Installing scraperwiki 3.0.2  Fetching scraperwiki-morph 0.1.1  Installing scraperwiki-morph 0.1.1  Fetching rgeo-geojson 0.4.3  Installing rgeo-geojson 0.4.3  Fetching atdis 0.3.13  Installing atdis 0.3.13  Using atdisplanningalertsfeed 0.0.1 from git://github.com/planningalerts-scrapers/atdisplanningalertsfeed.git (at master@7acbbe2)  Bundle complete! 1 Gemfile dependency, 26 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (17.21s)  Cleaning up the bundler cache. -----> Detecting rake tasks   ###### WARNING:  You have not declared a Ruby version in your Gemfile.  To set your Ruby version add this line to your Gemfile:  ruby '2.3.4'  # See https://devcenter.heroku.com/articles/ruby-versions for more information.   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... D, [2019-04-21T11:53:47.560964 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2019-04-21&lodgement_date_start=2019-03-22&page=3 D, [2019-04-21T11:53:52.124580 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2019-04-21&lodgement_date_start=2019-03-22&page=4 D, [2019-04-21T11:53:56.660604 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2019-04-21&lodgement_date_start=2019-03-22&page=5 D, [2019-04-21T11:54:01.277757 #16] DEBUG -- : Fetching http://masterview.bathurst.nsw.gov.au/atdis/1.0//applications.json?lodgement_date_end=2019-04-21&lodgement_date_start=2019-03-22&page=6 D, [2019-04-21T11:54:02.664885 #16] DEBUG -- : Fetching I, [2019-04-21T11:54:02.668014 #16] INFO -- : Skipping already saved record 010.2019.00000096.001 I, [2019-04-21T11:54:02.668223 #16] INFO -- : Skipping already saved record 010.2019.00000122.001 I, [2019-04-21T11:54:02.668398 #16] INFO -- : Skipping already saved record 010.2019.00000110.001 I, [2019-04-21T11:54:02.668555 #16] INFO -- : Skipping already saved record 010.2019.00000121.001 I, [2019-04-21T11:54:02.668741 #16] INFO -- : Skipping already saved record 010.2019.00000120.001 I, [2019-04-21T11:54:02.668900 #16] INFO -- : Skipping already saved record 010.2018.00000367.002 I, [2019-04-21T11:54:02.669047 #16] INFO -- : Skipping already saved record 010.2019.00000119.001 I, [2019-04-21T11:54:02.669237 #16] INFO -- : Skipping already saved record 018.2019.00000046.001 I, [2019-04-21T11:54:02.669390 #16] INFO -- : Skipping already saved record 010.2019.00000118.001 I, [2019-04-21T11:54:02.669537 #16] INFO -- : Skipping already saved record 010.2019.00000088.001 I, [2019-04-21T11:54:02.669699 #16] INFO -- : Skipping already saved record 010.2019.00000116.001 I, [2019-04-21T11:54:02.669854 #16] INFO -- : Skipping already saved record 010.2019.00000106.001 I, [2019-04-21T11:54:02.670034 #16] INFO -- : Skipping already saved record 010.2019.00000091.001 I, [2019-04-21T11:54:02.670237 #16] INFO -- : Skipping already saved record 010.2019.00000090.001 I, [2019-04-21T11:54:02.670391 #16] INFO -- : Skipping already saved record 010.2019.00000099.001 I, [2019-04-21T11:54:02.670538 #16] INFO -- : Skipping already saved record 010.2019.00000117.001 I, [2019-04-21T11:54:02.670702 #16] INFO -- : Skipping already saved record 010.2019.00000115.001 I, [2019-04-21T11:54:02.670854 #16] INFO -- : Skipping already saved record 010.2019.00000114.001 I, [2019-04-21T11:54:02.670998 #16] INFO -- : Skipping already saved record 018.2019.00000035.001 I, [2019-04-21T11:54:02.671180 #16] INFO -- : Skipping already saved record 018.2019.00000045.001 I, [2019-04-21T11:54:02.671325 #16] INFO -- : Skipping already saved record 010.2019.00000104.001 I, [2019-04-21T11:54:02.671489 #16] INFO -- : Skipping already saved record 010.2019.00000094.001 I, [2019-04-21T11:54:02.671637 #16] INFO -- : Skipping already saved record 018.2019.00000044.001 I, [2019-04-21T11:54:02.671789 #16] INFO -- : Skipping already saved record 018.2019.00000043.001 I, [2019-04-21T11:54:02.671939 #16] INFO -- : Skipping already saved record 010.2019.00000111.001 I, [2019-04-21T11:54:02.672120 #16] INFO -- : Skipping already saved record 010.2019.00000112.001 I, [2019-04-21T11:54:02.672265 #16] INFO -- : Skipping already saved record 010.2019.00000113.001 I, [2019-04-21T11:54:02.672416 #16] INFO -- : Skipping already saved record 010.2019.00000107.001 I, [2019-04-21T11:54:02.672563 #16] INFO -- : Skipping already saved record 018.2019.00000040.001 I, [2019-04-21T11:54:02.672737 #16] INFO -- : Skipping already saved record 018.2019.00000042.001 I, [2019-04-21T11:54:02.672886 #16] INFO -- : Skipping already saved record 010.2019.00000109.001 I, [2019-04-21T11:54:02.673037 #16] INFO -- : Skipping already saved record 010.2019.00000098.001 I, [2019-04-21T11:54:02.673193 #16] INFO -- : Skipping already saved record 010.2019.00000108.001 I, [2019-04-21T11:54:02.673386 #16] INFO -- : Skipping already saved record 018.2019.00000041.001 I, [2019-04-21T11:54:02.673547 #16] INFO -- : Skipping already saved record 010.2018.00000396.002 I, [2019-04-21T11:54:02.673694 #16] INFO -- : Skipping already saved record 010.2019.00000105.001 I, [2019-04-21T11:54:02.673840 #16] INFO -- : Skipping already saved record 010.2019.00000103.001 I, [2019-04-21T11:54:02.674032 #16] INFO -- : Skipping already saved record 010.2019.00000102.001 I, [2019-04-21T11:54:02.674189 #16] INFO -- : Skipping already saved record 010.2019.00000101.001 I, [2019-04-21T11:54:02.674334 #16] INFO -- : Skipping already saved record 010.2019.00000100.001 I, [2019-04-21T11:54:02.674478 #16] INFO -- : Skipping already saved record 018.2019.00000037.001 I, [2019-04-21T11:54:02.674621 #16] INFO -- : Skipping already saved record 010.2019.00000097.001 I, [2019-04-21T11:54:02.674794 #16] INFO -- : Skipping already saved record 010.2019.00000095.001 I, [2019-04-21T11:54:02.674942 #16] INFO -- : Skipping already saved record 010.2019.00000093.001 I, [2019-04-21T11:54:02.675107 #16] INFO -- : Skipping already saved record 010.2019.00000089.001 I, [2019-04-21T11:54:02.675251 #16] INFO -- : Skipping already saved record 010.2019.00000092.001 I, [2019-04-21T11:54:02.675428 #16] INFO -- : Skipping already saved record 018.2019.00000039.001 I, [2019-04-21T11:54:02.675573 #16] INFO -- : Skipping already saved record 018.2019.00000038.001 I, [2019-04-21T11:54:02.675742 #16] INFO -- : Skipping already saved record 018.2019.00000034.001 I, [2019-04-21T11:54:02.675884 #16] INFO -- : Skipping already saved record 010.2019.00000087.001 I, [2019-04-21T11:54:02.676043 #16] INFO -- : Skipping already saved record 018.2019.00000036.001

Data

Downloaded 2268 times by openaustralia LoveMyData WilliamYun jesse-a-reilly TheZepto MikeRalphson srsmonty Jedwards157 jayzhouny

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (720 KB) Use the API

rows 10 / 2255

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to
010.2016.00000058.001
457 White Rock RD, WHITE ROCK, NSW 2795
Second rural dwelling
2016-03-21
2016-02-25
010.2016.00000073.001
Mountain STGT, MOUNT PANORAMA, NSW 2795
Construction of temporary structures
2016-03-21
2016-03-14
018.2016.00000046.001
10 Barney ST, WINDRADYNE, NSW 2795
Construction of a single storey dwelling with attached garage
2016-03-21
2016-03-18
018.2016.00000047.001
29 Barr ST, WINDRADYNE, NSW 2795
Single storey dwelling and attached garage
2016-03-21
2016-03-21
010.2016.00000077.001
92 Corporation AVE, ROBIN HILL, NSW 2795
Commercial change of use and 12 advertising signs
2016-03-21
2016-03-21
010.2016.00000075.001
220 Lambert ST, BATHURST, NSW 2795
Additions and alterations to dwelling
2016-03-21
2016-03-18
010.2016.00000074.001
16 Halfpenny DR, KELSO, NSW 2795
Garage
2016-03-21
2016-03-17
010.2016.00000063.001
13 Oakes ST, BATHURST, NSW 2795
1. Change of use to motor vehicle repair station 2. One flush wall sign
2016-03-21
2016-03-01
010.2016.00000061.001
126A Lambert ST, BATHURST, NSW 2795
Additions and alterations
2016-03-21
2016-02-29
018.2016.00000044.001
14 Maxwell DR, EGLINTON, NSW 2795
Construction of a single storey dwelling with attached garage
2016-03-21
2016-03-16

Statistics

Average successful run time: 2 minutes

Total run time: 1 day

Total cpu time used: 22 minutes

Total disk space used: 741 KB

History

  • Auto ran revision 01683dd7 and completed successfully .
    nothing changed in the database
  • Auto ran revision 01683dd7 and completed successfully .
    nothing changed in the database
  • Auto ran revision 01683dd7 and completed successfully .
    2 records added in the database
  • Auto ran revision 01683dd7 and completed successfully .
    3 records added in the database
  • Auto ran revision 01683dd7 and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

bathurst / scraper.rb