planningalerts-scrapers / bogan

Bogan Development Applications


Bogan Shire Council Scraper

  • Server - Unknown but I like it
  • Cookie tracking - Yes
  • Agreement - Yes
  • Pagnation - No
  • JSON - Yes - Yay....

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default, today -10 days)
  • thismonth
  • lastmonth
  • year (eg. 2016)

Enjoy

Contributors CloCkWeRX LoveMyData

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.json; requirements  from dependencies in composer.lock will be used for selection  - php (7.2.6)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - nginx (1.8.1)  - apache (2.4.33) -----> Installing dependencies...  Composer version 1.5.2 2017-09-11 16:59:25  Loading composer repositories with package information  Installing dependencies from lock file  Package operations: 4 installs, 0 updates, 0 removals  - Installing pguardiario/phpuri (1.0): Downloading (connecting...)Downloading (100%)   - Installing electrolinux/phpquery (0.9.6): Downloading (connecting...)Downloading (100%)   - Installing byjg/pgbrowser (1.0.0): Downloading (connecting...)Downloading (100%)   - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253  Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data between 14/06/2018 and 24/06/2018, changable via MORPH_PERIOD environment No data returned from feed

Data

Downloaded 377 times by LoveMyData TheZepto MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (17 KB) Use the API

rows 10 / 52

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to
010.2016.00000018.001
27 Ellen St, Nyngan, NSW 2825
New Garage/Carport/Pergola/Shed
2016-08-03
2016-08-02
010.2016.00000020.001
13 Mudal St, Nyngan, NSW 2825
New Dual Occupancy
2016-09-06
2016-09-02
010.2016.00000023.001
31 Dandaloo St, Nyngan, NSW 2825
New Garage and Relocation of Pergola
2016-09-15
2016-09-15
010.2016.00000022.001
876 Barrier Hwy, NYNGAN, NSW 2825
New Education Building
2016-09-15
2016-09-12
010.2016.00000024.001
Lawlor St, NYNGAN, NSW 2825
New Industrial Building
2016-09-27
2016-09-26
010.2017.00000006.001
54 Oatley St, Nyngan, NSW 2825
New Transportable Dwelling
2017-04-10
2017-03-14
2017-04-10
2017-04-21
DA 2017/34
51-57 Flashman Ave, Nyngan 2825 NSW
New Single Dwelling
mailto:admin@bogan.nsw.gov.au
2017-10-29
2017-10-27
DA 2017/33
5 Cathundril St, Nyngan 2825 NSW
Mobile Vending Vehicle
mailto:admin@bogan.nsw.gov.au
2017-10-29
2017-10-26
DA 2017/35
50-52 Cannonbar St, Nyngan 2825 NSW
New Garage/Carport/Pergola/Shed
mailto:admin@bogan.nsw.gov.au
2017-11-22
2017-11-22
CONS 2017/35
50-52 Cannonbar St, Nyngan 2825 NSW
New Garage/Carport/Pergola/Shed
mailto:admin@bogan.nsw.gov.au
2017-11-22
2017-11-22

Statistics

Average successful run time: less than a minute

Total run time: about 12 hours

Total cpu time used: 10 minutes

Total disk space used: 52 KB

History

  • Auto ran revision d7f52647 and completed successfully .
    nothing changed in the database
  • Auto ran revision d7f52647 and completed successfully .
    nothing changed in the database
  • Auto ran revision d7f52647 and completed successfully .
    nothing changed in the database
  • Auto ran revision d7f52647 and completed successfully .
    nothing changed in the database
  • Auto ran revision d7f52647 and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

bogan / scraper.php