LoveMyData / griffith

Scrapes datracking.griffith.nsw.gov.au

The official Griffith City Council Application Tracker


Griffith City Council scraper

This site is a typical .NET site involves the followings. * Cookie - Yes * Pagnation - No, hardcoded to fetch up to 100 records * Javascript - No * Ajax - Yes * JSON - Yes

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default, today - 10 days) * thismonth * lastmonth * custom = yyyy-mm ( eg. 2017-01 )

Enjoy

Contributors mlandauer LoveMyData

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.json; requirements  from dependencies in composer.lock will be used for selection  - php (7.2.4)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - nginx (1.8.1)  - apache (2.4.33) -----> Installing dependencies...  Composer version 1.5.2 2017-09-11 16:59:25  Loading composer repositories with package information  Installing dependencies from lock file  Package operations: 5 installs, 0 updates, 0 removals  - Installing pguardiario/phpuri (1.0): Downloading (connecting...)Downloading (100%)   - Installing electrolinux/phpquery (0.9.6): Downloading (connecting...)Downloading (100%)   - Installing byjg/pgbrowser (1.0.0): Downloading (connecting...)Downloading (100%)   - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253  - Installing sunra/php-simple-html-dom-parser (v1.5.2): Downloading (connecting...)Downloading (100%)   Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data between 13/04/2018 and 23/04/2018, changable via MORPH_PERIOD environment Skipping already saved record 519/2018.1 Skipping already saved record 26/2018.2 Skipping already saved record 81/2018.1 Skipping already saved record 80/2018.1 Skipping already saved record 302/2016.1 Skipping already saved record 82/2018.1 Skipping already saved record 76/2018.1 Skipping already saved record 249/2015.2 Skipping already saved record 536/2017.1 Skipping already saved record 79/2018.1 Skipping already saved record 24/2018.1 Skipping already saved record 77/2018.1 Skipping already saved record 78/2018.1 Skipping already saved record 525/2017.1

Data

Downloaded 498 times by LoveMyData henare

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (286 KB) Use the API

rows 10 / 994

id council_reference address description info_url comment_url date_scraped date_received
1
554/2016.1
68 Zirilli Av, Griffith 2680 NSW
Dwelling with attached garage
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-16
2
553/2016.1
21 Zirilli Av, Griffith 2680 NSW
Dwelling with attached garage
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-24
3
176/2016.1
220 Banna Av, Griffith 2680 NSW
Retail fitout for Australian Unity Client Centre
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-19
4
188/2016.1
62 Hyandra St, Griffith 2680 NSW
Shed
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
5
239/2015.1
20 Barellan St, Griffith 2680 NSW
Carport
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-30
6
179/2016.1
79 Spence Rd, Griffith 2680 NSW
Dwelling with attached garage
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-23
7
190/2016.1
54 Pilloni Rd, Yoogali 2680 NSW
New Shed/Awning
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
8
192/2016.1
38-40 Kidman Wy, Griffith 2680 NSW
Industrial shed for ancillary to existing trucking business
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
9
189/2016.1
7 Gordon Av, Griffith 2680 NSW
Shed
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
10
76/2016.1
Railway Pde, Yenda 2681 NSW
Patio
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-30

Statistics

Average successful run time: half a minute

Total run time: about 4 hours

Total cpu time used: 2 minutes

Total disk space used: 392 KB

History

  • Auto ran revision 900ebcfa and completed successfully .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 900ebcfa and completed successfully .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 900ebcfa and completed successfully .
    nothing changed in the database
    5 pages scraped
  • Auto ran revision 900ebcfa and completed successfully .
    1 record added, 1 record updated in the database
    5 pages scraped
  • Auto ran revision 900ebcfa and completed successfully .
    7 records added, 1 record updated in the database
    5 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

griffith / scraper.php