LoveMyData / griffith

Scrapes datracking.griffith.nsw.gov.au

The official Griffith City Council Application Tracker


Griffith City Council scraper

This site is a typical .NET site involves the followings. * Cookie - Yes * Pagnation - No, hardcoded to fetch up to 100 records * Javascript - No * Ajax - Yes * JSON - Yes

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default, today - 1 week) * thismonth * lastmonth

Enjoy

Contributors mlandauer LoveMyData

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.lock; using PHP ^5.5.17  - php (5.6.28)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - apache (2.4.20)  - nginx (1.8.1) -----> Installing dependencies...  Composer version 1.1.3 2016-06-26 15:42:08  Loading composer repositories with package information  Installing dependencies from lock file  - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0)  Cloning e996fe0253bb50330690f5d2bafb66f094dbacb8   Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data between 01/12/2016 and 08/12/2016, changable via MORPH_PERIOD environment PHP Warning: file_get_contents(http://datracking.griffith.nsw.gov.au/Application/GetApplications): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in /app/scraper.php on line 56 PHP Warning: Invalid argument supplied for foreach() in /app/scraper.php on line 61

Data

Downloaded 26 times by LoveMyData

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (66 KB) Use the API

rows 10 / 227

id council_reference address description info_url comment_url date_scraped date_received
1
554/2016.1
68 Zirilli Av, Griffith 2680 NSW
Dwelling with attached garage
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-16
2
553/2016.1
21 Zirilli Av, Griffith 2680 NSW
Dwelling with attached garage
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-24
3
176/2016.1
220 Banna Av, Griffith 2680 NSW
Retail fitout for Australian Unity Client Centre
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-19
4
188/2016.1
62 Hyandra St, Griffith 2680 NSW
Shed
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
5
239/2015.1
20 Barellan St, Griffith 2680 NSW
Carport
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-30
6
179/2016.1
79 Spence Rd, Griffith 2680 NSW
Dwelling with attached garage
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-23
7
190/2016.1
54 Pilloni Rd, Yoogali 2680 NSW
New Shed/Awning
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
8
192/2016.1
38-40 Kidman Wy, Griffith 2680 NSW
Industrial shed for ancillary to existing trucking business
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
9
189/2016.1
7 Gordon Av, Griffith 2680 NSW
Shed
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-31
10
76/2016.1
Railway Pde, Yenda 2681 NSW
Patio
mailto:admin@griffith.nsw.gov.au
2016-09-12
2016-08-30

Statistics

Average successful run time: less than 20 seconds

Total run time: 22 minutes

Total cpu time used: less than 20 seconds

Total disk space used: 454 KB

History

  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    4 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    58 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    60 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    3 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    66 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    5 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    4 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    16 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    7 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    4 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    3 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    14 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    60 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    3 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    7 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    6 records added, 1 record updated in the database
    262 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    7 records added, 1 record updated in the database
    3459 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    4 records added, 1 record updated in the database
    2 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    45 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    11 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    262 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    85 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    3 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    5 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    5 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    153 pages scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    1 record added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    5 records added, 1 record updated in the database
    1 page scraped
  • Auto ran revision 5c63cabc and completed successfully .
    2 records added, 1 record updated in the database
    1 page scraped
  • Manually ran revision 5c63cabc and completed successfully .
    nothing changed in the database
    1 page scraped
  • Manually ran revision 5c63cabc and completed successfully .
    27 records added, 1 record updated in the database
    1 page scraped
  • Manually ran revision 5c63cabc and completed successfully .
    83 records added in the database
    1 page scraped
  • Created on morph.io

Scraper code

PHP

griffith / scraper.php