LoveMyData / maitland

Maitland Development Applications


Maitland City Council Scraper

This council involves the followings * Server - nginx/1.8.0 * Cookie tracking - Yes * Pagnation - No - However, modified query to pull max 500 records * XML - Yes * Horizon system

Setup MORPH_PERIOD for data recovery, available options are * thisweek (default, today -7 days) * thismonth * lastmonth

Enjoy

Contributors CloCkWeRX LoveMyData equivalentideas

The scraper is queued to run. It was queued automatically .

Console output of last run

Data

Downloaded 57 times by LoveMyData

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (208 KB) Use the API

rows 10 / 678

council_reference address description info_url comment_url date_scraped date_received
CCP-2016-2159
5 Lorn Street LORN 2320
Pool
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2158
7 Justine Parade RUTHERFORD 2320
Tree Application
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2157
56 Kyle Street RUTHERFORD 2320
Recreation Facility
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CC-2016-2156
23 McKeachie Drive ABERGLASSLYN 2320
Garage
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2155
23 McKeachie Drive ABERGLASSLYN 2320
Garage
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CC-2016-2154
24 Rivergum Drive ABERGLASSLYN 2320
Dwelling House
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2153
24 Rivergum Drive ABERGLASSLYN 2320
Dwelling House
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CDP-2016-2152
52 Hillcrest Drive GILLIESTON HEIGHTS 2321
Dwelling House
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CCP-2016-2151
14 Mitchell Drive EAST MAITLAND 2323
Shop
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CDP-2016-2150
28 Queen Street RUTHERFORD 2320
Dwelling alteration/addition
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15

Statistics

Average successful run time: less than a minute

Total run time: about 1 hour

Total cpu time used: 2 minutes

Total disk space used: 256 KB

History

  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    19 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    24 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    9 records added in the database
    76 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    28 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    18 records added in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision 253daa56 and completed successfully .
    24 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    54 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    8 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    6 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    4 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    15 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    4 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    3 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    17 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    7 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    17 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    8 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    9 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    12 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    5 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    6 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    9 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    14 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    4 records added in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
  • Auto ran revision 253daa56 and completed successfully .
    18 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    7 records added in the database
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    1 record added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    5 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    6 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    9 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    15 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    26 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    165 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    4 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    13 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    6 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    14 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    6 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    9 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    13 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    25 records added in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 253daa56 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    19 records added in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    11 records added in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    12 records added in the database
    2 pages scraped
  • Auto ran revision a0d56316 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision a0d56316 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision a0d56316 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision dc50afbc and completed successfully .
    11 records added in the database
    2 pages scraped
  • Auto ran revision dc50afbc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision dc50afbc and completed successfully .
    15 records added in the database
    2 pages scraped
  • Auto ran revision dc50afbc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision dc50afbc and completed successfully .
    76 records added in the database
    68 pages scraped
  • Manually ran revision dc50afbc and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision dc50afbc and completed successfully .
    41 records added in the database
    2 pages scraped
  • Manually ran revision 22a5475a and failed .
    nothing changed in the database
    2 pages scraped
  • Created on morph.io

Scraper code

Ruby

maitland / scraper.rb