planningalerts-scrapers / townsville

Townsville City Council Development Applications

Scrapes eplanning.townsville.qld.gov.au

eplanning @ Townsville


City of Townsville Council scraper

  • Server - .NET - XC.Track
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Clearly defined data within a row - Yes
  • Scrape detail page - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default)
  • thismonth
  • lastmonth

Enjoy

Forked from ScraperWiki

Contributors mlandauer daveslutzkin henare hailspuds otherchirps LoveMyData

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> PHP app detected -----> Bootstrapping... -----> Installing platform packages...  NOTICE: No runtime required in composer.json; requirements  from dependencies in composer.lock will be used for selection  - php (7.2.1)  - ext-gd (bundled with php)  - ext-mbstring (bundled with php)  - ext-pdo_sqlite (bundled with php)  - ext-sqlite3 (bundled with php)  - apache (2.4.29)  - nginx (1.8.1) -----> Installing dependencies...  Composer version 1.5.2 2017-09-11 16:59:25  Loading composer repositories with package information  Installing dependencies from lock file  Package operations: 5 installs, 0 updates, 0 removals  - Installing pguardiario/phpuri (1.0): Downloading (connecting...)Downloading (100%)   - Installing electrolinux/phpquery (0.9.6): Downloading (connecting...)Downloading (100%)   - Installing byjg/pgbrowser (1.0.0): Downloading (connecting...)Downloading (100%)   - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253  - Installing sunra/php-simple-html-dom-parser (v1.5.2): Downloading (connecting...)Downloading (0%) Downloading (15%)Downloading (30%)Downloading (45%)Downloading (50%)Downloading (100%)  Generating optimized autoload files -----> Preparing runtime environment... -----> Checking for additional extensions to install...  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Getting data for `thisweek`, changable via MORPH_PERIOD environment Saving record CAR18/0009 - 9 Halley Street WULGURU QLD 4811 Saving record OP16/0110.13 - 254 Mt Jack Road BROOKHILL QLD 4816 Saving record OP16/0110.14 - 254 Mt Jack Road BROOKHILL QLD 4816 Saving record OP16/0115.04 - 22 Forrest Street ROSSLEA QLD 4812 Saving record OP17/0012.05 - 803 Dalrymple Road SHAW QLD 4818 Saving record POS18/0001 - 9 Tindall Court ALLIGATOR CREEK QLD 4816 Saving record CAR18/0011 - 24 Clifton Street NORTH WARD QLD 4810 Saving record POS18/0002 - 21 Barbeler Street CURRAJONG QLD 4812 Saving record CAR18/0012 - 39 Ninth Avenue RAILWAY ESTATE QLD 4810

Data

Downloaded 2708 times by slav123 openaustralia LoveMyData TheZepto cjrh90 henare peteclowes newmana MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.23 MB) Use the API

rows 10 / 5269

date_scraped description info_url date_received council_reference address comment_url
2011-09-11
Request to Extend the Relevant Period associated with R279.2572
2011-09-09
RC11/0015.02
82 Dunlop Street KELSO QLD 4815
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Request Approval to Allow Construction in 2 Stages associated with R279.2572
2011-09-02
RC11/0015.01
82 Dunlop Street KELSO QLD 4815
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House - Steep slopes Overlay and height Variation
2011-09-01
MC11/0071
29 Braemar Crescent CASTLE HILL QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House (Lot 1 RP 745347 Lease LZ No. 9)
2011-09-01
MI11/0049
7 - 9 Rae Crescent (Private) Cleveland Palms NOME QLD 4816
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House - Steep Lands Overlay
2011-08-30
MC11/0069
348 Stanley Street NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Side Retaining Wall
2011-09-02
OP11/0098
12 Clifton Street NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Congregational Mixed-Use Facility ancillary to existing Educational Establishment and demolition of Four (4) site buildings including a building listed on the Local Heritage Database
2011-08-30
MC11/0070
45 The Strand NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
North Shore Precinct 5 - Operational Works (Sewage Pump Station BU05 Rising Main and Gravity Sewer and Acid Sulphate Soils Test) associated with RC10/0189)
2011-08-30
OP11/0095
0 North Shore Boulevard BURDELL QLD 4818
mailto:eplanning@townsville.qld.gov.au
2011-09-11
North Shore Stage 520 - Operational Works associated with RC10/0189 (23 lots)
2011-09-05
OP11/0099
0 North Shore Boulevard BURDELL QLD 4818
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Vacuum Truck Waste Composting and Soil Conditioning Manufacturing
2011-09-09
MC11/0073
111 Shaw Road SHAW QLD 4818
mailto:eplanning@townsville.qld.gov.au

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.23 MB) Use the API

rows 10 / 3157

id name value_blob type
1
RC12/0052.04
RC12/0052.04
string
2
MC08/0299.11
MC08/0299.11
string
3
MI11/0046.03
MI11/0046.03
string
4
MI08/0077.06
MI08/0077.06
string
5
MC13/0020.01
MC13/0020.01
string
6
RC13/0046.02
RC13/0046.02
string
7
RC12/0010.10
RC12/0010.10
string
8
OP12/0113.11
OP12/0113.11
string
9
OP12/0109.09
OP12/0109.09
string
10
OP12/0104.23
OP12/0104.23
string

Statistics

Average successful run time: half a minute

Total run time: 1 day

Total cpu time used: 6 minutes

Total disk space used: 2.27 MB

History

  • Auto ran revision 910673ac and completed successfully .
    9 records added in the database
    10 pages scraped
  • Auto ran revision 910673ac and failed .
    nothing changed in the database
  • Auto ran revision 910673ac and failed .
    nothing changed in the database
  • Auto ran revision 910673ac and failed .
    nothing changed in the database
  • Auto ran revision 910673ac and completed successfully .
    nothing changed in the database
    1 page scraped
  • ...
  • Forked from ScraperWiki

Show complete history

Scraper code

PHP

townsville / scraper.php