ianheggie-oaf / townsville

Townsville City Council Development Applications


City of Townsville Council scraper

  • Server - .NET - XC.Track
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Clearly defined data within a row - Yes
  • Scrape detail page - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default)
  • thismonth
  • lastmonth

Enjoy

Contributors mlandauer LoveMyData henare otherchirps daveslutzkin jamezpolley hailspuds

Last run failed with status code 998.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> PHP app detected [91m-----> [0m [91mBootstrapping... [0m [91m-----> [0m [91mPreparing platform package installation... [0m [91m [1;33mNOTICE:[0m No runtime required in composer.json; requirements from dependencies in composer.lock will be used for selection [0m [91m [1;33mNOTICE:[0m No Composer platform-api-version recorded in composer.lock; file must be very old. Will attempt to use Composer 1 for installation. [0m [91m-----> [0m [91mInstalling platform packages... [0m [91m - php (7.4.33) [0m [91m - ext-sqlite3 (bundled with php) [0m [91m - ext-pdo_sqlite (bundled with php) [0m [91m - ext-mbstring (bundled with php) [0m [91m - ext-gd (bundled with php) [0m [91m - apache (2.4.57) [0m [91m - composer (1.10.26) [0m [91m - nginx (1.24.0) [0m [91m[1;33m ! WARNING: [0m [91mYour selected PHP version has reached end-of-life ! ! No updates or security fixes have been provided for your PHP [0m [91m ! version series by the PHP Group since 2022-11-28. ! ! It is strongly recommended you update your app to a version of ! PHP with "active support" status immediately to ensure you get [0m [91m ! the latest bugfixes and security updates each time you deploy. ! ! You may check the list of versions supported by the PHP Group ! and their EOL dates here: http://php.net/supported-versions.php ! ! For a list of supported runtimes & extensions on Heroku, please [0m [91m ! refer to: https://devcenter.heroku.com/articles/php-support [0m [91m[0m [0m [91m-----> [0m [91mInstalling dependencies... [0m [91m Composer version 1.10.26 2022-04-13 16:39:56 [0m [91m You are using Composer 1 which is deprecated. You should upgrade to Composer 2, see https://blog.packagist.com/deprecating-composer-1-support/ [0m [91m Loading composer repositories with package information [0m [91m Installing dependencies from lock file [0m [91m Package operations: 5 installs, 0 updates, 0 removals [0m [91m - Installing pguardiario/phpuri (1.0): Downloading (100%) [0m [91m - Installing electrolinux/phpquery (0.9.6): Downloading (100%) [0m [91m - Installing byjg/pgbrowser (1.0.0): Downloading (100%) [0m [91m - Installing openaustralia/scraperwiki (dev-morph_defaults e996fe0): Cloning e996fe0253 from cache [0m [91m - Installing torann/dom-parser (2.0.1): Downloading (100%) [0m [91m Generating optimized autoload files [0m [91m-----> [0m [91mPreparing runtime environment... [0m [91m-----> [0m [91mChecking for additional extensions to install... [0m [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Getting data for `thisweek`, changable via MORPH_PERIOD environment PHP Deprecated: Array and string offset access syntax with curly braces is deprecated in /app/vendor/electrolinux/phpquery/phpQuery/phpQuery/phpQueryObject.php on line 1040 PHP Deprecated: Array and string offset access syntax with curly braces is deprecated in /app/vendor/electrolinux/phpquery/phpQuery/phpQuery/phpQueryObject.php on line 1055 Scraper didn't create an SQLite database in your current working directory called data.sqlite. If you've just created your first scraper and not edited the code yet this is to be expected. To fix this make your scraper write to an SQLite database at data.sqlite.

Statistics

Total run time: half a minute

Total cpu time used: less than 5 seconds

Total disk space used: 33.9 KB

History

  • Manually ran revision c1643e85 and failed .
  • Manually ran revision d9dabe9a and failed .
  • Created on morph.io

Scraper code

PHP

townsville / scraper.php