planningalerts-scrapers / district_council_of_tumby_bay_sa_development_applications

District Council of Tumby Bay (South Australia) Development Applications


This is a scraper that runs on Morph. It scrapes lodged development applications from the South Australian District Council of Tumby Bay web site.

Contributors MichaelBone

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): 10.14.1  engines.npm (package.json): unspecified (use default)   Resolving node version 10.14.1...  Downloading and installing node 10.14.1...  Using default npm version: 6.4.1  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@4.0.6 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" is installed via remote  added 474 packages from 339 contributors and audited 4493 packages in 17.12s  found 0 vulnerabilities   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Retrieving page: https://www.tumbybay.sa.gov.au/page.aspx?u=140 { StatusCodeError: 404 - "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\r\n<html xml:lang=\"en\" lang=\"en\" xmlns=\"http://www.w3.org/1999/xhtml\">\r\n <head>\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css\" />\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/unity/css/global.css\" />\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/unity/css/error.css\" />\r\n <title>District Council of Tumby Bay - Page Not Found</title>\r\n <meta name=\"robots\" content=\"noindex,nofollow\" />\r\n </head>\r\n <body>\r\n <div class=\"main\">\r\n <div class=\"mainBg\">\r\n <div class=\"siteTitle\">District Council of Tumby Bay</div>\r\n <div class=\"mainHeader\">\r\n <div class=\"headerImage\">\r\n <img src=\"/unity/graphics/404.png\" />\r\n </div>\r\n <div class=\"headerTitles\">\r\n <div class=\"pageTitle\">Page Not Found</div>\r\n <div class=\"subTitle\">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class=\"message\">\r\n <p>Sorry, no page or file exists at that address:<br /><a href=\"https://www.tumbybay.sa.gov.au/page.aspx?u=140\">https://www.tumbybay.sa.gov.au/page.aspx?u=140</a></p>\r\n </div>\r\n <div class=\"links\">\r\n <div>Go to the <a href=\"/\">Home</a> page.</div>\r\n <div>Go <a href=\"javascript:history.go(-1)\">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>" at new StatusCodeError (/app/node_modules/request-promise-core/lib/errors.js:32:15) at Request.plumbing.callback (/app/node_modules/request-promise-core/lib/plumbing.js:104:33) at Request.RP$callback [as _callback] (/app/node_modules/request-promise-core/lib/plumbing.js:46:31) at Request.self.callback (/app/node_modules/request/request.js:185:22) at Request.emit (events.js:182:13) at Request.<anonymous> (/app/node_modules/request/request.js:1161:10) at Request.emit (events.js:182:13) at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1083:12) at Object.onceWrapper (events.js:273:13) at IncomingMessage.emit (events.js:187:15) name: 'StatusCodeError', statusCode: 404, message: '404 - "<!DOCTYPE html PUBLIC \\"-//W3C//DTD XHTML 1.0 Transitional//EN\\" \\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\\">\\r\\n<html xml:lang=\\"en\\" lang=\\"en\\" xmlns=\\"http://www.w3.org/1999/xhtml\\">\\r\\n <head>\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css\\" />\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/unity/css/global.css\\" />\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/unity/css/error.css\\" />\\r\\n <title>District Council of Tumby Bay - Page Not Found</title>\\r\\n <meta name=\\"robots\\" content=\\"noindex,nofollow\\" />\\r\\n </head>\\r\\n <body>\\r\\n <div class=\\"main\\">\\r\\n <div class=\\"mainBg\\">\\r\\n <div class=\\"siteTitle\\">District Council of Tumby Bay</div>\\r\\n <div class=\\"mainHeader\\">\\r\\n <div class=\\"headerImage\\">\\r\\n <img src=\\"/unity/graphics/404.png\\" />\\r\\n </div>\\r\\n <div class=\\"headerTitles\\">\\r\\n <div class=\\"pageTitle\\">Page Not Found</div>\\r\\n <div class=\\"subTitle\\">HTTP Error 404</div>\\r\\n </div>\\r\\n </div>\\r\\n <div class=\\"message\\">\\r\\n <p>Sorry, no page or file exists at that address:<br /><a href=\\"https://www.tumbybay.sa.gov.au/page.aspx?u=140\\">https://www.tumbybay.sa.gov.au/page.aspx?u=140</a></p>\\r\\n </div>\\r\\n <div class=\\"links\\">\\r\\n <div>Go to the <a href=\\"/\\">Home</a> page.</div>\\r\\n <div>Go <a href=\\"javascript:history.go(-1)\\">back</a> to your previous page.</div>\\r\\n </div>\\r\\n </div>\\r\\n </div>\\r\\n </body>\\r\\n</html>"', error: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">\r\n <head>\r\n <link rel="stylesheet" media="all" type="text/css" href="/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/global.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/error.css" />\r\n <title>District Council of Tumby Bay - Page Not Found</title>\r\n <meta name="robots" content="noindex,nofollow" />\r\n </head>\r\n <body>\r\n <div class="main">\r\n <div class="mainBg">\r\n <div class="siteTitle">District Council of Tumby Bay</div>\r\n <div class="mainHeader">\r\n <div class="headerImage">\r\n <img src="/unity/graphics/404.png" />\r\n </div>\r\n <div class="headerTitles">\r\n <div class="pageTitle">Page Not Found</div>\r\n <div class="subTitle">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class="message">\r\n <p>Sorry, no page or file exists at that address:<br /><a href="https://www.tumbybay.sa.gov.au/page.aspx?u=140">https://www.tumbybay.sa.gov.au/page.aspx?u=140</a></p>\r\n </div>\r\n <div class="links">\r\n <div>Go to the <a href="/">Home</a> page.</div>\r\n <div>Go <a href="javascript:history.go(-1)">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>', options: { url: 'https://www.tumbybay.sa.gov.au/page.aspx?u=140', rejectUnauthorized: false, callback: [Function: RP$callback], transform: undefined, simple: true, resolveWithFullResponse: false, transform2xxOnly: false }, response: IncomingMessage { _readableState: ReadableState { objectMode: false, highWaterMark: 16384, buffer: BufferList { head: null, tail: null, length: 0 }, length: 0, pipes: null, pipesCount: 0, flowing: true, ended: true, endEmitted: true, reading: false, sync: true, needReadable: false, emittedReadable: false, readableListening: false, resumeScheduled: false, emitClose: true, destroyed: false, defaultEncoding: 'utf8', awaitDrain: 0, readingMore: true, decoder: null, encoding: null }, readable: false, _events: { end: [Array], close: [Array], data: [Function], error: [Function] }, _eventsCount: 4, _maxListeners: undefined, socket: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.tumbybay.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.tumbybay.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 9, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, connection: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.tumbybay.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.tumbybay.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 9, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, headers: { 'cache-control': 'private', 'content-type': 'text/html; charset=utf-8', server: 'Microsoft-IIS/8.5', 'x-server-name': 'web-01', 'x-xss-protection': '1; mode=block', 'x-frame-origin': 'SAMEORIGIN', date: 'Tue, 12 Nov 2019 23:20:41 GMT', connection: 'close', 'content-length': '1555' }, rawHeaders: [ 'Cache-Control', 'private', 'Content-Type', 'text/html; charset=utf-8', 'Server', 'Microsoft-IIS/8.5', 'X-Server-Name', 'web-01', 'X-XSS-Protection', '1; mode=block', 'X-Frame-Origin', 'SAMEORIGIN', 'Date', 'Tue, 12 Nov 2019 23:20:41 GMT', 'Connection', 'close', 'Content-Length', '1555' ], trailers: {}, rawTrailers: [], aborted: false, upgrade: false, url: '', method: null, statusCode: 404, statusMessage: 'Not Found', client: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.tumbybay.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.tumbybay.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 9, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, _consuming: false, _dumped: false, req: ClientRequest { _events: [Object], _eventsCount: 5, _maxListeners: undefined, output: [], outputEncodings: [], outputCallbacks: [], outputSize: 0, writable: true, _last: true, chunkedEncoding: false, shouldKeepAlive: false, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, socket: [TLSSocket], connection: [TLSSocket], _header: 'GET /page.aspx?u=140 HTTP/1.1\r\nhost: www.tumbybay.sa.gov.au\r\nConnection: close\r\n\r\n', _onPendingData: [Function: noopPendingOutput], agent: [Agent], socketPath: undefined, timeout: undefined, method: 'GET', path: '/page.aspx?u=140', _ended: true, res: [Circular], aborted: undefined, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, [Symbol(isCorked)]: false, [Symbol(outHeadersKey)]: [Object] }, request: Request { _events: [Object], _eventsCount: 5, _maxListeners: undefined, rejectUnauthorized: false, readable: true, writable: true, _qs: [Querystring], _auth: [Auth], _oauth: [OAuth], _multipart: [Multipart], _redirect: [Redirect], _tunnel: [Tunnel], _rp_resolve: [Function], _rp_reject: [Function], _rp_promise: [Promise], _rp_callbackOrig: undefined, callback: [Function], _rp_options: [Object], headers: {}, setHeader: [Function], hasHeader: [Function], getHeader: [Function], removeHeader: [Function], method: 'GET', localAddress: undefined, pool: [Object], dests: [], __isRequestRequest: true, _callback: [Function: RP$callback], uri: [Url], proxy: null, tunnel: true, setHost: true, originalCookieHeader: undefined, _disableCookies: true, _jar: undefined, port: 443, host: 'www.tumbybay.sa.gov.au', path: '/page.aspx?u=140', httpModule: [Object], agentClass: [Function], agent: [Agent], _started: true, href: 'https://www.tumbybay.sa.gov.au/page.aspx?u=140', req: [ClientRequest], ntick: true, response: [Circular], originalHost: 'www.tumbybay.sa.gov.au', originalHostHeaderName: 'host', responseContent: [Circular], _destdata: true, _ended: true, _callbackCalled: true }, toJSON: [Function: responseToJSON], caseless: Caseless { dict: [Object] }, body: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">\r\n <head>\r\n <link rel="stylesheet" media="all" type="text/css" href="/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/global.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/error.css" />\r\n <title>District Council of Tumby Bay - Page Not Found</title>\r\n <meta name="robots" content="noindex,nofollow" />\r\n </head>\r\n <body>\r\n <div class="main">\r\n <div class="mainBg">\r\n <div class="siteTitle">District Council of Tumby Bay</div>\r\n <div class="mainHeader">\r\n <div class="headerImage">\r\n <img src="/unity/graphics/404.png" />\r\n </div>\r\n <div class="headerTitles">\r\n <div class="pageTitle">Page Not Found</div>\r\n <div class="subTitle">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class="message">\r\n <p>Sorry, no page or file exists at that address:<br /><a href="https://www.tumbybay.sa.gov.au/page.aspx?u=140">https://www.tumbybay.sa.gov.au/page.aspx?u=140</a></p>\r\n </div>\r\n <div class="links">\r\n <div>Go to the <a href="/">Home</a> page.</div>\r\n <div>Go <a href="javascript:history.go(-1)">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>' } }

Data

Downloaded 186 times by openaustralia MichaelBone jesse-a-reilly

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (140 KB) Use the API

rows 10 / 379

council_reference address description info_url comment_url date_scraped date_received
21/2018/64
Sec 313 LINCOLN HIGHWAY, LIPSON SA 5607
New Shed - Rural
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-11-30
21/2018/65
9 SEVENTH STREET, LIPSON SA 5607
New Shed - Residential
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-12-19
21/2018/50
430 THURUNA ROAD, TUMBY BAY SA 5605
New Shed - Rural
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-10-08
21/2018/58
2 178 CHILMANS ROAD, BUTLER SA 5606
New Shed - Rural
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-11-06
21/2018/57
1897 CHILMANS ROAD, BUTLER SA 5606
New Shed - Rural
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-11-01
21/2018/59
19 TUMBY TERRACE, TUMBY BAY SA 5605
New Verandah - Residential
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-11-07
21/2018/60
684 FOOTHILLS ROAD, TUMBY BAY SA 5605
Addition Dwelling - Residential
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-11-19
21/2018/53
24 TREZISE STREET, TUMBY BAY SA 5605
New Dwelling - Residential
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-10-19
21/2018/61
9 LAKIN CRESCENT, TUMBY BAY SA 5605
New Shed - Residential
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-11-19
21/2018/30
2 LAWRIE STREET, TUMBY BAY SA 5605
New Shed - Residential
mailto:dctumby@tumbybay.sa.gov.au
2019-09-19
2018-06-27

Statistics

Average successful run time: 1 minute

Total run time: about 4 hours

Total cpu time used: 39 minutes

Total disk space used: 407 KB

History

  • Auto ran revision 517ce403 and completed successfully .
    nothing changed in the database
  • Auto ran revision 517ce403 and completed successfully .
    nothing changed in the database
  • Auto ran revision 517ce403 and completed successfully .
    nothing changed in the database
  • Auto ran revision 517ce403 and completed successfully .
    nothing changed in the database
  • Auto ran revision 517ce403 and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history