planningalerts-scrapers / district_council_of_streaky_bay_sa_development_applications

District Council of Streaky Bay (South Australia) Development Applications


This is a scraper that runs on Morph. It scrapes lodged development applications from the South Australian District Council of Streaky Bay web site.

Contributors MichaelBone

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): 10.14.1  engines.npm (package.json): unspecified (use default)   Resolving node version 10.14.1...  Downloading and installing node 10.14.1...  Using default npm version: 6.4.1  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@4.0.6 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" is installed via remote  added 471 packages from 339 contributors and audited 4494 packages in 18.571s  found 0 vulnerabilities   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Retrieving page: https://www.streakybay.sa.gov.au/page.aspx?u=513 Retrieving current year page: https://www.streakybay.sa.gov.au/page.aspx?c=10985&admin=1 { StatusCodeError: 404 - "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\r\n<html xml:lang=\"en\" lang=\"en\" xmlns=\"http://www.w3.org/1999/xhtml\">\r\n <head>\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css\" />\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/unity/css/global.css\" />\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/unity/css/error.css\" />\r\n <title>Streaky Bay District Council - Page Not Found</title>\r\n <meta name=\"robots\" content=\"noindex,nofollow\" />\r\n </head>\r\n <body>\r\n <div class=\"main\">\r\n <div class=\"mainBg\">\r\n <div class=\"siteTitle\">Streaky Bay District Council</div>\r\n <div class=\"mainHeader\">\r\n <div class=\"headerImage\">\r\n <img src=\"/unity/graphics/404.png\" />\r\n </div>\r\n <div class=\"headerTitles\">\r\n <div class=\"pageTitle\">Page Not Found</div>\r\n <div class=\"subTitle\">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class=\"message\">\r\n <p>Sorry, no page or file exists at that address:<br /><a href=\"http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1\">http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1</a></p>\r\n </div>\r\n <div class=\"links\">\r\n <div>Go to the <a href=\"/\">Home</a> page.</div>\r\n <div>Go <a href=\"javascript:history.go(-1)\">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>" at new StatusCodeError (/app/node_modules/request-promise-core/lib/errors.js:32:15) at Request.plumbing.callback (/app/node_modules/request-promise-core/lib/plumbing.js:104:33) at Request.RP$callback [as _callback] (/app/node_modules/request-promise-core/lib/plumbing.js:46:31) at Request.self.callback (/app/node_modules/request/request.js:185:22) at Request.emit (events.js:182:13) at Request.<anonymous> (/app/node_modules/request/request.js:1161:10) at Request.emit (events.js:182:13) at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1083:12) at Object.onceWrapper (events.js:273:13) at IncomingMessage.emit (events.js:187:15) name: 'StatusCodeError', statusCode: 404, message: '404 - "<!DOCTYPE html PUBLIC \\"-//W3C//DTD XHTML 1.0 Transitional//EN\\" \\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\\">\\r\\n<html xml:lang=\\"en\\" lang=\\"en\\" xmlns=\\"http://www.w3.org/1999/xhtml\\">\\r\\n <head>\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css\\" />\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/unity/css/global.css\\" />\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/unity/css/error.css\\" />\\r\\n <title>Streaky Bay District Council - Page Not Found</title>\\r\\n <meta name=\\"robots\\" content=\\"noindex,nofollow\\" />\\r\\n </head>\\r\\n <body>\\r\\n <div class=\\"main\\">\\r\\n <div class=\\"mainBg\\">\\r\\n <div class=\\"siteTitle\\">Streaky Bay District Council</div>\\r\\n <div class=\\"mainHeader\\">\\r\\n <div class=\\"headerImage\\">\\r\\n <img src=\\"/unity/graphics/404.png\\" />\\r\\n </div>\\r\\n <div class=\\"headerTitles\\">\\r\\n <div class=\\"pageTitle\\">Page Not Found</div>\\r\\n <div class=\\"subTitle\\">HTTP Error 404</div>\\r\\n </div>\\r\\n </div>\\r\\n <div class=\\"message\\">\\r\\n <p>Sorry, no page or file exists at that address:<br /><a href=\\"http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1\\">http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1</a></p>\\r\\n </div>\\r\\n <div class=\\"links\\">\\r\\n <div>Go to the <a href=\\"/\\">Home</a> page.</div>\\r\\n <div>Go <a href=\\"javascript:history.go(-1)\\">back</a> to your previous page.</div>\\r\\n </div>\\r\\n </div>\\r\\n </div>\\r\\n </body>\\r\\n</html>"', error: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">\r\n <head>\r\n <link rel="stylesheet" media="all" type="text/css" href="/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/global.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/error.css" />\r\n <title>Streaky Bay District Council - Page Not Found</title>\r\n <meta name="robots" content="noindex,nofollow" />\r\n </head>\r\n <body>\r\n <div class="main">\r\n <div class="mainBg">\r\n <div class="siteTitle">Streaky Bay District Council</div>\r\n <div class="mainHeader">\r\n <div class="headerImage">\r\n <img src="/unity/graphics/404.png" />\r\n </div>\r\n <div class="headerTitles">\r\n <div class="pageTitle">Page Not Found</div>\r\n <div class="subTitle">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class="message">\r\n <p>Sorry, no page or file exists at that address:<br /><a href="http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1">http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1</a></p>\r\n </div>\r\n <div class="links">\r\n <div>Go to the <a href="/">Home</a> page.</div>\r\n <div>Go <a href="javascript:history.go(-1)">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>', options: { url: 'https://www.streakybay.sa.gov.au/page.aspx?c=10985&admin=1', rejectUnauthorized: false, callback: [Function: RP$callback], transform: undefined, simple: true, resolveWithFullResponse: false, transform2xxOnly: false }, response: IncomingMessage { _readableState: ReadableState { objectMode: false, highWaterMark: 16384, buffer: BufferList { head: null, tail: null, length: 0 }, length: 0, pipes: null, pipesCount: 0, flowing: true, ended: true, endEmitted: true, reading: false, sync: true, needReadable: false, emittedReadable: false, readableListening: false, resumeScheduled: false, emitClose: true, destroyed: false, defaultEncoding: 'utf8', awaitDrain: 0, readingMore: true, decoder: null, encoding: null }, readable: false, _events: { end: [Array], close: [Array], data: [Function], error: [Function] }, _eventsCount: 4, _maxListeners: undefined, socket: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.streakybay.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.streakybay.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 35, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, connection: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.streakybay.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.streakybay.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 35, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, headers: { 'cache-control': 'private', 'content-type': 'text/html; charset=utf-8', server: 'Microsoft-IIS/8.5', 'x-server-name': 'web-01', 'x-xss-protection': '1; mode=block', 'x-frame-origin': 'SAMEORIGIN', date: 'Fri, 20 Sep 2019 01:08:27 GMT', connection: 'close', 'content-length': '1583' }, rawHeaders: [ 'Cache-Control', 'private', 'Content-Type', 'text/html; charset=utf-8', 'Server', 'Microsoft-IIS/8.5', 'X-Server-Name', 'web-01', 'X-XSS-Protection', '1; mode=block', 'X-Frame-Origin', 'SAMEORIGIN', 'Date', 'Fri, 20 Sep 2019 01:08:27 GMT', 'Connection', 'close', 'Content-Length', '1583' ], trailers: {}, rawTrailers: [], aborted: false, upgrade: false, url: '', method: null, statusCode: 404, statusMessage: 'Not Found', client: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.streakybay.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.streakybay.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 35, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, _consuming: false, _dumped: false, req: ClientRequest { _events: [Object], _eventsCount: 5, _maxListeners: undefined, output: [], outputEncodings: [], outputCallbacks: [], outputSize: 0, writable: true, _last: true, chunkedEncoding: false, shouldKeepAlive: false, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, socket: [TLSSocket], connection: [TLSSocket], _header: 'GET /page.aspx?c=10985&admin=1 HTTP/1.1\r\nhost: www.streakybay.sa.gov.au\r\nConnection: close\r\n\r\n', _onPendingData: [Function: noopPendingOutput], agent: [Agent], socketPath: undefined, timeout: undefined, method: 'GET', path: '/page.aspx?c=10985&admin=1', _ended: true, res: [Circular], aborted: undefined, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, [Symbol(isCorked)]: false, [Symbol(outHeadersKey)]: [Object] }, request: Request { _events: [Object], _eventsCount: 5, _maxListeners: undefined, rejectUnauthorized: false, readable: true, writable: true, _qs: [Querystring], _auth: [Auth], _oauth: [OAuth], _multipart: [Multipart], _redirect: [Redirect], _tunnel: [Tunnel], _rp_resolve: [Function], _rp_reject: [Function], _rp_promise: [Promise], _rp_callbackOrig: undefined, callback: [Function], _rp_options: [Object], headers: {}, setHeader: [Function], hasHeader: [Function], getHeader: [Function], removeHeader: [Function], method: 'GET', localAddress: undefined, pool: [Object], dests: [], __isRequestRequest: true, _callback: [Function: RP$callback], uri: [Url], proxy: null, tunnel: true, setHost: true, originalCookieHeader: undefined, _disableCookies: true, _jar: undefined, port: 443, host: 'www.streakybay.sa.gov.au', path: '/page.aspx?c=10985&admin=1', httpModule: [Object], agentClass: [Function], agent: [Agent], _started: true, href: 'https://www.streakybay.sa.gov.au/page.aspx?c=10985&admin=1', req: [ClientRequest], ntick: true, response: [Circular], originalHost: 'www.streakybay.sa.gov.au', originalHostHeaderName: 'host', responseContent: [Circular], _destdata: true, _ended: true, _callbackCalled: true }, toJSON: [Function: responseToJSON], caseless: Caseless { dict: [Object] }, body: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">\r\n <head>\r\n <link rel="stylesheet" media="all" type="text/css" href="/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/global.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/error.css" />\r\n <title>Streaky Bay District Council - Page Not Found</title>\r\n <meta name="robots" content="noindex,nofollow" />\r\n </head>\r\n <body>\r\n <div class="main">\r\n <div class="mainBg">\r\n <div class="siteTitle">Streaky Bay District Council</div>\r\n <div class="mainHeader">\r\n <div class="headerImage">\r\n <img src="/unity/graphics/404.png" />\r\n </div>\r\n <div class="headerTitles">\r\n <div class="pageTitle">Page Not Found</div>\r\n <div class="subTitle">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class="message">\r\n <p>Sorry, no page or file exists at that address:<br /><a href="http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1">http://www.streakybay.sa.gov.au/page.aspx?c=10985&amp;admin=1</a></p>\r\n </div>\r\n <div class="links">\r\n <div>Go to the <a href="/">Home</a> page.</div>\r\n <div>Go <a href="javascript:history.go(-1)">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>' } }

Data

Downloaded 134 times by openaustralia MichaelBone

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (68 KB) Use the API

rows 10 / 143

council_reference address description info_url comment_url date_scraped date_received
781/0074/2016
10 ANASTASIA AVENUE, STREAKY BAY SA 5680
Carport and Swimming pool
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-15
2016-12-19
781/0002/2017
13 THE PARADE, SCEALE BAY SA 5680
Domestic outbuilding
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-15
2017-01-19
781/0004/2017
10 SPEED AVENUE, STREAKY BAY SA 5680
Verandah
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-15
2017-02-13
781/0003/2017
18 SEAVIEW ROAD, PERLUBIE SA 5680
Verandah
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-15
2017-02-13
781/0008/2016
Allotment 33 MABIE ROAD, STREAKY BAY SA 5680
Amendment to 781/0039/2015 - Single Storey Detached Dwelling an d implement shed
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-29
2016-02-22
781/0073/2017
Lot 22 KENNEDY ROAD, STREAKY BAY SA 5680
Dwelling
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-30
2017-11-20
781/0006/2018
2363 SCEALE BAY ROAD, SCEALE BAY SA 5680
enclosing lower leve l of pole dwelling
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-30
2018-02-05
781/0019/2018
11 DEOR WAY, EBA ANCHORAGE SA 5680
Domestic outbuilding
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-30
2018-05-02
781/0071/2017
10 Government ROAD, BAIRD BAY SA 5671
Relocating dwelling and new carport
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-30
2017-11-09
781/0018/2018
34 DODGSON DRIVE, STREAKY BAY SA 5680
Domestic outbuilding
mailto:dcstreaky@streakybay.sa.gov.au
2019-05-30
2018-05-02

Statistics

Average successful run time: 1 minute

Total run time: about 3 hours

Total cpu time used: 11 minutes

Total disk space used: 283 KB

History

  • Auto ran revision 2d51b9f5 and completed successfully .
    nothing changed in the database
  • Auto ran revision 2d51b9f5 and completed successfully .
    nothing changed in the database
  • Auto ran revision 2d51b9f5 and completed successfully .
    nothing changed in the database
  • Auto ran revision 2d51b9f5 and completed successfully .
    nothing changed in the database
  • Auto ran revision 2d51b9f5 and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history