planningalerts-scrapers / copper_coast_council_sa_development_applications

Copper Coast Council (South Australia) Development Applications


This is a scraper that runs on Morph. It scrapes lodged development applications from the South Australian Copper Coast Council web site.

Contributors MichaelBone mlandauer

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): 10.6.0  engines.npm (package.json): unspecified (use default)   Resolving node version 10.6.0...  Downloading and installing node 10.6.0...  Using default npm version: 6.1.0  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@4.1.0 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" is installed via remote  added 163 packages from 128 contributors and audited 303 packages in 8.9s  found 0 vulnerabilities   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Retrieving page: http://www.coppercoast.sa.gov.au/page.aspx?u=1832 { StatusCodeError: 404 - "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\r\n<html xml:lang=\"en\" lang=\"en\" xmlns=\"http://www.w3.org/1999/xhtml\">\r\n <head>\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css\" />\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/unity/css/global.css\" />\r\n <link rel=\"stylesheet\" media=\"all\" type=\"text/css\" href=\"/unity/css/error.css\" />\r\n <title>Copper Coast Council - Page Not Found</title>\r\n <meta name=\"robots\" content=\"noindex,nofollow\" />\r\n </head>\r\n <body>\r\n <div class=\"main\">\r\n <div class=\"mainBg\">\r\n <div class=\"siteTitle\">Copper Coast Council</div>\r\n <div class=\"mainHeader\">\r\n <div class=\"headerImage\">\r\n <img src=\"/unity/graphics/404.png\" />\r\n </div>\r\n <div class=\"headerTitles\">\r\n <div class=\"pageTitle\">Page Not Found</div>\r\n <div class=\"subTitle\">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class=\"message\">\r\n <p>Sorry, no page or file exists at that address:<br /><a href=\"https://www.coppercoast.sa.gov.au/page.aspx?u=1832\">https://www.coppercoast.sa.gov.au/page.aspx?u=1832</a></p>\r\n </div>\r\n <div class=\"links\">\r\n <div>Go to the <a href=\"/\">Home</a> page.</div>\r\n <div>Go <a href=\"javascript:history.go(-1)\">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>" at new StatusCodeError (/app/node_modules/request-promise-core/lib/errors.js:32:15) at Request.plumbing.callback (/app/node_modules/request-promise-core/lib/plumbing.js:104:33) at Request.RP$callback [as _callback] (/app/node_modules/request-promise-core/lib/plumbing.js:46:31) at Request.self.callback (/app/node_modules/request/request.js:185:22) at Request.emit (events.js:182:13) at Request.<anonymous> (/app/node_modules/request/request.js:1161:10) at Request.emit (events.js:182:13) at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1083:12) at Object.onceWrapper (events.js:273:13) at IncomingMessage.emit (events.js:187:15) name: 'StatusCodeError', statusCode: 404, message: '404 - "<!DOCTYPE html PUBLIC \\"-//W3C//DTD XHTML 1.0 Transitional//EN\\" \\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\\">\\r\\n<html xml:lang=\\"en\\" lang=\\"en\\" xmlns=\\"http://www.w3.org/1999/xhtml\\">\\r\\n <head>\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css\\" />\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/unity/css/global.css\\" />\\r\\n <link rel=\\"stylesheet\\" media=\\"all\\" type=\\"text/css\\" href=\\"/unity/css/error.css\\" />\\r\\n <title>Copper Coast Council - Page Not Found</title>\\r\\n <meta name=\\"robots\\" content=\\"noindex,nofollow\\" />\\r\\n </head>\\r\\n <body>\\r\\n <div class=\\"main\\">\\r\\n <div class=\\"mainBg\\">\\r\\n <div class=\\"siteTitle\\">Copper Coast Council</div>\\r\\n <div class=\\"mainHeader\\">\\r\\n <div class=\\"headerImage\\">\\r\\n <img src=\\"/unity/graphics/404.png\\" />\\r\\n </div>\\r\\n <div class=\\"headerTitles\\">\\r\\n <div class=\\"pageTitle\\">Page Not Found</div>\\r\\n <div class=\\"subTitle\\">HTTP Error 404</div>\\r\\n </div>\\r\\n </div>\\r\\n <div class=\\"message\\">\\r\\n <p>Sorry, no page or file exists at that address:<br /><a href=\\"https://www.coppercoast.sa.gov.au/page.aspx?u=1832\\">https://www.coppercoast.sa.gov.au/page.aspx?u=1832</a></p>\\r\\n </div>\\r\\n <div class=\\"links\\">\\r\\n <div>Go to the <a href=\\"/\\">Home</a> page.</div>\\r\\n <div>Go <a href=\\"javascript:history.go(-1)\\">back</a> to your previous page.</div>\\r\\n </div>\\r\\n </div>\\r\\n </div>\\r\\n </body>\\r\\n</html>"', error: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">\r\n <head>\r\n <link rel="stylesheet" media="all" type="text/css" href="/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/global.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/error.css" />\r\n <title>Copper Coast Council - Page Not Found</title>\r\n <meta name="robots" content="noindex,nofollow" />\r\n </head>\r\n <body>\r\n <div class="main">\r\n <div class="mainBg">\r\n <div class="siteTitle">Copper Coast Council</div>\r\n <div class="mainHeader">\r\n <div class="headerImage">\r\n <img src="/unity/graphics/404.png" />\r\n </div>\r\n <div class="headerTitles">\r\n <div class="pageTitle">Page Not Found</div>\r\n <div class="subTitle">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class="message">\r\n <p>Sorry, no page or file exists at that address:<br /><a href="https://www.coppercoast.sa.gov.au/page.aspx?u=1832">https://www.coppercoast.sa.gov.au/page.aspx?u=1832</a></p>\r\n </div>\r\n <div class="links">\r\n <div>Go to the <a href="/">Home</a> page.</div>\r\n <div>Go <a href="javascript:history.go(-1)">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>', options: { url: 'http://www.coppercoast.sa.gov.au/page.aspx?u=1832', callback: [Function: RP$callback], transform: undefined, simple: true, resolveWithFullResponse: false, transform2xxOnly: false }, response: IncomingMessage { _readableState: ReadableState { objectMode: false, highWaterMark: 16384, buffer: BufferList { length: 0 }, length: 0, pipes: null, pipesCount: 0, flowing: true, ended: true, endEmitted: true, reading: false, sync: true, needReadable: false, emittedReadable: false, readableListening: false, resumeScheduled: false, emitClose: true, destroyed: false, defaultEncoding: 'utf8', awaitDrain: 0, readingMore: true, decoder: null, encoding: null }, readable: false, _events: { end: [Array], close: [Array], data: [Function], error: [Function] }, _eventsCount: 4, _maxListeners: undefined, socket: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.coppercoast.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.coppercoast.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 27, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, connection: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.coppercoast.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.coppercoast.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 27, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, headers: { 'cache-control': 'private', 'content-type': 'text/html; charset=utf-8', server: 'Microsoft-IIS/8.5', 'x-server-name': 'web-01', 'x-xss-protection': '1; mode=block', 'x-frame-origin': 'SAMEORIGIN', date: 'Tue, 15 Oct 2019 02:29:19 GMT', connection: 'close', 'content-length': '1545' }, rawHeaders: [ 'Cache-Control', 'private', 'Content-Type', 'text/html; charset=utf-8', 'Server', 'Microsoft-IIS/8.5', 'X-Server-Name', 'web-01', 'X-XSS-Protection', '1; mode=block', 'X-Frame-Origin', 'SAMEORIGIN', 'Date', 'Tue, 15 Oct 2019 02:29:19 GMT', 'Connection', 'close', 'Content-Length', '1545' ], trailers: {}, rawTrailers: [], aborted: false, upgrade: false, url: '', method: null, statusCode: 404, statusMessage: 'Not Found', client: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'www.coppercoast.sa.gov.au', alpnProtocol: false, authorized: true, authorizationError: null, encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'www.coppercoast.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 27, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, _consuming: false, _dumped: false, req: ClientRequest { _events: [Object], _eventsCount: 5, _maxListeners: undefined, output: [], outputEncodings: [], outputCallbacks: [], outputSize: 0, writable: true, _last: true, chunkedEncoding: false, shouldKeepAlive: false, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, socket: [TLSSocket], connection: [TLSSocket], _header: 'GET /page.aspx?u=1832 HTTP/1.1\r\nreferer: http://www.coppercoast.sa.gov.au/page.aspx?u=1832\r\nhost: www.coppercoast.sa.gov.au\r\nConnection: close\r\n\r\n', _onPendingData: [Function: noopPendingOutput], agent: [Agent], socketPath: undefined, timeout: undefined, method: 'GET', path: '/page.aspx?u=1832', _ended: true, res: [Circular], aborted: undefined, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, [Symbol(isCorked)]: false, [Symbol(outHeadersKey)]: [Object] }, request: Request { _events: [Object], _eventsCount: 5, _maxListeners: undefined, readable: true, writable: true, _qs: [Querystring], _auth: [Auth], _oauth: [OAuth], _multipart: [Multipart], _redirect: [Redirect], _tunnel: [Tunnel], _rp_resolve: [Function], _rp_reject: [Function], _rp_promise: [Promise], _rp_callbackOrig: undefined, callback: [Function], _rp_options: [Object], headers: [Object], setHeader: [Function], hasHeader: [Function], getHeader: [Function], removeHeader: [Function], method: 'GET', localAddress: undefined, pool: {}, dests: [], __isRequestRequest: true, _callback: [Function: RP$callback], uri: [Url], proxy: null, tunnel: true, setHost: true, originalCookieHeader: undefined, _disableCookies: true, _jar: undefined, port: 443, host: 'www.coppercoast.sa.gov.au', path: '/page.aspx?u=1832', httpModule: [Object], agentClass: [Function], href: 'https://www.coppercoast.sa.gov.au/page.aspx?u=1832', ntick: true, response: [Circular], originalHost: 'www.coppercoast.sa.gov.au', originalHostHeaderName: 'host', agent: [Agent], _started: true, req: [ClientRequest], responseContent: [Circular], _destdata: true, _ended: true, _callbackCalled: true }, toJSON: [Function: responseToJSON], caseless: Caseless { dict: [Object] }, body: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\r\n<html xml:lang="en" lang="en" xmlns="http://www.w3.org/1999/xhtml">\r\n <head>\r\n <link rel="stylesheet" media="all" type="text/css" href="/addOns/yui/build/reset-fonts-grids/reset-fonts-grids.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/global.css" />\r\n <link rel="stylesheet" media="all" type="text/css" href="/unity/css/error.css" />\r\n <title>Copper Coast Council - Page Not Found</title>\r\n <meta name="robots" content="noindex,nofollow" />\r\n </head>\r\n <body>\r\n <div class="main">\r\n <div class="mainBg">\r\n <div class="siteTitle">Copper Coast Council</div>\r\n <div class="mainHeader">\r\n <div class="headerImage">\r\n <img src="/unity/graphics/404.png" />\r\n </div>\r\n <div class="headerTitles">\r\n <div class="pageTitle">Page Not Found</div>\r\n <div class="subTitle">HTTP Error 404</div>\r\n </div>\r\n </div>\r\n <div class="message">\r\n <p>Sorry, no page or file exists at that address:<br /><a href="https://www.coppercoast.sa.gov.au/page.aspx?u=1832">https://www.coppercoast.sa.gov.au/page.aspx?u=1832</a></p>\r\n </div>\r\n <div class="links">\r\n <div>Go to the <a href="/">Home</a> page.</div>\r\n <div>Go <a href="javascript:history.go(-1)">back</a> to your previous page.</div>\r\n </div>\r\n </div>\r\n </div>\r\n </body>\r\n</html>' } }

Data

Downloaded 313 times by openaustralia jesse-a-reilly Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.41 MB) Use the API

rows 10 / 3854

council_reference address description info_url comment_url date_scraped date_received
340/124/18
34 KEEN STREET, MOONTA BAY SA 5558
TWO STOREY DWELLING & GARAGE UMR
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-03-20
340/093/18
915 CUNLIFFE ROAD, BOORS PLAIN SA 5554
GARAGE
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-02-27
340/031/18
3 BENNETT COURT, PORT HUGHES SA 5558
CARPORT
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-01-25
340/028/18
19 DEEBLE STREET, EAST MOONTA SA 5558
CARPORT
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-01-23
340/056/18
39 WATERHOUSE STREET, EAST MOONTA SA 5558
SHED EXTENSION
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-02-07
340/025/18
45 STOCKER STREET, MOONTA BAY SA 5558
GARAGE
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-01-19
340/112/18
42 OLIVE PARADE, NEW TOWN SA 5554
DECK
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-03-08
340/081/18
61 BAY ROAD, MOONTA BAY SA 5558
CARPORT & PERGOLA
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-02-21
340/050/18
12 TRELAWNEY STREET, MOONTA BAY SA 5558
CARPORT
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-02-07
340/019/18
53 DAVID STREET, NEW TOWN SA 5554
GARAGE & RAINWATER TANK
mailto:info@coppercoast.sa.gov.au
2018-12-04
2018-01-17

Statistics

Average successful run time: less than a minute

Total run time: about 5 hours

Total cpu time used: 24 minutes

Total disk space used: 1.63 MB

History

  • Auto ran revision cc4cea74 and completed successfully .
    nothing changed in the database
  • Auto ran revision cc4cea74 and completed successfully .
    nothing changed in the database
  • Auto ran revision cc4cea74 and completed successfully .
    nothing changed in the database
  • Auto ran revision cc4cea74 and completed successfully .
    nothing changed in the database
  • Auto ran revision cc4cea74 and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history