planningalerts-scrapers / town_of_gawler_sa_development_applications

Town of Gawler (South Australia) Development Applications


This is a scraper that runs on Morph. It scrapes lodged development applications from the South Australian Town of Gawler web site.

Contributors MichaelBone

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): 10.6.0  engines.npm (package.json): unspecified (use default)   Resolving node version 10.6.0...  Downloading and installing node 10.6.0...  Using default npm version: 6.1.0  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@4.0.6 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" is installed via remote  added 143 packages from 115 contributors and audited 274 packages in 7.083s  found 0 vulnerabilities   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Retrieving page: https://eservices.gawler.sa.gov.au/eservice/daEnquiryInit.do?doc_typ=4&nodeNum=3228 { StatusCodeError: 502 - "<html>\n <head>\n <title>502 Bad Gateway</title>\n </head>\n <body>\n <h1>502 Bad Gateway</h1>\n <p>Certificate verification error for eservices.gawler.sa.gov.au: unable to get local issuer certificate (errno: 20, depth: 0)</p>\n </body>\n </html>" at new StatusCodeError (/app/node_modules/request-promise-core/lib/errors.js:32:15) at Request.plumbing.callback (/app/node_modules/request-promise-core/lib/plumbing.js:104:33) at Request.RP$callback [as _callback] (/app/node_modules/request-promise-core/lib/plumbing.js:46:31) at Request.self.callback (/app/node_modules/request/request.js:185:22) at Request.emit (events.js:182:13) at Request.<anonymous> (/app/node_modules/request/request.js:1161:10) at Request.emit (events.js:182:13) at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1083:12) at Object.onceWrapper (events.js:273:13) at IncomingMessage.emit (events.js:187:15) name: 'StatusCodeError', statusCode: 502, message: '502 - "<html>\\n <head>\\n <title>502 Bad Gateway</title>\\n </head>\\n <body>\\n <h1>502 Bad Gateway</h1>\\n <p>Certificate verification error for eservices.gawler.sa.gov.au: unable to get local issuer certificate (errno: 20, depth: 0)</p>\\n </body>\\n </html>"', error: '<html>\n <head>\n <title>502 Bad Gateway</title>\n </head>\n <body>\n <h1>502 Bad Gateway</h1>\n <p>Certificate verification error for eservices.gawler.sa.gov.au: unable to get local issuer certificate (errno: 20, depth: 0)</p>\n </body>\n </html>', options: { url: 'https://eservices.gawler.sa.gov.au/eservice/daEnquiryInit.do?doc_typ=4&nodeNum=3228', jar: RequestJar { _jar: [CookieJar] }, rejectUnauthorized: false, callback: [Function: RP$callback], transform: undefined, simple: true, resolveWithFullResponse: false, transform2xxOnly: false }, response: IncomingMessage { _readableState: ReadableState { objectMode: false, highWaterMark: 16384, buffer: BufferList { length: 0 }, length: 0, pipes: null, pipesCount: 0, flowing: true, ended: true, endEmitted: true, reading: false, sync: false, needReadable: false, emittedReadable: false, readableListening: false, resumeScheduled: false, emitClose: true, destroyed: false, defaultEncoding: 'utf8', awaitDrain: 0, readingMore: false, decoder: null, encoding: null }, readable: false, _events: { end: [Array], close: [Array], data: [Function], error: [Function] }, _eventsCount: 4, _maxListeners: undefined, socket: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'eservices.gawler.sa.gov.au', alpnProtocol: false, authorized: false, authorizationError: 'SELF_SIGNED_CERT_IN_CHAIN', encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'eservices.gawler.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 9, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, connection: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'eservices.gawler.sa.gov.au', alpnProtocol: false, authorized: false, authorizationError: 'SELF_SIGNED_CERT_IN_CHAIN', encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'eservices.gawler.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 9, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, headers: { server: 'mitmproxy 4.0.4', connection: 'close', 'content-length': '327', 'content-type': 'text/html' }, rawHeaders: [ 'Server', 'mitmproxy 4.0.4', 'Connection', 'close', 'Content-Length', '327', 'Content-Type', 'text/html' ], trailers: {}, rawTrailers: [], aborted: false, upgrade: false, url: '', method: null, statusCode: 502, statusMessage: 'Bad Gateway', client: TLSSocket { _tlsOptions: [Object], _secureEstablished: true, _securePending: false, _newSessionPending: false, _controlReleased: true, _SNICallback: null, servername: 'eservices.gawler.sa.gov.au', alpnProtocol: false, authorized: false, authorizationError: 'SELF_SIGNED_CERT_IN_CHAIN', encrypted: true, _events: [Object], _eventsCount: 8, connecting: false, _hadError: false, _handle: [TLSWrap], _parent: null, _host: 'eservices.gawler.sa.gov.au', _readableState: [ReadableState], readable: true, _maxListeners: undefined, _writableState: [WritableState], writable: false, allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: undefined, _server: null, ssl: [TLSWrap], _requestCert: true, _rejectUnauthorized: false, parser: null, _httpMessage: [ClientRequest], [Symbol(res)]: [TLSWrap], [Symbol(asyncId)]: 9, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(connect-options)]: [Object] }, _consuming: true, _dumped: false, req: ClientRequest { _events: [Object], _eventsCount: 5, _maxListeners: undefined, output: [], outputEncodings: [], outputCallbacks: [], outputSize: 0, writable: true, _last: true, chunkedEncoding: false, shouldKeepAlive: false, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, socket: [TLSSocket], connection: [TLSSocket], _header: 'GET /eservice/daEnquiryInit.do?doc_typ=4&nodeNum=3228 HTTP/1.1\r\nhost: eservices.gawler.sa.gov.au\r\nConnection: close\r\n\r\n', _onPendingData: [Function: noopPendingOutput], agent: [Agent], socketPath: undefined, timeout: undefined, method: 'GET', path: '/eservice/daEnquiryInit.do?doc_typ=4&nodeNum=3228', _ended: true, res: [Circular], aborted: undefined, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, [Symbol(isCorked)]: false, [Symbol(outHeadersKey)]: [Object] }, request: Request { _events: [Object], _eventsCount: 5, _maxListeners: undefined, rejectUnauthorized: false, readable: true, writable: true, _qs: [Querystring], _auth: [Auth], _oauth: [OAuth], _multipart: [Multipart], _redirect: [Redirect], _tunnel: [Tunnel], _rp_resolve: [Function], _rp_reject: [Function], _rp_promise: [Promise], _rp_callbackOrig: undefined, callback: [Function], _rp_options: [Object], headers: {}, setHeader: [Function], hasHeader: [Function], getHeader: [Function], removeHeader: [Function], method: 'GET', localAddress: undefined, pool: [Object], dests: [], __isRequestRequest: true, _callback: [Function: RP$callback], uri: [Url], proxy: null, tunnel: true, setHost: true, originalCookieHeader: undefined, _jar: [RequestJar], port: 443, host: 'eservices.gawler.sa.gov.au', path: '/eservice/daEnquiryInit.do?doc_typ=4&nodeNum=3228', httpModule: [Object], agentClass: [Function], agent: [Agent], _started: true, href: 'https://eservices.gawler.sa.gov.au/eservice/daEnquiryInit.do?doc_typ=4&nodeNum=3228', req: [ClientRequest], ntick: true, response: [Circular], originalHost: 'eservices.gawler.sa.gov.au', originalHostHeaderName: 'host', responseContent: [Circular], _destdata: true, _ended: true, _callbackCalled: true }, toJSON: [Function: responseToJSON], caseless: Caseless { dict: [Object] }, body: '<html>\n <head>\n <title>502 Bad Gateway</title>\n </head>\n <body>\n <h1>502 Bad Gateway</h1>\n <p>Certificate verification error for eservices.gawler.sa.gov.au: unable to get local issuer certificate (errno: 20, depth: 0)</p>\n </body>\n </html>' } }

Data

Downloaded 247 times by openaustralia jesse-a-reilly Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (116 KB) Use the API

rows 10 / 453

council_reference address description info_url comment_url date_scraped date_received on_notice_from on_notice_to
361/2018
76 Adelaide Road GAWLER SOUTH SA 5118
Verandah (8.4m x 5m x 2.7m)
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-20
374/2018
16 Andreas Avenue EVANSTON GARDENS SA 5116
Verandah (8.3m x 3.2m x 2.6m)
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-27
348/2018
Balmoral Road GAWLER EAST SA 5118
Variation to Previous Application
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-10
376/2018
Balmoral Road GAWLER EAST SA 5118
Detached Dwelling
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-30
382/2018
Balmoral Road GAWLER EAST SA 5118
Detached Dwelling
mailto:council@gawler.sa.gov.au
2018-08-09
2018-08-01
383/2018
Balmoral Road GAWLER EAST SA 5118
Detached Dwelling
mailto:council@gawler.sa.gov.au
2018-08-09
2018-08-01
384/2018
Balmoral Road GAWLER EAST SA 5118
Detached Dwelling
mailto:council@gawler.sa.gov.au
2018-08-09
2018-08-01
362/2018
3 Bartel Court GAWLER EAST SA 5118
Dwelling Additions
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-23
364/2018
12 Blanch Street GAWLER EAST SA 5118
Fence (1.4 metres Max Height)
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-23
353/2018
17 Brown Street WILLASTON SA 5118
Carport (4.07m x 6.1m x 2.7m)
mailto:council@gawler.sa.gov.au
2018-08-09
2018-07-13

Statistics

Average successful run time: less than a minute

Total run time: about 3 hours

Total cpu time used: 6 minutes

Total disk space used: 156 KB

History

  • Auto ran revision 6a7f8e8c and completed successfully .
    nothing changed in the database
  • Auto ran revision 6a7f8e8c and completed successfully .
    nothing changed in the database
  • Auto ran revision 6a7f8e8c and completed successfully .
    nothing changed in the database
  • Auto ran revision 6a7f8e8c and completed successfully .
    nothing changed in the database
  • Auto ran revision 6a7f8e8c and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history