njenkins / water_nsw_supplementary_announcements

Water NSW Supplementary announcements advise when additional water is available for licence holders on regulated rivers

Scrapes www.waternsw.com.au

WaterNSW is Australia’s largest water supplier and NSW’s major supplier of raw water. We deliver raw water from our 42 large dams, pipelines and the State’s rivers. We ensure the water we supply is reliable and, where that water is to be used by consumers for drinking, it meets relevant water quality standards.


This is a scraper that runs on Morph. To get started see the documentation

  • PDFs on the web suck

Currently this scraper returns the title, url,year and month/year of publication where it is available. Link / date format is inconsistent, however where at least a month/year is able to be extracted a timestamp value is also generated Maybe someone can build something useful with this information.

Contributors njenkins

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): unspecified  engines.npm (package.json): unspecified (use default)   Resolving node version 6.x...  Downloading and installing node 6.14.1...  Using default npm version: 3.10.10  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@4.0.0 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v48-linux-x64/node_sqlite3.node" is installed via remote  /tmp/build  +-- cheerio@1.0.0-rc.2  | +-- css-select@1.2.0  | | +-- boolbase@1.0.0  | | +-- css-what@2.1.0  | | +-- domutils@1.5.1  | | `-- nth-check@1.0.1  | +-- dom-serializer@0.1.0  | | `-- domelementtype@1.1.3  | +-- entities@1.1.1  | +-- htmlparser2@3.9.2  | | +-- domelementtype@1.3.0  | | +-- domhandler@2.4.1  | | +-- inherits@2.0.3  | | `-- readable-stream@2.3.6  | | +-- core-util-is@1.0.2  | | +-- isarray@1.0.0  | | +-- process-nextick-args@2.0.0  | | +-- string_decoder@1.1.1  | | `-- util-deprecate@1.0.2  | +-- lodash@4.17.10  | `-- parse5@3.0.3  | `-- @types/node@9.6.6  +-- request@2.85.0  | +-- aws-sign2@0.7.0  | +-- aws4@1.7.0  | +-- caseless@0.12.0  | +-- combined-stream@1.0.6  | | `-- delayed-stream@1.0.0  | +-- extend@3.0.1  | +-- forever-agent@0.6.1  | +-- form-data@2.3.2  | | `-- asynckit@0.4.0  | +-- har-validator@5.0.3  | | +-- ajv@5.5.2  | | | +-- co@4.6.0  | | | +-- fast-deep-equal@1.1.0  | | | +-- fast-json-stable-stringify@2.0.0  | | | `-- json-schema-traverse@0.3.1  | | `-- har-schema@2.0.0  | +-- hawk@6.0.2  | | +-- boom@4.3.1  | | +-- cryptiles@3.1.2  | | | `-- boom@5.2.0  | | +-- hoek@4.2.1  | | `-- sntp@2.1.0  | +-- http-signature@1.2.0  | | +-- assert-plus@1.0.0  | | +-- jsprim@1.4.1  | | | +-- extsprintf@1.3.0  | | | +-- json-schema@0.2.3  | | | `-- verror@1.10.0  | | `-- sshpk@1.14.1  | | +-- asn1@0.2.3  | | +-- bcrypt-pbkdf@1.0.1  | | +-- dashdash@1.14.1  | | +-- ecc-jsbn@0.1.1  | | +-- getpass@0.1.7  | | +-- jsbn@0.1.1  | | `-- tweetnacl@0.14.5  | +-- is-typedarray@1.0.0  | +-- isstream@0.1.2  | +-- json-stringify-safe@5.0.1  | +-- mime-types@2.1.18  | | `-- mime-db@1.33.0  | +-- oauth-sign@0.8.2  | +-- performance-now@2.1.0  | +-- qs@6.5.1  | +-- safe-buffer@5.1.2  | +-- stringstream@0.0.5  | +-- tough-cookie@2.3.4  | | `-- punycode@1.4.1  | +-- tunnel-agent@0.6.0  | `-- uuid@3.2.1  `-- sqlite3@4.0.0  +-- nan@2.9.2  `-- node-pre-gyp@0.9.0  +-- detect-libc@1.0.3  +-- mkdirp@0.5.1  | `-- minimist@0.0.8  +-- needle@2.2.0  | +-- debug@2.6.9  | | `-- ms@2.0.0  | +-- iconv-lite@0.4.19  | `-- sax@1.2.4  +-- nopt@4.0.1  | +-- abbrev@1.1.1  | `-- osenv@0.1.5  | +-- os-homedir@1.0.2  | `-- os-tmpdir@1.0.2  +-- npm-packlist@1.1.10  | +-- ignore-walk@3.0.1  | | `-- minimatch@3.0.4  | | `-- brace-expansion@1.1.11  | | +-- balanced-match@1.0.0  | | `-- concat-map@0.0.1  | `-- npm-bundled@1.0.3  +-- npmlog@4.1.2  | +-- are-we-there-yet@1.1.4  | | +-- delegates@1.0.0  | | `-- readable-stream@2.3.5  | | +-- core-util-is@1.0.2  | | +-- isarray@1.0.0  | | +-- process-nextick-args@2.0.0  | | +-- safe-buffer@5.1.1  | | +-- string_decoder@1.0.3  | | `-- util-deprecate@1.0.2  | +-- console-control-strings@1.1.0  | +-- gauge@2.7.4  | | +-- aproba@1.2.0  | | +-- has-unicode@2.0.1  | | +-- object-assign@4.1.1  | | +-- signal-exit@3.0.2  | | +-- string-width@1.0.2  | | | +-- code-point-at@1.1.0  | | | `-- is-fullwidth-code-point@1.0.0  | | | `-- number-is-nan@1.0.1  | | +-- strip-ansi@3.0.1  | | | `-- ansi-regex@2.1.1  | | `-- wide-align@1.1.2  | `-- set-blocking@2.0.0  +-- rc@1.2.6  | +-- deep-extend@0.4.2  | +-- ini@1.3.5  | +-- minimist@1.2.0  | `-- strip-json-comments@2.0.1  +-- rimraf@2.6.2  | `-- glob@7.1.2  | +-- fs.realpath@1.0.0  | +-- inflight@1.0.6  | | `-- wrappy@1.0.2  | +-- inherits@2.0.3  | +-- once@1.4.0  | `-- path-is-absolute@1.0.1  +-- semver@5.5.0  `-- tar@4.4.0  +-- chownr@1.0.1  +-- fs-minipass@1.2.5  +-- minipass@2.2.1  +-- minizlib@1.1.0  `-- yallist@3.0.2   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... 26 Mar 2018 (PDF 318.4 KB) null: null null null: null 21 Jan (PDF 319.8 KB) null: null 27 December 2017 (PDF 435.6 KB) null: null 27 December 2017 (PDF 432.9 KB) null: null

Data

Downloaded 10 times by njenkins MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (259 KB) Use the API

rows 8 / 8

Statistics

Average successful run time: less than a minute

Total run time: about 1 month

Total cpu time used: 12 minutes

Total disk space used: 287 KB

History

  • Auto ran revision 5812db46 and completed successfully .
    2 records updated in the database
    2 pages scraped
  • Auto ran revision 5812db46 and completed successfully .
    3 records updated in the database
    2 pages scraped
  • Auto ran revision 5812db46 and completed successfully .
    3 records updated in the database
    2 pages scraped
  • Auto ran revision 5812db46 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 5812db46 and completed successfully .
    3 records updated in the database
    2 pages scraped
  • ...
  • Created on morph.io

Show complete history