njenkins / water_nsw_supplementary_announcements

Water NSW Supplementary announcements advise when additional water is available for licence holders on regulated rivers


This is a scraper that runs on Morph. To get started see the documentation

  • PDFs on the web suck

Currently this scraper returns the title, url,year and month/year of publication where it is available. Link / date format is inconsistent, however where at least a month/year is able to be extracted a timestamp value is also generated Maybe someone can build something useful with this information.

Contributors njenkins

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_VERBOSE=false  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): unspecified  engines.npm (package.json): unspecified (use default)   Resolving node version 6.x...  Downloading and installing node 6.17.1...  Using default npm version: 3.10.10  -----> Restoring cache  Skipping cache restore (not-found)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@4.1.0 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v48-linux-x64/node_sqlite3.node" is installed via remote  /tmp/build  +-- cheerio@1.0.0-rc.3  | +-- css-select@1.2.0  | | +-- boolbase@1.0.0  | | +-- css-what@2.1.3  | | +-- domutils@1.5.1  | | `-- nth-check@1.0.2  | +-- dom-serializer@0.1.1  | | `-- domelementtype@1.3.1  | +-- entities@1.1.2  | +-- htmlparser2@3.10.1  | | +-- domhandler@2.4.2  | | +-- inherits@2.0.4  | | `-- readable-stream@3.4.0  | | +-- string_decoder@1.3.0  | | `-- util-deprecate@1.0.2  | +-- lodash@4.17.15  | `-- parse5@3.0.3  | `-- @types/node@12.7.5  +-- request@2.88.0  | +-- aws-sign2@0.7.0  | +-- aws4@1.8.0  | +-- caseless@0.12.0  | +-- combined-stream@1.0.8  | | `-- delayed-stream@1.0.0  | +-- extend@3.0.2  | +-- forever-agent@0.6.1  | +-- form-data@2.3.3  | | `-- asynckit@0.4.0  | +-- har-validator@5.1.3  | | +-- ajv@6.10.2  | | | +-- fast-deep-equal@2.0.1  | | | +-- fast-json-stable-stringify@2.0.0  | | | +-- json-schema-traverse@0.4.1  | | | `-- uri-js@4.2.2  | | | `-- punycode@2.1.1  | | `-- har-schema@2.0.0  | +-- http-signature@1.2.0  | | +-- assert-plus@1.0.0  | | +-- jsprim@1.4.1  | | | +-- extsprintf@1.3.0  | | | +-- json-schema@0.2.3  | | | `-- verror@1.10.0  | | | `-- core-util-is@1.0.2  | | `-- sshpk@1.16.1  | | +-- asn1@0.2.4  | | +-- bcrypt-pbkdf@1.0.2  | | +-- dashdash@1.14.1  | | +-- ecc-jsbn@0.1.2  | | +-- getpass@0.1.7  | | +-- jsbn@0.1.1  | | +-- safer-buffer@2.1.2  | | `-- tweetnacl@0.14.5  | +-- is-typedarray@1.0.0  | +-- isstream@0.1.2  | +-- json-stringify-safe@5.0.1  | +-- mime-types@2.1.24  | | `-- mime-db@1.40.0  | +-- oauth-sign@0.9.0  | +-- performance-now@2.1.0  | +-- qs@6.5.2  | +-- safe-buffer@5.2.0  | +-- tough-cookie@2.4.3  | | +-- psl@1.4.0  | | `-- punycode@1.4.1  | +-- tunnel-agent@0.6.0  | `-- uuid@3.3.3  `-- sqlite3@4.1.0  +-- nan@2.14.0  `-- node-pre-gyp@0.11.0  +-- detect-libc@1.0.3  +-- mkdirp@0.5.1  | `-- minimist@0.0.8  +-- needle@2.4.0  | +-- debug@3.2.6  | | `-- ms@2.1.2  | +-- iconv-lite@0.4.24  | `-- sax@1.2.4  +-- nopt@4.0.1  | +-- abbrev@1.1.1  | `-- osenv@0.1.5  | +-- os-homedir@1.0.2  | `-- os-tmpdir@1.0.2  +-- npm-packlist@1.4.4  | +-- ignore-walk@3.0.2  | | `-- minimatch@3.0.4  | | `-- brace-expansion@1.1.11  | | +-- balanced-match@1.0.0  | | `-- concat-map@0.0.1  | `-- npm-bundled@1.0.6  +-- npmlog@4.1.2  | +-- are-we-there-yet@1.1.5  | | +-- delegates@1.0.0  | | `-- readable-stream@2.3.6  | | +-- isarray@1.0.0  | | +-- process-nextick-args@2.0.1  | | +-- safe-buffer@5.1.2  | | `-- string_decoder@1.1.1  | +-- console-control-strings@1.1.0  | +-- gauge@2.7.4  | | +-- aproba@1.2.0  | | +-- has-unicode@2.0.1  | | +-- object-assign@4.1.1  | | +-- signal-exit@3.0.2  | | +-- string-width@1.0.2  | | | +-- code-point-at@1.1.0  | | | `-- is-fullwidth-code-point@1.0.0  | | | `-- number-is-nan@1.0.1  | | +-- strip-ansi@3.0.1  | | | `-- ansi-regex@2.1.1  | | `-- wide-align@1.1.3  | `-- set-blocking@2.0.0  +-- rc@1.2.8  | +-- deep-extend@0.6.0  | +-- ini@1.3.5  | +-- minimist@1.2.0  | `-- strip-json-comments@2.0.1  +-- rimraf@2.7.1  | `-- glob@7.1.4  | +-- fs.realpath@1.0.0  | +-- inflight@1.0.6  | | `-- wrappy@1.0.2  | +-- once@1.4.0  | `-- path-is-absolute@1.0.1  +-- semver@5.7.1  `-- tar@4.4.11  +-- chownr@1.1.2  +-- fs-minipass@1.2.7  +-- minipass@2.6.5  +-- minizlib@1.2.2  `-- yallist@3.0.3   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... 17 May 2019 (PDF 113.9 KB) 2019: 1558051200000 null 2019: null null 2019: null null 2019: null null 2019: null 7 February 2019.pdf (PDF 80.4 KB) 2019: null 5 February 2019.pdf (PDF 80.4 KB) 2019: null null 2019: null null 2019: null 9 April 2019 (PDF 110.5 KB) 2019: 1554768000000

Data

Downloaded 10 times by njenkins MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (259 KB) Use the API

rows 10 / 10

title url year daymonth timestamp
Bega Brogo supplementary access
2019
17 May 2019 (PDF 113.9 KB)
1558051200000
Hunter Supplementary access 31 March 2019.pdf (PDF 86.6 KB)
2019
Bega Brogo Supplemnetary access 19 March 2019 AA.pdf (PDF 70.6 KB)
2019
Bega Brogo Supplemnetary access 27 February 2019 AA.pdf (PDF 70.3 KB)
2019
Bega Brogo Supplementary access 12 February 2019.pdf (PDF 98.5 KB)
2019
Bega Brogo supplementary access
2019
7 February 2019.pdf (PDF 80.4 KB)
Bega Brogo supplementary access
2019
5 February 2019.pdf (PDF 80.4 KB)
Bega Brogo Supplementary access 2 May 2019 (PDF 70.4 KB)
2019
Bega Brogo Supplementary access 23 April 2019 (PDF 70.6 KB)
2019
Hunter supplementary access
2019
9 April 2019 (PDF 110.5 KB)
1554768000000

Statistics

Average successful run time: less than a minute

Total run time: about 1 month

Total cpu time used: 22 minutes

Total disk space used: 287 KB

History

  • Auto ran revision 5812db46 and completed successfully .
    3 records updated in the database
  • Auto ran revision 5812db46 and completed successfully .
    3 records updated in the database
  • Auto ran revision 5812db46 and completed successfully .
    5 records updated in the database
  • Auto ran revision 5812db46 and completed successfully .
    2 records updated in the database
  • Auto ran revision 5812db46 and completed successfully .
    3 records updated in the database
  • ...
  • Created on morph.io

Show complete history