drzax / aec-enrolment-statistics

Gazetted monthly enrolment statistics by state and federal division.

Scrapes www.aec.gov.au and register.redcross.org.au

Australian Electoral Commission


This is a scraper that runs on Morph.

It scrapes monthly gazetted enrollment statistics from the Australian Electoral Commission's website by state and division (electorate).

Good luck!

Contributors drzax

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Node.js app detected  -----> Creating runtime environment   NPM_CONFIG_LOGLEVEL=error  NPM_CONFIG_PRODUCTION=true  NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt  NODE_ENV=production  NODE_TLS_REJECT_UNAUTHORIZED=0  NODE_MODULES_CACHE=true  -----> Installing binaries  engines.node (package.json): unspecified  engines.npm (package.json): unspecified (use default)   Resolving node version (latest stable) via semver.io...  Downloading and installing node 5.11.1...  Using default npm version: 3.8.6  -----> Restoring cache  Skipping cache restore (new runtime signature)  -----> Building dependencies  Installing node modules (package.json)   > sqlite3@3.1.6 install /tmp/build/node_modules/sqlite3  > node-pre-gyp install --fallback-to-build   [sqlite3] Success: "/tmp/build/node_modules/sqlite3/lib/binding/node-v47-linux-x64/node_sqlite3.node" is installed via remote  /tmp/build  +-- cheerio@0.22.0  | +-- css-select@1.2.0  | | +-- boolbase@1.0.0  | | +-- css-what@2.1.0  | | +-- domutils@1.5.1  | | `-- nth-check@1.0.1  | +-- dom-serializer@0.1.0  | | `-- domelementtype@1.1.3  | +-- entities@1.1.1  | +-- htmlparser2@3.9.2  | | +-- domelementtype@1.3.0  | | +-- domhandler@2.3.0  | | +-- inherits@2.0.3  | | `-- readable-stream@2.1.5  | | +-- buffer-shims@1.0.0  | | +-- core-util-is@1.0.2  | | +-- isarray@1.0.0  | | +-- process-nextick-args@1.0.7  | | +-- string_decoder@0.10.31  | | `-- util-deprecate@1.0.2  | +-- lodash.assignin@4.2.0  | +-- lodash.bind@4.2.1  | +-- lodash.defaults@4.2.0  | +-- lodash.filter@4.6.0  | +-- lodash.flatten@4.4.0  | +-- lodash.foreach@4.5.0  | +-- lodash.map@4.6.0  | +-- lodash.merge@4.6.0  | +-- lodash.pick@4.4.0  | +-- lodash.reduce@4.6.0  | +-- lodash.reject@4.6.0  | `-- lodash.some@4.6.0  +-- power-ranger@1.1.0  +-- request@2.75.0  | +-- aws-sign2@0.6.0  | +-- aws4@1.5.0  | +-- bl@1.1.2  | | `-- readable-stream@2.0.6  | +-- caseless@0.11.0  | +-- combined-stream@1.0.5  | | `-- delayed-stream@1.0.0  | +-- extend@3.0.0  | +-- forever-agent@0.6.1  | +-- form-data@2.0.0  | | `-- asynckit@0.4.0  | +-- har-validator@2.0.6  | | +-- chalk@1.1.3  | | | +-- ansi-styles@2.2.1  | | | +-- escape-string-regexp@1.0.5  | | | +-- has-ansi@2.0.0  | | | | `-- ansi-regex@2.0.0  | | | +-- strip-ansi@3.0.1  | | | `-- supports-color@2.0.0  | | +-- commander@2.9.0  | | | `-- graceful-readlink@1.0.1  | | +-- is-my-json-valid@2.15.0  | | | +-- generate-function@2.0.0  | | | +-- generate-object-property@1.2.0  | | | | `-- is-property@1.0.2  | | | +-- jsonpointer@4.0.0  | | | `-- xtend@4.0.1  | | `-- pinkie-promise@2.0.1  | | `-- pinkie@2.0.4  | +-- hawk@3.1.3  | | +-- boom@2.10.1  | | +-- cryptiles@2.0.5  | | +-- hoek@2.16.3  | | `-- sntp@1.0.9  | +-- http-signature@1.1.1  | | +-- assert-plus@0.2.0  | | +-- jsprim@1.3.1  | | | +-- extsprintf@1.0.2  | | | +-- json-schema@0.2.3  | | | `-- verror@1.3.6  | | `-- sshpk@1.10.1  | | +-- asn1@0.2.3  | | +-- assert-plus@1.0.0  | | +-- bcrypt-pbkdf@1.0.0  | | +-- dashdash@1.14.0  | | | `-- assert-plus@1.0.0  | | +-- ecc-jsbn@0.1.1  | | +-- getpass@0.1.6  | | | `-- assert-plus@1.0.0  | | +-- jodid25519@1.0.2  | | +-- jsbn@0.1.0  | | `-- tweetnacl@0.14.3  | +-- is-typedarray@1.0.0  | +-- isstream@0.1.2  | +-- json-stringify-safe@5.0.1  | +-- mime-types@2.1.12  | | `-- mime-db@1.24.0  | +-- node-uuid@1.4.7  | +-- oauth-sign@0.8.2  | +-- qs@6.2.1  | +-- stringstream@0.0.5  | +-- tough-cookie@2.3.1  | `-- tunnel-agent@0.4.3  `-- sqlite3@3.1.6  +-- nan@2.3.5  `-- node-pre-gyp@0.6.30  +-- mkdirp@0.5.1  | `-- minimist@0.0.8  +-- nopt@3.0.6  | `-- abbrev@1.0.9  +-- npmlog@4.0.0  | +-- are-we-there-yet@1.1.2  | | +-- delegates@1.0.0  | | `-- readable-stream@2.1.5  | | +-- buffer-shims@1.0.0  | | +-- core-util-is@1.0.2  | | +-- inherits@2.0.3  | | +-- isarray@1.0.0  | | +-- process-nextick-args@1.0.7  | | +-- string_decoder@0.10.31  | | `-- util-deprecate@1.0.2  | +-- console-control-strings@1.1.0  | +-- gauge@2.6.0  | | +-- aproba@1.0.4  | | +-- has-color@0.1.7  | | +-- has-unicode@2.0.1  | | +-- object-assign@4.1.0  | | +-- signal-exit@3.0.1  | | +-- string-width@1.0.2  | | | +-- code-point-at@1.0.1  | | | | `-- number-is-nan@1.0.1  | | | `-- is-fullwidth-code-point@1.0.0  | | | `-- number-is-nan@1.0.1  | | +-- strip-ansi@3.0.1  | | | `-- ansi-regex@2.0.0  | | `-- wide-align@1.1.0  | `-- set-blocking@2.0.0  +-- rc@1.1.6  | +-- deep-extend@0.4.1  | +-- ini@1.3.4  | +-- minimist@1.2.0  | `-- strip-json-comments@1.0.4  +-- request@2.75.0  | +-- aws-sign2@0.6.0  | +-- aws4@1.4.1  | +-- bl@1.1.2  | | `-- readable-stream@2.0.6  | | +-- core-util-is@1.0.2  | | +-- inherits@2.0.3  | | +-- isarray@1.0.0  | | +-- process-nextick-args@1.0.7  | | +-- string_decoder@0.10.31  | | `-- util-deprecate@1.0.2  | +-- caseless@0.11.0  | +-- combined-stream@1.0.5  | | `-- delayed-stream@1.0.0  | +-- extend@3.0.0  | +-- forever-agent@0.6.1  | +-- form-data@2.0.0  | | `-- asynckit@0.4.0  | +-- har-validator@2.0.6  | | +-- chalk@1.1.3  | | | +-- ansi-styles@2.2.1  | | | +-- escape-string-regexp@1.0.5  | | | +-- has-ansi@2.0.0  | | | | `-- ansi-regex@2.0.0  | | | +-- strip-ansi@3.0.1  | | | | `-- ansi-regex@2.0.0  | | | `-- supports-color@2.0.0  | | +-- commander@2.9.0  | | | `-- graceful-readlink@1.0.1  | | +-- is-my-json-valid@2.15.0  | | | +-- generate-function@2.0.0  | | | +-- generate-object-property@1.2.0  | | | | `-- is-property@1.0.2  | | | +-- jsonpointer@4.0.0  | | | `-- xtend@4.0.1  | | `-- pinkie-promise@2.0.1  | | `-- pinkie@2.0.4  | +-- hawk@3.1.3  | | +-- boom@2.10.1  | | +-- cryptiles@2.0.5  | | +-- hoek@2.16.3  | | `-- sntp@1.0.9  | +-- http-signature@1.1.1  | | +-- assert-plus@0.2.0  | | +-- jsprim@1.3.1  | | | +-- extsprintf@1.0.2  | | | +-- json-schema@0.2.3  | | | `-- verror@1.3.6  | | `-- sshpk@1.10.1  | | +-- asn1@0.2.3  | | +-- assert-plus@1.0.0  | | +-- bcrypt-pbkdf@1.0.0  | | +-- dashdash@1.14.0  | | +-- ecc-jsbn@0.1.1  | | +-- getpass@0.1.6  | | +-- jodid25519@1.0.2  | | +-- jsbn@0.1.0  | | `-- tweetnacl@0.14.3  | +-- is-typedarray@1.0.0  | +-- isstream@0.1.2  | +-- json-stringify-safe@5.0.1  | +-- mime-types@2.1.12  | | `-- mime-db@1.24.0  | +-- node-uuid@1.4.7  | +-- oauth-sign@0.8.2  | +-- qs@6.2.1  | +-- stringstream@0.0.5  | +-- tough-cookie@2.3.1  | `-- tunnel-agent@0.4.3  +-- rimraf@2.5.4  | `-- glob@7.1.0  | +-- fs.realpath@1.0.0  | +-- inflight@1.0.5  | | `-- wrappy@1.0.2  | +-- inherits@2.0.3  | +-- minimatch@3.0.3  | | `-- brace-expansion@1.1.6  | | +-- balanced-match@0.4.2  | | `-- concat-map@0.0.1  | +-- once@1.4.0  | | `-- wrappy@1.0.2  | `-- path-is-absolute@1.0.1  +-- semver@5.3.0  +-- tar@2.2.1  | +-- block-stream@0.0.9  | +-- fstream@1.0.10  | | `-- graceful-fs@4.1.9  | `-- inherits@2.0.3  `-- tar-pack@3.1.4  +-- debug@2.2.0  | `-- ms@0.7.1  +-- fstream@1.0.10  | +-- graceful-fs@4.1.9  | `-- inherits@2.0.3  +-- fstream-ignore@1.0.5  | +-- inherits@2.0.3  | `-- minimatch@3.0.3  | `-- brace-expansion@1.1.6  | +-- balanced-match@0.4.2  | `-- concat-map@0.0.1  +-- once@1.3.3  | `-- wrappy@1.0.2  +-- readable-stream@2.1.5  | +-- buffer-shims@1.0.0  | +-- core-util-is@1.0.2  | +-- inherits@2.0.3  | +-- isarray@1.0.0  | +-- process-nextick-args@1.0.7  | +-- string_decoder@0.10.31  | `-- util-deprecate@1.0.2  `-- uid-number@0.0.6   -----> Caching build  Clearing previous node cache  Saving 2 cacheDirectories (default):  - node_modules  - bower_components (nothing to cache)  -----> Build succeeded!  +-- cheerio@0.22.0  +-- power-ranger@1.1.0  +-- request@2.75.0  `-- sqlite3@3.1.6   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Records exist for 2004-1 Records exist for 2004-2 Records exist for 2004-3 Records exist for 2004-4 Records exist for 2004-5 Records exist for 2004-6 Records exist for 2004-7 Records exist for 2004-8 Fetching: http://www.aec.gov.au/Enrolling_to_vote/Enrolment_stats/gazetted/2004/09.htm Records exist for 2004-10 Records exist for 2004-11 Records exist for 2004-12 Records exist for 2005-1 Records exist for 2005-2 Records exist for 2005-3 Records exist for 2005-4 Records exist for 2005-5 Records exist for 2005-6 Records exist for 2005-7 Records exist for 2005-8 Records exist for 2005-9 Records exist for 2005-10 Records exist for 2005-11 Records exist for 2005-12 Records exist for 2006-1 Records exist for 2006-2 Records exist for 2006-3 Records exist for 2006-4 Records exist for 2006-5 Records exist for 2006-6 Records exist for 2006-7 Records exist for 2006-8 Records exist for 2006-9 Records exist for 2006-10 Records exist for 2006-11 Records exist for 2006-12 Records exist for 2007-1 Records exist for 2007-2 Records exist for 2007-3 Records exist for 2007-4 Records exist for 2007-5 Records exist for 2007-6 Records exist for 2007-7 Records exist for 2007-8 Records exist for 2007-9 Records exist for 2007-10 Records exist for 2007-11 Records exist for 2007-12 Records exist for 2008-1 Records exist for 2008-2 Records exist for 2008-3 Records exist for 2008-4 Records exist for 2008-5 Records exist for 2008-6 Records exist for 2008-7 Records exist for 2008-8 Records exist for 2008-9 Records exist for 2008-10 Records exist for 2008-11 Records exist for 2008-12 Records exist for 2009-1 Records exist for 2009-2 Records exist for 2009-3 Records exist for 2009-4 Records exist for 2009-5 Records exist for 2009-6 Records exist for 2009-7 Records exist for 2009-8 Records exist for 2009-9 Records exist for 2009-10 Records exist for 2009-11 Records exist for 2009-12 Records exist for 2010-1 Records exist for 2010-2 Records exist for 2010-3 Records exist for 2010-4 Records exist for 2010-5 Records exist for 2010-6 Records exist for 2010-7 Records exist for 2010-8 Records exist for 2010-9 Records exist for 2010-10 Records exist for 2010-11 Records exist for 2010-12 Records exist for 2011-1 Records exist for 2011-2 Records exist for 2011-3 Records exist for 2011-4 Records exist for 2011-5 Records exist for 2011-6 Records exist for 2011-7 Records exist for 2011-8 Records exist for 2011-9 Records exist for 2011-10 Records exist for 2011-11 Records exist for 2011-12 Records exist for 2012-1 Records exist for 2012-2 Records exist for 2012-3 Records exist for 2012-4 Records exist for 2012-5 Records exist for 2012-6 Records exist for 2012-7 Records exist for 2012-8 Records exist for 2012-9 Records exist for 2012-10 Records exist for 2012-11 Records exist for 2012-12 Records exist for 2013-1 Records exist for 2013-2 Records exist for 2013-3 Records exist for 2013-4 Records exist for 2013-5 Records exist for 2013-6 Records exist for 2013-7 Records exist for 2013-8 Records exist for 2013-9 Records exist for 2013-10 Records exist for 2013-11 Records exist for 2013-12 Records exist for 2014-1 Records exist for 2014-2 Records exist for 2014-3 Records exist for 2014-4 Records exist for 2014-5 Records exist for 2014-6 Records exist for 2014-7 Records exist for 2014-8 Records exist for 2014-9 Records exist for 2014-10 Records exist for 2014-11 Records exist for 2014-12 Records exist for 2015-1 Records exist for 2015-2 Records exist for 2015-3 Records exist for 2015-4 Records exist for 2015-5 Records exist for 2015-6 Records exist for 2015-7 Records exist for 2015-8 Records exist for 2015-9 Records exist for 2015-10 Records exist for 2015-11 Records exist for 2015-12 Records exist for 2016-1 Records exist for 2016-2 Records exist for 2016-3 Records exist for 2016-4 Fetching: http://www.aec.gov.au/Enrolling_to_vote/Enrolment_stats/gazetted/2016/05.htm Fetching: http://www.aec.gov.au/Enrolling_to_vote/Enrolment_stats/gazetted/2016/06.htm Fetching: http://www.aec.gov.au/Enrolling_to_vote/Enrolment_stats/gazetted/2016/07.htm Fetching: http://www.aec.gov.au/Enrolling_to_vote/Enrolment_stats/gazetted/2016/08.htm Fetching: http://www.aec.gov.au/Enrolling_to_vote/Enrolment_stats/gazetted/2016/09.htm 0 records found for 2004-9 150 records found for 2016-9 150 records found for 2016-6 150 records found for 2016-8 150 records found for 2016-7 150 records found for 2016-5

Data

Downloaded 4 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.39 MB) Use the API

rows 10 / 22201

year month division state enrollment deviation
2004
3
BANKS
nsw
82
-2.76
2004
3
BLAXLAND
nsw
82
-3.55
2004
3
BRADFIELD
nsw
89
4.69
2004
3
CALARE
nsw
87
2.09
2004
3
CHARLTON
nsw
84
-1.26
2004
3
CHIFLEY
nsw
83
-1.85
2004
3
COOK
nsw
81
-4.55
2004
3
COWPER
nsw
83
-2.23
2004
3
BARTON
nsw
83
-1.58
2004
3
DOBELL
nsw
83
-2.06

Statistics

Average successful run time: 2 minutes

Total run time: 6 minutes

Total cpu time used: half a minute

Total disk space used: 1.41 MB

History

  • Manually ran revision 0fb65db4 and completed successfully .
    nothing changed in the database
    6 pages scraped
  • Manually ran revision 0fb65db4 and completed successfully .
    450 records added in the database
  • Manually ran revision 0fb65db4 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Manually ran revision 3793d49b and completed successfully .
    21751 records added in the database
    146 pages scraped
  • Created on morph.io

Scraper code

Node.js

aec-enrolment-statistics / scraper.js