Br3nda / aotearoa_hansard_transcripts

Transcripts from NZ parliament

README.md

aotearoahansardtranscripts

A morph.io scraper for NZ Parliament's per-day Hansard transcript pages, which need a real browser (Radware's bot protection blocks plain HTTP requests) - that's the only reason this is a separate scraper from aotearoa_hansard, which handles the Hansard search API and sitting calendar without needing one.

Data lives on morph.io, not in this repo: https://morph.io/Br3nda/aotearoa_hansard_transcripts

That page has the scraper's run history and a data.sqlite you can query directly (via the page itself, or morph.io's API) - this repo is just the code that produces it.

Downstream, hotair is what actually turns this (and aotearoa_hansard's) raw data into the structured, linked Debates/DebateItems/Members people browse - this scraper only stores raw content, it doesn't parse anything itself.

Status: not built yet - still the default morph.io scraper template. See aotearoa_hansard's PLAN.md for where this fits into the overall plan.

Contributors Br3nda

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... 2: Example Domain

Data

Downloaded 1 time by Br3nda

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (12 KB) Use the API

rows 1 / 1

name
Example Domain

Statistics

Average successful run time: half a minute

Total run time: 2 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 40.2 KB

History

  • Manually ran revision 42935ece and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision 39b60611 and completed successfully .
    1 record added in the database
  • Manually ran revision d495f7c4 and failed .
  • Manually ran revision d495f7c4 and failed .
  • Created on morph.io