cofiem / australia_parliament_attendance

The attendance record of senators and members for each sitting day.


Australian Parliament Attendance for Senators and Members

The attendance record of senators and members for each sitting day.

This is a scraper that runs on Morph. To get started see the documentation.

Contributors cofiem

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-3.6.2 -----> Installing pip -----> Installing requirements with pip  Collecting lxml==4.6.3  Downloading lxml-4.6.3-cp36-cp36m-manylinux2014_x86_64.whl (6.3 MB)  Collecting requests==2.25.1  Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)  Collecting cssselect==1.1.0  Downloading cssselect-1.1.0-py2.py3-none-any.whl (16 kB)  Collecting dataclasses==0.8  Downloading dataclasses-0.8-py3-none-any.whl (19 kB)  Collecting urllib3<1.27,>=1.21.1  Downloading urllib3-1.26.7-py2.py3-none-any.whl (138 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)  Collecting chardet<5,>=3.0.2  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)  Collecting idna<3,>=2.5  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)  Installing collected packages: urllib3, idna, chardet, certifi, requests, lxml, dataclasses, cssselect  Successfully installed certifi-2021.10.8 chardet-4.0.0 cssselect-1.1.0 dataclasses-0.8 idna-2.10 lxml-4.6.3 requests-2.25.1 urllib3-1.26.7   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Processing page 0. Building attendance url for 2021-10-16 17:16:43.355289 to 2021-10-23 17:16:43.355301 page 0 count 10. Parsed 42 items for House of Representatives sitting on 2021-10-21 for item 0. Added 0 new attendance items of 42 total. Could not determine clerk. skipping 'Senate', '2021-10-21', '' from 'https://parlinfo.aph.gov.au:443/parlInfo/search/display/display.w3p;query=Id%3A%22chamber%2Fjournals%2F2d4f24ed-6160-41fa-8fc3-80e1be9b62bb%2F0048%22'. Parsed 21 items for Senate sitting on 2021-10-21 for item 1. Added 0 new attendance items of 21 total. Could not determine clerk. Could not extract names. No attendance items found for item 2. Added 0 new attendance items of 0 total. Could not determine clerk. skipping 'Senate', '2021-10-20', '' from 'https://parlinfo.aph.gov.au:443/parlInfo/search/display/display.w3p;query=Id%3A%22chamber%2Fjournals%2F127f9314-384b-44ac-9d91-bc3ff62fd84d%2F0035%22'. Parsed 21 items for Senate sitting on 2021-10-20 for item 3. Added 0 new attendance items of 21 total. Parsed 43 items for House of Representatives sitting on 2021-10-20 for item 4. Added 0 new attendance items of 43 total. Could not determine clerk. skipping 'Senate', '2021-10-19', '' from 'https://parlinfo.aph.gov.au:443/parlInfo/search/display/display.w3p;query=Id%3A%22chamber%2Fjournals%2F95d7c5b0-322b-44f5-87e4-6f36361148ea%2F0031%22'. Parsed 21 items for Senate sitting on 2021-10-19 for item 5. Added 0 new attendance items of 21 total. Parsed 45 items for House of Representatives sitting on 2021-10-19 for item 6. Added 0 new attendance items of 45 total. Could not determine clerk. Could not extract names. No attendance items found for item 7. Added 0 new attendance items of 0 total. Could not determine clerk. skipping 'Senate', '2021-10-18', '' from 'https://parlinfo.aph.gov.au:443/parlInfo/search/display/display.w3p;query=Id%3A%22chamber%2Fjournals%2Ffc07ef74-54ed-4fdb-aa66-a0887446dc0d%2F0040%22'. Parsed 21 items for Senate sitting on 2021-10-18 for item 8. Added 0 new attendance items of 21 total. Parsed 45 items for House of Representatives sitting on 2021-10-18 for item 9. Added 0 new attendance items of 45 total. Processing page 1. Building attendance url for 2021-10-16 17:16:43.355289 to 2021-10-23 17:16:43.355301 page 1 count 10. No content for url, stopping. Finished gathering attendance.

Data

Downloaded 1 time by cofiem

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (655 KB) Use the API

rows 10 / 2117

id url assembly sitting_date person_name is_leave retrieved_date
1
House of Representatives
2021-05-13
Ms L. M. Chesters
1
2021-05-16 03:45:43
2
House of Representatives
2021-05-13
Mr Ted O'Brien
0
2021-05-16 03:45:43
3
Senate
2021-05-13
Bilyk
1
2021-05-16 03:45:47
4
Senate
2021-05-13
Gallacher
1
2021-05-16 03:45:47
5
Senate
2021-05-13
Green
1
2021-05-16 03:45:47
6
Senate
2021-05-13
Griff
1
2021-05-16 03:45:47
7
Senate
2021-05-13
Molan
1
2021-05-16 03:45:47
8
Senate
2021-05-13
O'Neill
1
2021-05-16 03:45:47
9
Senate
2021-05-13
Payne
1
2021-05-16 03:45:47
10
Senate
2021-05-13
Scarr
1
2021-05-16 03:45:47

Statistics

Average successful run time: less than a minute

Total run time: about 2 hours

Total cpu time used: 2 minutes

Total disk space used: 8.76 MB

History

  • Auto ran revision 4f2bc86d and completed successfully .
    nothing changed in the database
  • Auto ran revision 4f2bc86d and completed successfully .
    63 records added in the database
  • Auto ran revision 4f2bc86d and completed successfully .
    64 records added in the database
  • Auto ran revision 4f2bc86d and completed successfully .
    66 records added in the database
  • Auto ran revision 4f2bc86d and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history