keithamoss / lobbyists-registers

Scraper for the federal and state lobbyists registers

Scrapes www.lobbyists.wa.gov.au and www.dpc.sa.gov.au

Register of Lobbyists


An abandoned attempt at writing a morph.io scraper for the various lobbyists registers in Australia.

Abanoned on finding that the wonderful @disclosurelogs had already done the same thing. Code maintained here for posterity.

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Traceback (most recent call last): File "scraper.py", line 7, in <module> for l in sa.lobbyists(): File "/app/registers/sa.py", line 10, in lobbyists for row in soup.find("th", text="LAST UPDATED").find_parent("table").find("tbody").find_all("tr"): AttributeError: 'NoneType' object has no attribute 'find_parent'

Data

Downloaded 1 time by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (219 KB) Use the API

rows 10 / 1012

url business_name location abn last_update trading_name
Gryphon Management Australia
wa
87109077607
07/07/2014
Andrew McNamara
qld
77535221862
14/05/2014
Andrew McNamara
Peter Reith
qld
37219154897
03/07/2014
Peter Reith
23D pty ltd
fed
52129656337
15/07/2014
23D pty ltd
4514 Avenue D Pty Ltd
fed
95159350240
13/07/2014
Hartley Blyth and Associates
Border Management Advisory Services Pty Ltd
fed
79093899462
03/07/2014
Border Management Advisory Services Pty Ltd
Broadcasting & Radio Technology Services
fed
10/07/2014
Broadcasting & Radio Technology Services
BRUNSWICK MEDIA & STRATEGIC PTY. LTD.
fed
36162977246
07/03/2014
BRUNSWICK MEDIA & STRATEGIC PTY. LTD.
Butcher and Co Pty Ltd
fed
19144522038
16/06/2014
Butcher and Co
George Brownbill Consulting Pty Ltd
fed
52009261965
04/07/2014
George Brownbill Consulting Pty Ltd

Statistics

Average successful run time: 4 minutes

Total run time: 1 day

Total cpu time used: 29 minutes

Total disk space used: 252 KB

History

  • Auto ran revision 35ebf346 and failed .
    112 records added, 112 records removed in the database
    3 pages scraped
  • Auto ran revision 35ebf346 and failed .
    112 records added, 112 records removed in the database
    3 pages scraped
  • Auto ran revision 35ebf346 and failed .
    112 records added, 112 records removed in the database
    3 pages scraped
  • Auto ran revision 35ebf346 and failed .
    112 records added, 112 records removed in the database
    3 pages scraped
  • Auto ran revision 35ebf346 and failed .
    112 records added, 110 records removed in the database
    3 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

lobbyists-registers / scraper.py