This repository is a python script to scrape the Austrian lobbying register. The scraper was written for the Gute Taten für gute Daten project from Open Knowledge Austria and is available under the MIT open source license.
This repository provides the code and keeps track of bugs as well as feature requests.
Some information about the Austrian lobbying register:
Types of lobbying organisations
A1 Lobbying-corporations or lobbyists (Lobbying-Unternehmen bzw. Lobbyisten)
A2 Areas of activity of lobbying corporations (not public) (Aufgabenbereiche der Lobbying-Unternehmen (nicht öffentlich))
B Companies or company-/(in-house-)lobbyists (Unternehmen bzw. Unternehmens-/(In-House-)Lobbyisten)
C Self-governing bodies (Selbstverwaltungskörper)
D Interest groups (Interessenverbände)
To run the python script, just enter this in the terminal when you are in the root folder of the repository.
To ease the server, you should download the html files just once and then work locally. To do this, just uncomment in the main section the lines with the
FetchHtmlOrganisations() call and change the ts variable to the name of the directory with the downloaded html-files.
computational chain 1. Fetch the website and store the html locally - pack files after download into tar-ball and delete html-files. 2. Extract facts from html and store it in a json-file 2. Compare actual data with past one data 3. update past one to the new state
In the spirit of free software, everyone is encouraged to help improve this project.
Here are some ways you can contribute:
When you are ready, submit a pull request.
We use the GitHub issue tracker to track bugs and features. Before submitting a bug report or feature request, check to make sure it hasn't already been submitted. When submitting a bug report, please try to provide a screenshot that demonstrates the problem.
This program is free software: you can redistribute it and/or modify it under the terms of the MIT License.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Visit http://opensource.org/licenses/MIT to learn more about the MIT License.
Total run time: less than 20 seconds
Total cpu time used: less than 5 seconds
Total disk space used: 69 KB