stefanw / bgbl

Bundesgesetzblatt Table of Contents

Scrapes www.bgbl.de

Zugriff auf das komplette Online-Archiv des Bundesgesetzblattes (BGBl.) Teil I und Teil II sowie auf die Fundstellennachweise A (FNA) und Fundstellennachweise B (FNB)


This is a scraper that runs on Morph. To get started see the documentation

Contributors stefanw

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-3.6.2 -----> Installing pip -----> Installing requirements with pip  Collecting lxml (from -r /tmp/build/requirements.txt (line 1))  Downloading https://files.pythonhosted.org/packages/a7/b9/ccf46cea0f698b40bca2a9c1a44039c336fe1988b82de4f7353be7a8396a/lxml-4.2.1-cp36-cp36m-manylinux1_x86_64.whl (5.6MB)  Collecting cssselect (from -r /tmp/build/requirements.txt (line 2))  Downloading https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl  Collecting requests (from -r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)  Collecting dataset (from -r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/ba/76/5b8fe80f487bfa90b8d0094c3a27150fe797b1fbe02dec31b048fd52448d/dataset-1.1.0-py2.py3-none-any.whl  Collecting chardet<3.1.0,>=3.0.2 (from requests->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)  Collecting urllib3<1.23,>=1.21.1 (from requests->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)  Collecting idna<2.7,>=2.5 (from requests->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)  Collecting certifi>=2017.4.17 (from requests->-r /tmp/build/requirements.txt (line 3))  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)  Collecting normality>=0.5.1 (from dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/9e/b9/75da5b8c929a52f6d1b5f9104a2d126ed3d6352ba101682468c249a389e3/normality-0.6.1-py2.py3-none-any.whl  Collecting six>=1.11.0 (from dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl  Collecting alembic>=0.6.2 (from dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/89/03/756d5b8e1c90bf283c3f435766aa3f20208d1c3887579dd8f2122e01d5f4/alembic-0.9.9.tar.gz (1.0MB)  Collecting sqlalchemy>=1.1.0 (from dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/b4/9c/411a9bac1a471bed54ec447dc183aeed12a75c1b648307e18b56e3829363/SQLAlchemy-1.2.8.tar.gz (5.6MB)  Collecting banal (from normality>=0.5.1->dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/20/05/d3df93ba3fcfcd8716e1e3e3947b191be9adae0b3f9c01d802d32d636930/banal-0.3.7-py2.py3-none-any.whl  Collecting Mako (from alembic>=0.6.2->dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz (564kB)  Collecting python-editor>=0.3 (from alembic>=0.6.2->dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/65/1e/adf6e000ea5dc909aa420352d6ba37f16434c8a3c2fa030445411a1ed545/python-editor-1.0.3.tar.gz  Collecting python-dateutil (from alembic>=0.6.2->dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB)  Collecting MarkupSafe>=0.9.2 (from Mako->alembic>=0.6.2->dataset->-r /tmp/build/requirements.txt (line 4))  Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz  Installing collected packages: lxml, cssselect, chardet, urllib3, idna, certifi, requests, six, banal, normality, sqlalchemy, MarkupSafe, Mako, python-editor, python-dateutil, alembic, dataset  Running setup.py install for sqlalchemy: started  Running setup.py install for sqlalchemy: finished with status 'done'  Running setup.py install for MarkupSafe: started  Running setup.py install for MarkupSafe: finished with status 'done'  Running setup.py install for Mako: started  Running setup.py install for Mako: finished with status 'done'  Running setup.py install for python-editor: started  Running setup.py install for python-editor: finished with status 'done'  Running setup.py install for alembic: started  Running setup.py install for alembic: finished with status 'done'  Successfully installed Mako-1.0.7 MarkupSafe-1.0 alembic-0.9.9 banal-0.3.7 certifi-2018.4.16 chardet-3.0.4 cssselect-1.0.3 dataset-1.1.0 idna-2.6 lxml-4.2.1 normality-0.6.1 python-dateutil-2.7.3 python-editor-1.0.3 requests-2.18.4 six-1.11.0 sqlalchemy-1.2.8 urllib3-1.22   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... 1 Getting Year 2018 for 1 Getting Year 2017 for 1 Getting Year 2016 for 1 Getting Year 2015 for 1 Getting Year 2014 for 1 Getting Year 2013 for 1 Getting Year 2012 for 1 Getting Year 2011 for 1 1 Getting Year 2018 for 1 Getting Year 2017 for 1 Getting Year 2016 for 1 Getting Year 2015 for 1 Getting Year 2014 for 1 Getting Year 2013 for 1 Getting Year 2012 for 1 Getting Year 2011 for 1 Getting Year 2010 for 1 Getting Year 2009 for 1 1 Getting Year 2018 for 1 Getting Year 2017 for 1 Getting Year 2016 for 1 Getting Year 2015 for 1 Getting Year 2014 for 1 Getting Year 2013 for 1 Getting Year 2012 for 1 Getting Year 2011 for 1 Getting Year 2010 for 1 Getting Year 2009 for 1 Getting Year 2008 for 1 Getting Year 2007 for 1 Getting Year 2006 for 1 Getting Year 2005 for 1 Getting Year 2004 for 1 1 Getting Year 2018 for 1 Getting Year 2017 for 1 Getting Year 2016 for 1 Getting Year 2015 for 1 Getting Year 2014 for 1 Getting Year 2013 for 1 Getting Year 2012 for 1 Getting Year 2011 for 1 Getting Year 2010 for 1 Getting Year 2009 for 1 Getting Year 2008 for 1 Getting Year 2007 for 1 Getting Year 2006 for 1 Getting Year 2005 for 1 Getting Year 2004 for 1 Getting Year 2003 for 1 Getting Year 2002 for 1 Getting Year 2001 for 1 Getting Year 2000 for 1 Getting Year 1999 for 1 Getting Year 1998 for 1 Getting Year 1997 for 1 Getting Year 1996 for 1 Getting Year 1995 for 1 Getting Year 1994 for 1 Getting Year 1993 for 1 Getting Year 1992 for 1 Getting Year 1991 for 1 Getting Year 1990 for 1 Getting Year 1989 for 1 Getting Year 1988 for 1 Getting Year 1987 for 1 Getting Year 1986 for 1 Getting Year 1985 for 1 Getting Year 1984 for 1 Getting Year 1983 for 1 Getting Year 1982 for 1 Getting Year 1981 for 1 Getting Year 1980 for 1 Getting Year 1979 for 1 Getting Year 1978 for 1 Getting Year 1977 for 1 Getting Year 1976 for 1 Getting Year 1975 for 1 Getting Year 1974 for 1 Getting Year 1973 for 1 Getting Year 1972 for 1 Getting Year 1971 for 1 Getting Year 1970 for 1 Getting Year 1969 for 1 Getting Year 1968 for 1 Getting Year 1967 for 1 Getting Year 1966 for 1 Getting Year 1965 for 1 Getting Year 1964 for 1 Getting Year 1963 for 1 Getting Year 1962 for 1 Getting Year 1961 for 1 Getting Year 2001 for 1 Getting Year 2000 for 1 Getting Year 1999 for 1 Getting Year 1998 for 1 Getting Year 1997 for 1 Getting Year 1996 for 1 Getting Year 1995 for 1 Getting Year 1994 for 1 Getting Year 1993 for 1 Getting Year 1992 for 1 Getting Year 1991 for 1 Getting Year 1990 for 1 Getting Year 1989 for 1 Getting Year 1988 for 1 Getting Year 1987 for 1 Getting Year 1986 for 1 Getting Year 1985 for 1 Getting Year 1984 for 1 Getting Year 1983 for 1 Getting Year 1982 for 1 Getting Year 1981 for 1 Getting Year 1980 for 1 Getting Year 1979 for 1 Getting Year 1978 for 1 Getting Year 1977 for 1 Getting Year 1976 for 1 Getting Year 1975 for 1 Getting Year 1974 for 1 Getting Year 1973 for 1 Getting Year 1972 for 1 Getting Year 1971 for 1 Getting Year 1970 for 1 Getting Year 1969 for 1 Getting Year 1968 for 1 Getting Year 1967 for 1 Getting Year 1966 for 1 Getting Year 1965 for 1 Getting Year 1964 for 1 Getting Year 1963 for 1 Getting Year 1962 for 1 Getting Year 1961 for 1 Getting Year 1960 for 1 Getting Year 1959 for 1 Getting Year 1958 for 1 Getting Year 1957 for 1 Getting Year 1956 for 1 Getting Year 1955 for 1 Getting Year 1954 for 1 Getting Year 1953 for 1 Getting Year 1952 for 1 Getting Year 1951 for 1 Getting Year 1950 for 1 Getting Year 1949 for 1 2 Getting Year 2018 for 2 Getting Year 2017 for 2 Getting Year 2016 for 2 Getting Year 2015 for 2 Getting Year 2014 for 2 Getting Year 2013 for 2 Getting Year 2012 for 2 Getting Year 2011 for 2 Getting Year 2010 for 2 Getting Year 2009 for 2 Getting Year 2008 for 2 Getting Year 2007 for 2 Getting Year 2006 for 2 Getting Year 2005 for 2 Getting Year 2004 for 2 Getting Year 2003 for 2 Getting Year 2002 for 2 Getting Year 2001 for 2 Getting Year 2000 for 2 Getting Year 1999 for 2 Getting Year 1998 for 2 Getting Year 1997 for 2 Getting Year 1996 for 2 Getting Year 1995 for 2 Getting Year 1994 for 2 Getting Year 1993 for 2 Getting Year 1992 for 2 Getting Year 1991 for 2 Getting Year 1990 for 2 Getting Year 1989 for 2 Getting Year 1988 for 2 Getting Year 1987 for 2 Getting Year 1986 for 2 Getting Year 1985 for 2 Getting Year 1984 for 2 Getting Year 1983 for 2 Getting Year 1982 for 2 Getting Year 2001 for 1 Getting Year 2000 for 1 Getting Year 1999 for 1 Getting Year 1998 for 1 Getting Year 1997 for 1 Getting Year 1996 for 1 Getting Year 1995 for 1 Getting Year 1994 for 1 Getting Year 1993 for 1 Getting Year 1992 for 1 Getting Year 1991 for 1 Getting Year 1990 for 1 Getting Year 1989 for 1 Getting Year 1988 for 1 Getting Year 1987 for 1 Getting Year 1986 for 1 Getting Year 1985 for 1 Getting Year 1984 for 1 Getting Year 1983 for 1 Getting Year 1982 for 1 Getting Year 1981 for 1 Getting Year 1980 for 1 Getting Year 1979 for 1 Getting Year 1978 for 1 Getting Year 1977 for 1 Getting Year 1976 for 1 Getting Year 1975 for 1 Getting Year 1974 for 1 Getting Year 1973 for 1 Getting Year 1972 for 1 Getting Year 1971 for 1 Getting Year 1970 for 1 Getting Year 1969 for 1 Getting Year 1968 for 1 Getting Year 1967 for 1 Getting Year 1966 for 1 Getting Year 1965 for 1 Getting Year 1964 for 1 Getting Year 1963 for 1 Getting Year 1962 for 1 Getting Year 1961 for 1 Getting Year 1960 for 1 Getting Year 1959 for 1 Getting Year 1958 for 1 Getting Year 1957 for 1 Getting Year 1956 for 1 Getting Year 1955 for 1 Getting Year 1954 for 1 Getting Year 1953 for 1 Getting Year 1952 for 1 Getting Year 1951 for 1 Getting Year 1950 for 1 Getting Year 1949 for 1 2 Getting Year 2018 for 2 Getting Year 2017 for 2 Getting Year 2016 for 2 Getting Year 2015 for 2 Getting Year 2014 for 2 Getting Year 2013 for 2 Getting Year 2012 for 2 Getting Year 2011 for 2 Getting Year 2010 for 2 Getting Year 2009 for 2 Getting Year 2008 for 2 Getting Year 2007 for 2 Getting Year 2006 for 2 Getting Year 2005 for 2 Getting Year 2004 for 2 Getting Year 2003 for 2 Getting Year 2002 for 2 Getting Year 2001 for 2 Getting Year 2000 for 2 Getting Year 1999 for 2 Getting Year 1998 for 2 Getting Year 1997 for 2 Getting Year 1996 for 2 Getting Year 1995 for 2 Getting Year 1994 for 2 Getting Year 1993 for 2 Getting Year 1992 for 2 Getting Year 1991 for 2 Getting Year 1990 for 2 Getting Year 1989 for 2 Getting Year 1988 for 2 Getting Year 1987 for 2 Getting Year 1986 for 2 Getting Year 1985 for 2 Getting Year 1984 for 2 Getting Year 1983 for 2 Getting Year 1982 for 2 Getting Year 1981 for 2 Getting Year 1980 for 2 Getting Year 1979 for 2 Getting Year 1978 for 2 Getting Year 1977 for 2 Getting Year 1976 for 2 Getting Year 1975 for 2 Getting Year 2001 for 1 Getting Year 2000 for 1 Getting Year 1999 for 1 Getting Year 1998 for 1 Getting Year 1997 for 1 Getting Year 1996 for 1 Getting Year 1995 for 1 Getting Year 1994 for 1 Getting Year 1993 for 1 Getting Year 1992 for 1 Getting Year 1991 for 1 Getting Year 1990 for 1 Getting Year 1989 for 1 Getting Year 1988 for 1 Getting Year 1987 for 1 Getting Year 1986 for 1 Getting Year 1985 for 1 Getting Year 1984 for 1 Getting Year 1983 for 1 Getting Year 1982 for 1 Getting Year 1981 for 1 Getting Year 1980 for 1 Getting Year 1979 for 1 Getting Year 1978 for 1 Getting Year 1977 for 1 Getting Year 1976 for 1 Getting Year 1975 for 1 Getting Year 1974 for 1 Getting Year 1973 for 1 Getting Year 1972 for 1 Getting Year 1971 for 1 Getting Year 1970 for 1 Getting Year 1969 for 1 Getting Year 1968 for 1 Getting Year 1967 for 1 Getting Year 1966 for 1 Getting Year 1965 for 1 Getting Year 1964 for 1 Getting Year 1963 for 1 Getting Year 1962 for 1 Getting Year 1961 for 1 Getting Year 1960 for 1 Getting Year 1959 for 1 Getting Year 1958 for 1 Getting Year 1957 for 1 Getting Year 1956 for 1 Getting Year 1955 for 1 Getting Year 1954 for 1 Getting Year 1953 for 1 Getting Year 1952 for 1 Getting Year 1951 for 1 Getting Year 1950 for 1 Getting Year 1949 for 1 2 Getting Year 2018 for 2 Getting Year 2017 for 2 Getting Year 2016 for 2 Getting Year 2015 for 2 Getting Year 2014 for 2 Getting Year 2013 for 2 Getting Year 2012 for 2 Getting Year 2011 for 2 Getting Year 2010 for 2 Getting Year 2009 for 2 Getting Year 2008 for 2 Getting Year 2007 for 2 Getting Year 2006 for 2 Getting Year 2005 for 2 Getting Year 2004 for 2 Getting Year 2003 for 2 Getting Year 2002 for 2 Getting Year 2001 for 2 Getting Year 2000 for 2 Getting Year 1999 for 2 Getting Year 1998 for 2 Getting Year 1997 for 2 Getting Year 1996 for 2 Getting Year 1995 for 2 Getting Year 1994 for 2 Getting Year 1993 for 2 Getting Year 1992 for 2 Getting Year 1991 for 2 Getting Year 1990 for 2 Getting Year 1989 for 2 Getting Year 1988 for 2 Getting Year 1987 for 2 Getting Year 1986 for 2 Getting Year 1985 for 2 Getting Year 1984 for 2 Getting Year 1983 for 2 Getting Year 1982 for 2 Getting Year 1981 for 2 Getting Year 1980 for 2 Getting Year 1979 for 2 Getting Year 1978 for 2 Getting Year 1977 for 2 Getting Year 1976 for 2 Getting Year 1975 for 2 Getting Year 1974 for 2 Getting Year 1973 for 2 Getting Year 1972 for 2 Getting Year 1971 for 2 Getting Year 1970 for 2 Getting Year 1969 for 2 Getting Year 1968 for 2 Getting Year 1967 for 2 Getting Year 1966 for 2 Getting Year 1965 for 2 Getting Year 1964 for 2 Getting Year 1963 for 2 Getting Year 1962 for 2 Getting Year 1961 for 2 Getting Year 1960 for 2 Getting Year 1959 for 2 Getting Year 1958 for 2 Getting Year 1957 for 2 Getting Year 1956 for 2 Getting Year 1955 for 2 Getting Year 1954 for 2 Getting Year 1953 for 2 Getting Year 1952 for 2 Getting Year 1951 for 2

Data

Downloaded 5 times by klueserthan stefanw

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (36 MB) Use the API

rows 10 / 76345

id row_id part order year toc_doc_id doc_did number date law_date kind name href page doc_url
1
1_2018_5_1
1
1
2018
17542684
264520
5
29.01.2018
meta
Komplette Ausgabe
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264521%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118005.pdf
157
2
1_2018_5_2
1
2
2018
17542684
264520
5
29.01.2018
meta
Inhaltsverzeichnis
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264522%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0157.pdf
157
3
1_2018_5_3
1
3
2018
17542684
264520
5
29.01.2018
16.01.2018
entry
Verordnung zur Übertragung von Zuständigkeiten auf Hauptzollämter für den Bereich mehrerer Hauptzollämter (Hauptzollamtszuständigkeitsverordnung – HZAZustV)
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264523%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0158.pdf
158
4
1_2018_5_4
1
4
2018
17542684
264520
5
29.01.2018
22.01.2018
entry
Verordnung zur Anrechnung von Upstream-Emissionsminderungen auf die Treibhausgasquote (Upstream-Emissionsminderungs-Verordnung – UERV)
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264524%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0169.pdf
169
5
1_2018_5_5
1
5
2018
17542684
264520
5
29.01.2018
25.01.2018
entry
Zweiundzwanzigste Verordnung zur Änderung der Verordnung zur Übertragung von Befugnissen zum Erlass von Rechtsverordnungen auf die Bundesanstalt für Finanzdienstleistungsaufsicht
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264525%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0184.pdf
184
6
1_2018_5_6
1
6
2018
17542684
264520
5
29.01.2018
meta
Hinweis: Rechtsvorschriften der Europäischen Union
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264526%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0185.pdf
185
7
1_2018_4_1
1
1
2018
17542691
264527
4
23.01.2018
meta
Komplette Ausgabe
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264528%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118004.pdf
133
8
1_2018_4_2
1
2
2018
17542691
264527
4
23.01.2018
meta
Inhaltsverzeichnis
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264529%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0133.pdf
133
9
1_2018_4_3
1
3
2018
17542691
264527
4
23.01.2018
16.01.2018
entry
Verordnung zur Änderung der Prüfungsberichtsverordnung
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264530%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0134.pdf
134
10
1_2018_4_4
1
4
2018
17542691
264527
4
23.01.2018
17.01.2018
entry
Verordnung über die Prüfung der Wertpapierdienstleistungsunternehmen nach § 89 des Wertpapierhandelsgesetzes (Wertpapierdienstleistungs-Prüfungsverordnung – WpDPV)
text.xav?SID=&start=%2F%2F*[%40node_id%3D%27264531%27]&skin=&tf=xaver.component.Text_0&hlf=xaver.component.Hitlist_0#bgbl118s0140.pdf
140

Statistics

Average successful run time: about 3 hours

Total run time: about 13 hours

Total cpu time used: 24 minutes

Total disk space used: 36 MB

History

  • Manually ran revision c8c94ed8 and completed successfully .
    4974 records added, 71371 records updated in the database
    16882 pages scraped
  • Manually ran revision 32aa7a77 and failed .
    71371 records added in the database
    38245 pages scraped
  • Manually ran revision 1100ec2f and failed .
    7 pages scraped
  • Manually ran revision 3ce81ed4 and failed .
    1794 pages scraped
  • Manually ran revision 3ce81ed4 and failed .
    8422 pages scraped
  • Created on morph.io

Scraper code

Python

bgbl / scraper.py