Feng-Gao / lyon-opendata

data.grandlyon.com


This is a scraper that runs on Morph. To get started see the documentation

Contributors Feng-Gao

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected  ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported).  ! We recommend upgrading by specifying the latest version (python-2.7.14).  Learn More: https://devcenter.heroku.com/articles/python-runtimes -----> Installing python-2.7.9 -----> Installing pip -----> Installing requirements with pip  DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki  Running command git clone -q http://github.com/openaustralia/scraperwiki-python.git /app/.heroku/src/scraperwiki  Running command git checkout -b morph_defaults --track origin/morph_defaults  Switched to a new branch 'morph_defaults'  Branch morph_defaults set up to track remote branch morph_defaults from origin.  Collecting beautifulsoup4==4.4.0  Downloading beautifulsoup4-4.4.0-py2-none-any.whl (81 kB)  Collecting requests==2.21.0  Downloading requests-2.21.0-py2.py3-none-any.whl (57 kB)  Collecting lxml==3.4.4  Downloading lxml-3.4.4.tar.gz (3.5 MB)  Collecting cssselect==0.9.1  Downloading cssselect-0.9.1.tar.gz (32 kB)  Collecting fake_useragent==0.1.11  Downloading fake-useragent-0.1.11.tar.gz (13 kB)  Collecting dumptruck>=0.1.2  Downloading dumptruck-0.1.6.tar.gz (15 kB)  Collecting urllib3<1.25,>=1.21.1  Downloading urllib3-1.24.3-py2.py3-none-any.whl (118 kB)  Collecting chardet<3.1.0,>=3.0.2  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)  Collecting idna<2.9,>=2.5  Downloading idna-2.8-py2.py3-none-any.whl (58 kB)  Building wheels for collected packages: lxml, cssselect, fake-useragent, dumptruck  Building wheel for lxml (setup.py): started  Building wheel for lxml (setup.py): still running...  Building wheel for lxml (setup.py): finished with status 'done'  Created wheel for lxml: filename=lxml-3.4.4-cp27-cp27m-linux_x86_64.whl size=2989829 sha256=0d5f6db8c9932123fd1bc19db6bce4b975c55aa084bba21e4901b7d0045161f0  Stored in directory: /tmp/pip-ephem-wheel-cache-JAIJ1Q/wheels/d6/de/81/11ae6edd05c75aac677e67dd154c85da758ba6f3e8e80e962e  Building wheel for cssselect (setup.py): started  Building wheel for cssselect (setup.py): finished with status 'done'  Created wheel for cssselect: filename=cssselect-0.9.1-py2-none-any.whl size=26992 sha256=c2a4eb738f590a847d6d6c81c12717442c8e3d18744a0f6be2972061780e2133  Stored in directory: /tmp/pip-ephem-wheel-cache-JAIJ1Q/wheels/85/fe/00/b94036d8583cec9791d8cda24c184f2d2ac1397822f7f0e8d4  Building wheel for fake-useragent (setup.py): started  Building wheel for fake-useragent (setup.py): finished with status 'done'  Created wheel for fake-useragent: filename=fake_useragent-0.1.11-py2-none-any.whl size=13486 sha256=beaa893b6042d25d9e443863d3dcad5e473662aecfa4b442b1d42fbc305b3a50  Stored in directory: /tmp/pip-ephem-wheel-cache-JAIJ1Q/wheels/31/cd/fd/01593bc9cfb3f38fc789c2646fce5dab2d9fef4af01547d447  Building wheel for dumptruck (setup.py): started  Building wheel for dumptruck (setup.py): finished with status 'done'  Created wheel for dumptruck: filename=dumptruck-0.1.6-py2-none-any.whl size=11844 sha256=2847b8facc2fdd8fc139be25f42c3cac494c5b0a67c59654988d4c839cd4d1c9  Stored in directory: /tmp/pip-ephem-wheel-cache-JAIJ1Q/wheels/dc/75/e9/1e61c4080c73e7bda99614549591f83b53bcc2d682f26fce62  Successfully built lxml cssselect fake-useragent dumptruck  Installing collected packages: dumptruck, urllib3, chardet, certifi, idna, requests, scraperwiki, beautifulsoup4, lxml, cssselect, fake-useragent  Running setup.py develop for scraperwiki  Successfully installed beautifulsoup4-4.4.0 certifi-2021.10.8 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 fake-useragent-0.1.11 idna-2.8 lxml-3.4.4 requests-2.21.0 scraperwiki urllib3-1.24.3 DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.    -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... No handlers could be found for logger "root" https://data.grandlyon.com/acceder-aux-donnees/?&P=1 https://data.grandlyon.com/acceder-aux-donnees/?&P=2 https://data.grandlyon.com/acceder-aux-donnees/?&P=3 https://data.grandlyon.com/acceder-aux-donnees/?&P=4 https://data.grandlyon.com/acceder-aux-donnees/?&P=5 https://data.grandlyon.com/acceder-aux-donnees/?&P=6 https://data.grandlyon.com/acceder-aux-donnees/?&P=7 https://data.grandlyon.com/acceder-aux-donnees/?&P=8 https://data.grandlyon.com/acceder-aux-donnees/?&P=9 https://data.grandlyon.com/acceder-aux-donnees/?&P=10 https://data.grandlyon.com/acceder-aux-donnees/?&P=11 https://data.grandlyon.com/acceder-aux-donnees/?&P=12 https://data.grandlyon.com/acceder-aux-donnees/?&P=13 https://data.grandlyon.com/acceder-aux-donnees/?&P=14 https://data.grandlyon.com/acceder-aux-donnees/?&P=15 https://data.grandlyon.com/acceder-aux-donnees/?&P=16 https://data.grandlyon.com/acceder-aux-donnees/?&P=17 https://data.grandlyon.com/acceder-aux-donnees/?&P=18 https://data.grandlyon.com/acceder-aux-donnees/?&P=19 https://data.grandlyon.com/acceder-aux-donnees/?&P=20 https://data.grandlyon.com/acceder-aux-donnees/?&P=21 https://data.grandlyon.com/acceder-aux-donnees/?&P=22 https://data.grandlyon.com/acceder-aux-donnees/?&P=23 https://data.grandlyon.com/acceder-aux-donnees/?&P=24 https://data.grandlyon.com/acceder-aux-donnees/?&P=25 https://data.grandlyon.com/acceder-aux-donnees/?&P=26 https://data.grandlyon.com/acceder-aux-donnees/?&P=27 https://data.grandlyon.com/acceder-aux-donnees/?&P=28 https://data.grandlyon.com/acceder-aux-donnees/?&P=29 https://data.grandlyon.com/acceder-aux-donnees/?&P=30 https://data.grandlyon.com/acceder-aux-donnees/?&P=31 https://data.grandlyon.com/acceder-aux-donnees/?&P=32 https://data.grandlyon.com/acceder-aux-donnees/?&P=33 https://data.grandlyon.com/acceder-aux-donnees/?&P=34 https://data.grandlyon.com/acceder-aux-donnees/?&P=35 https://data.grandlyon.com/acceder-aux-donnees/?&P=36 https://data.grandlyon.com/acceder-aux-donnees/?&P=37 https://data.grandlyon.com/acceder-aux-donnees/?&P=38 https://data.grandlyon.com/acceder-aux-donnees/?&P=39 https://data.grandlyon.com/acceder-aux-donnees/?&P=40 https://data.grandlyon.com/acceder-aux-donnees/?&P=41 https://data.grandlyon.com/acceder-aux-donnees/?&P=42 https://data.grandlyon.com/acceder-aux-donnees/?&P=43 https://data.grandlyon.com/acceder-aux-donnees/?&P=44 https://data.grandlyon.com/acceder-aux-donnees/?&P=45 https://data.grandlyon.com/acceder-aux-donnees/?&P=46 https://data.grandlyon.com/acceder-aux-donnees/?&P=47 https://data.grandlyon.com/acceder-aux-donnees/?&P=48 https://data.grandlyon.com/acceder-aux-donnees/?&P=49 https://data.grandlyon.com/acceder-aux-donnees/?&P=50 https://data.grandlyon.com/acceder-aux-donnees/?&P=51 https://data.grandlyon.com/acceder-aux-donnees/?&P=52 https://data.grandlyon.com/acceder-aux-donnees/?&P=53 https://data.grandlyon.com/acceder-aux-donnees/?&P=54 https://data.grandlyon.com/acceder-aux-donnees/?&P=55 https://data.grandlyon.com/acceder-aux-donnees/?&P=56 https://data.grandlyon.com/acceder-aux-donnees/?&P=57 https://data.grandlyon.com/acceder-aux-donnees/?&P=58 https://data.grandlyon.com/acceder-aux-donnees/?&P=59 https://data.grandlyon.com/acceder-aux-donnees/?&P=60 https://data.grandlyon.com/acceder-aux-donnees/?&P=61 https://data.grandlyon.com/acceder-aux-donnees/?&P=62 https://data.grandlyon.com/acceder-aux-donnees/?&P=63 https://data.grandlyon.com/acceder-aux-donnees/?&P=64 https://data.grandlyon.com/acceder-aux-donnees/?&P=65 https://data.grandlyon.com/acceder-aux-donnees/?&P=66 https://data.grandlyon.com/acceder-aux-donnees/?&P=67 https://data.grandlyon.com/acceder-aux-donnees/?&P=68 https://data.grandlyon.com/acceder-aux-donnees/?&P=69 https://data.grandlyon.com/acceder-aux-donnees/?&P=70 https://data.grandlyon.com/acceder-aux-donnees/?&P=71 https://data.grandlyon.com/acceder-aux-donnees/?&P=72 https://data.grandlyon.com/acceder-aux-donnees/?&P=73 https://data.grandlyon.com/acceder-aux-donnees/?&P=74 https://data.grandlyon.com/acceder-aux-donnees/?&P=75 https://data.grandlyon.com/acceder-aux-donnees/?&P=76 https://data.grandlyon.com/acceder-aux-donnees/?&P=77 https://data.grandlyon.com/acceder-aux-donnees/?&P=78 https://data.grandlyon.com/acceder-aux-donnees/?&P=79 https://data.grandlyon.com/acceder-aux-donnees/?&P=80 https://data.grandlyon.com/acceder-aux-donnees/?&P=81 https://data.grandlyon.com/acceder-aux-donnees/?&P=82 https://data.grandlyon.com/acceder-aux-donnees/?&P=83 https://data.grandlyon.com/acceder-aux-donnees/?&P=84 https://data.grandlyon.com/acceder-aux-donnees/?&P=85 https://data.grandlyon.com/acceder-aux-donnees/?&P=86 https://data.grandlyon.com/acceder-aux-donnees/?&P=87 https://data.grandlyon.com/acceder-aux-donnees/?&P=88 https://data.grandlyon.com/acceder-aux-donnees/?&P=89 https://data.grandlyon.com/acceder-aux-donnees/?&P=90 https://data.grandlyon.com/acceder-aux-donnees/?&P=91 https://data.grandlyon.com/acceder-aux-donnees/?&P=92 https://data.grandlyon.com/acceder-aux-donnees/?&P=93 https://data.grandlyon.com/acceder-aux-donnees/?&P=94 https://data.grandlyon.com/acceder-aux-donnees/?&P=95 https://data.grandlyon.com/acceder-aux-donnees/?&P=96 https://data.grandlyon.com/acceder-aux-donnees/?&P=97 https://data.grandlyon.com/acceder-aux-donnees/?&P=98 https://data.grandlyon.com/acceder-aux-donnees/?&P=99 https://data.grandlyon.com/acceder-aux-donnees/?&P=100 https://data.grandlyon.com/acceder-aux-donnees/?&P=101 https://data.grandlyon.com/acceder-aux-donnees/?&P=102 https://data.grandlyon.com/acceder-aux-donnees/?&P=103 https://data.grandlyon.com/acceder-aux-donnees/?&P=104 https://data.grandlyon.com/acceder-aux-donnees/?&P=105 https://data.grandlyon.com/acceder-aux-donnees/?&P=106 https://data.grandlyon.com/acceder-aux-donnees/?&P=107 https://data.grandlyon.com/acceder-aux-donnees/?&P=108 https://data.grandlyon.com/acceder-aux-donnees/?&P=109 https://data.grandlyon.com/acceder-aux-donnees/?&P=110 https://data.grandlyon.com/acceder-aux-donnees/?&P=111 https://data.grandlyon.com/acceder-aux-donnees/?&P=112 https://data.grandlyon.com/acceder-aux-donnees/?&P=113 https://data.grandlyon.com/acceder-aux-donnees/?&P=114 https://data.grandlyon.com/acceder-aux-donnees/?&P=115 https://data.grandlyon.com/acceder-aux-donnees/?&P=116 https://data.grandlyon.com/acceder-aux-donnees/?&P=117 https://data.grandlyon.com/acceder-aux-donnees/?&P=118 https://data.grandlyon.com/acceder-aux-donnees/?&P=119 https://data.grandlyon.com/acceder-aux-donnees/?&P=120 https://data.grandlyon.com/acceder-aux-donnees/?&P=121 https://data.grandlyon.com/acceder-aux-donnees/?&P=122 https://data.grandlyon.com/acceder-aux-donnees/?&P=123 https://data.grandlyon.com/acceder-aux-donnees/?&P=124 https://data.grandlyon.com/acceder-aux-donnees/?&P=125 https://data.grandlyon.com/acceder-aux-donnees/?&P=126 https://data.grandlyon.com/acceder-aux-donnees/?&P=127 https://data.grandlyon.com/acceder-aux-donnees/?&P=128 https://data.grandlyon.com/acceder-aux-donnees/?&P=129 https://data.grandlyon.com/acceder-aux-donnees/?&P=130 https://data.grandlyon.com/acceder-aux-donnees/?&P=131 https://data.grandlyon.com/acceder-aux-donnees/?&P=132 https://data.grandlyon.com/acceder-aux-donnees/?&P=133 https://data.grandlyon.com/acceder-aux-donnees/?&P=134 everything OK now. Total dataset_count is0

Data

Downloaded 3 times by Feng-Gao

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (40.7 MB) Use the API

rows 10 / 41690

org tags id view url format today created name topics desc frequency
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
1
153
zip
04/14/2019
jeudi 01 janvier 1970
"Nuage de points LIDAR 2015 du Grand Lyon - Dalle 1857_5178"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
2
152
zip
04/14/2019
jeudi 01 janvier 1970
"Nuage de points LIDAR 2015 du Grand Lyon - Dalle 1857_5184"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
3
202
zip
04/14/2019
jeudi 01 janvier 1970
"Nuage de points LIDAR 2015 du Grand Lyon - Dalle 1860_5175"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
4
203
zip
04/14/2019
jeudi 01 janvier 1970
"Points LIDAR 2015 de la Métropole de Lyon - Dalle 1830_5196"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
5
131
zip
04/14/2019
jeudi 01 janvier 1970
"Points LIDAR 2015 de la Métropole de Lyon - Dalle 1833_5151"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
6
173
zip
04/14/2019
jeudi 01 janvier 1970
"Points LIDAR 2015 de la Métropole de Lyon - Dalle 1830_5193"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
7
207
zip
04/14/2019
jeudi 01 janvier 1970
"Points LIDAR 2015 de la Métropole de Lyon - Dalle 1830_5166"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon"
Services d'utilité publique et services publics| données ouvertes
8
27
WMS|WFS|KML|geo-json|Shape-zip|json
04/14/2019
mardi 15 octobre 2019
"Déchèteries de la Métropole de Lyon (ver. 3.0.0)"
Environnement|Équipements|Services|
"Données ponctuelles représentant l'emplacement des déchèteries de la Métropole de Lyon enrichi des horaires, des types de déchets et des véhicules autorisés."
Mensuelle
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
9
206
zip
04/14/2019
jeudi 01 janvier 1970
"Points LIDAR 2015 de la Métropole de Lyon - Dalle 1830_5157"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière
"Métropole de Lyon / Direction Innovation Numérique et Systèmes d'Information..."
Ortho-imagerie| données ouvertes
10
287
zip
04/14/2019
jeudi 01 janvier 1970
"Points LIDAR 2015 de la Métropole de Lyon - Dalle 1830_5151"
Imagerie|Occupation du sol
"Nuage de points au format LAS 1.2. en dalles de 1 x 1 km sur une emprise de 1 400 km².selon le quadrillage RGF93 CC46, selon une précision de +/- 0,15 m. La densité des points varie de 9 pts/m² à 14 pts/m²."
Irrégulière

Statistics

Average successful run time: 17 minutes

Total run time: 15 days

Total cpu time used: about 4 hours

Total disk space used: 40.7 MB

History

  • Auto ran revision ee2298bc and completed successfully .
    nothing changed in the database
  • Auto ran revision ee2298bc and completed successfully .
    nothing changed in the database
  • Auto ran revision ee2298bc and completed successfully .
    nothing changed in the database
  • Auto ran revision ee2298bc and completed successfully .
    nothing changed in the database
  • Auto ran revision ee2298bc and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

lyon-opendata / scraper.py