weatherpattern / noaa_15min_prec

ncdc noaa 15minute precipitation


Contributors weatherpattern

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Python app detected [1G-----> Installing python-2.7.9 [1G $ pip install -r requirements.txt [1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6)) [1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to /app/.heroku/src/scraperwiki [1G Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 8)) [1G Downloading lxml-3.4.4.tar.gz (3.5MB) [1G Collecting cssselect==0.9.1 (from -r /tmp/build/requirements.txt (line 9)) [1G Downloading cssselect-0.9.1.tar.gz [1G Collecting requests (from -r /tmp/build/requirements.txt (line 10)) [1G Downloading requests-2.17.3-py2.py3-none-any.whl (87kB) [1G Collecting beautifulsoup4 (from -r /tmp/build/requirements.txt (line 11)) [1G Downloading beautifulsoup4-4.6.0-py2-none-any.whl (86kB) [1G Collecting datetime (from -r /tmp/build/requirements.txt (line 12)) [1G Downloading DateTime-4.2-py2.py3-none-any.whl (60kB) [1G Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 6)) [1G Downloading dumptruck-0.1.6.tar.gz [1G Collecting urllib3<1.22,>=1.21.1 (from requests->-r /tmp/build/requirements.txt (line 10)) [1G Downloading urllib3-1.21.1-py2.py3-none-any.whl (131kB) [1G Collecting idna<2.6,>=2.5 (from requests->-r /tmp/build/requirements.txt (line 10)) [1G Downloading idna-2.5-py2.py3-none-any.whl (55kB) [1G Collecting certifi>=2017.4.17 (from requests->-r /tmp/build/requirements.txt (line 10)) [1G Downloading certifi-2017.4.17-py2.py3-none-any.whl (375kB) [1G Collecting chardet<3.1.0,>=3.0.2 (from requests->-r /tmp/build/requirements.txt (line 10)) [1G Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB) [1G Collecting pytz (from datetime->-r /tmp/build/requirements.txt (line 12)) [1G Downloading pytz-2017.2-py2.py3-none-any.whl (484kB) [1G Collecting zope.interface (from datetime->-r /tmp/build/requirements.txt (line 12)) [1G Downloading zope.interface-4.4.1-cp27-cp27m-manylinux1_x86_64.whl (169kB) [1G Installing collected packages: dumptruck, urllib3, idna, certifi, chardet, requests, scraperwiki, lxml, cssselect, beautifulsoup4, pytz, zope.interface, datetime [1G Running setup.py install for dumptruck: started [1G Running setup.py install for dumptruck: finished with status 'done' [1G Running setup.py develop for scraperwiki [1G Running setup.py install for lxml: started [1G Running setup.py install for lxml: still running... [1G Running setup.py install for lxml: finished with status 'done' [1G Running setup.py install for cssselect: started [1G Running setup.py install for cssselect: finished with status 'done' [1G Successfully installed beautifulsoup4-4.6.0 certifi-2017.4.17 chardet-3.0.4 cssselect-0.9.1 datetime-4.2 dumptruck-0.1.6 idna-2.5 lxml-3.4.4 pytz-2017.2 requests-2.17.3 scraperwiki urllib3-1.21.1 zope.interface-4.4.1 [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/?C=N;O=D https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/?C=M;O=A https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/?C=S;O=A https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/?C=D;O=A https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260//pub/data/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/01/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/02/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/03/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/04/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/05/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/06/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/07/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/08/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/09/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/10/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/11/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/12/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/13/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/14/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/15/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/16/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/17/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/18/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/19/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/20/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/21/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/22/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/23/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/24/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/25/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/26/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/27/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/28/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/29/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/30/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/31/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/32/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/33/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/34/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/35/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/36/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/37/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/38/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/39/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/40/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/41/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/42/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/43/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/44/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/45/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/46/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/47/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/48/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/50/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/51/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/66/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/67/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/91/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/by_month2011/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/by_month2012/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/by_month2013/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/by_month2014/ https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/dsi3260.doc https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/dsi3260.pdf https://www1.ncdc.noaa.gov/pub/data/15min_precip-3260/readme.txt

Data

Downloaded 2 times by JoshCMoore blairdgeo

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (29 KB) Use the API

rows 1 / 1

run_id url UUID timestamp body_content body_SHA256 headers
1
0000
2017-06-12 12:26:01.069522
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <html> <head> <title>Index of /pub/data/15min_precip-3260</title> </head> <body> <h1>Index of /pub/data/15min_precip-3260</h1> <table><tr><th><img src="/icons/blank.gif" alt="[ICO]"></th><th><a href="?C=N;O=D">Name</a></th><th><a href="?C=M;O=A">Last modified</a></th><th><a href="?C=S;O=A">Size</a></th><th><a href="?C=D;O=A">Description</a></th></tr><tr><th colspan="5"><hr></th></tr> <tr><td valign="top"><img src="/icons/back.gif" alt="[DIR]"></td><td><a href="/pub/data/">Parent Directory</a></td><td> </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="01/">01/</a></td><td align="right">03-Oct-2012 11:48 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="02/">02/</a></td><td align="right">03-Oct-2012 11:49 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="03/">03/</a></td><td align="right">03-Oct-2012 11:50 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="04/">04/</a></td><td align="right">03-Oct-2012 11:50 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="05/">05/</a></td><td align="right">03-Oct-2012 11:51 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="06/">06/</a></td><td align="right">03-Oct-2012 11:51 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="07/">07/</a></td><td align="right">03-Oct-2012 11:52 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="08/">08/</a></td><td align="right">03-Oct-2012 11:52 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="09/">09/</a></td><td align="right">03-Oct-2012 11:53 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="10/">10/</a></td><td align="right">03-Oct-2012 11:54 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="11/">11/</a></td><td align="right">03-Oct-2012 11:54 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="12/">12/</a></td><td align="right">03-Oct-2012 11:54 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="13/">13/</a></td><td align="right">03-Oct-2012 11:55 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="14/">14/</a></td><td align="right">03-Oct-2012 11:55 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="15/">15/</a></td><td align="right">03-Oct-2012 11:56 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="16/">16/</a></td><td align="right">03-Oct-2012 11:56 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="17/">17/</a></td><td align="right">03-Oct-2012 11:56 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="18/">18/</a></td><td align="right">03-Oct-2012 11:57 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="19/">19/</a></td><td align="right">03-Oct-2012 11:57 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="20/">20/</a></td><td align="right">03-Oct-2012 11:58 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="21/">21/</a></td><td align="right">03-Oct-2012 11:59 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="22/">22/</a></td><td align="right">03-Oct-2012 11:59 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="23/">23/</a></td><td align="right">03-Oct-2012 11:59 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="24/">24/</a></td><td align="right">03-Oct-2012 12:00 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="25/">25/</a></td><td align="right">03-Oct-2012 12:00 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="26/">26/</a></td><td align="right">03-Oct-2012 12:00 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="27/">27/</a></td><td align="right">03-Oct-2012 12:00 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="28/">28/</a></td><td align="right">03-Oct-2012 12:01 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="29/">29/</a></td><td align="right">03-Oct-2012 12:01 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="30/">30/</a></td><td align="right">03-Oct-2012 12:02 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="31/">31/</a></td><td align="right">03-Oct-2012 12:02 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="32/">32/</a></td><td align="right">03-Oct-2012 12:03 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="33/">33/</a></td><td align="right">03-Oct-2012 12:03 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="34/">34/</a></td><td align="right">03-Oct-2012 12:04 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="35/">35/</a></td><td align="right">03-Oct-2012 12:04 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="36/">36/</a></td><td align="right">03-Oct-2012 12:05 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="37/">37/</a></td><td align="right">01-Aug-2012 14:17 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="38/">38/</a></td><td align="right">03-Oct-2012 12:05 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="39/">39/</a></td><td align="right">03-Oct-2012 12:06 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="40/">40/</a></td><td align="right">03-Oct-2012 12:06 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="41/">41/</a></td><td align="right">03-Oct-2012 12:06 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="42/">42/</a></td><td align="right">03-Oct-2012 12:07 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="43/">43/</a></td><td align="right">03-Oct-2012 12:07 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="44/">44/</a></td><td align="right">03-Oct-2012 12:07 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="45/">45/</a></td><td align="right">03-Oct-2012 12:21 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="46/">46/</a></td><td align="right">03-Oct-2012 12:21 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="47/">47/</a></td><td align="right">03-Oct-2012 12:22 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="48/">48/</a></td><td align="right">03-Oct-2012 12:22 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="50/">50/</a></td><td align="right">03-Oct-2012 12:23 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="51/">51/</a></td><td align="right">03-Oct-2012 12:23 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="66/">66/</a></td><td align="right">03-Oct-2012 12:23 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="67/">67/</a></td><td align="right">03-Oct-2012 12:24 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="91/">91/</a></td><td align="right">03-Oct-2012 12:24 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="by_month2011/">by_month2011/</a></td><td align="right">30-Mar-2016 16:27 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="by_month2012/">by_month2012/</a></td><td align="right">12-Feb-2015 10:24 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="by_month2013/">by_month2013/</a></td><td align="right">23-Mar-2015 08:18 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="by_month2014/">by_month2014/</a></td><td align="right">08-Dec-2014 13:48 </td><td align="right"> - </td><td> </td></tr> <tr><td valign="top"><img src="/icons/unknown.gif" alt="[ ]"></td><td><a href="dsi3260.doc">dsi3260.doc</a></td><td align="right">01-Aug-2012 16:06 </td><td align="right"> 98K</td><td> </td></tr> <tr><td valign="top"><img src="/icons/layout.gif" alt="[ ]"></td><td><a href="dsi3260.pdf">dsi3260.pdf</a></td><td align="right">01-Aug-2012 16:06 </td><td align="right"> 68K</td><td> </td></tr> <tr><td valign="top"><img src="/icons/text.gif" alt="[TXT]"></td><td><a href="readme.txt">readme.txt</a></td><td align="right">11-Jun-2014 15:10 </td><td align="right">2.1K</td><td> </td></tr> <tr><th colspan="5"><hr></th></tr> </table> </body></html>
566a775b0c5f983146d891d1bbceb8844e57d661f584ba68bb5e693c88f62895
{"Content-Length": "1036", "Date": "Mon, 12 Jun 2017 12:26:03 GMT", "Server": "Apache", "Content-Type": "text/html;charset=UTF-8", "Access-Control-Allow-Origin": "*", "Strict-Transport-Security": "max-age=31536000", "Connection": "close", "Content-Encoding": "gzip", "Vary": "Accept-Encoding"}

Statistics

Average successful run time: 1 minute

Total run time: 10 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 54.8 KB

History

  • Manually ran revision 108431f0 and completed successfully .
    67 records added in the database
  • Manually ran revision 108431f0 and completed successfully .
    nothing changed in the database
  • Manually ran revision 108431f0 and completed successfully .
    67 records added, 65 records removed in the database
  • Manually ran revision 108431f0 and completed successfully .
    nothing changed in the database
  • Manually ran revision 108431f0 and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

noaa_15min_prec