daniel-beckham / apod-scraper

Scraper for the "Astronomy Picture of the Day" website


Astronomy Picture of the Day Scraper

This is a web scraper for NASA's Astronomy Picture of the Day website that runs on morph.io. The images are stored in a database that can then be queried using the morph.io API.

As noted in openjck/apod-scraper, there is now an official NASA API that can be used for this same purpose. However, the morph.io API can still be useful as a fallback option. This is demonstrated in my NASA Imagery Fetcher project.

Last run failed with status code 999.

Console output of last run

Please install the Morph Github App on daniel-beckham so that Morph can access this repository on GitHub. Please go to https://github.com/apps/morph-io/installations/new/permissions?repository_ids%5B%5D=53825531&suggested_target_id=6511449 Why? See https://morph.io/documentation/github_app

Data

Downloaded 82925 times by db-morph-io daniel-beckham

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (17.3 MB) Use the API

rows 10 / 7570

url date title credit explanation picture_url video_url
2016-03-14
Dark Nebulas across Taurus
Image Processing & Copyright: <a href="mailto: oliver dot czernetz at gmail dot com">Oliver Czernetz</a> - Data: <a href="http://archive.stsci.edu/dss/copyright.html">Digitized Sky Survey</a> (POSS-II)
Sometimes even the dark dust of interstellar space has a serene beauty. One such place occurs toward the constellation of Taurus. The <a href="http://www.astrobin.com/238942/">filaments featured here</a> can be found on the <a href="http://apod.nasa.gov/apod/ap150128.html">sky between</a> the <a href="http://apod.nasa.gov/apod/ap150617.html">Pleiades star cluster</a> and the <a href="http://apod.nasa.gov/apod/ap160112.html">California Nebula</a>. This dust is not known not for its bright glow but for its absorption and opaqueness. Several bright stars are visible with their blue light seen <a href="https://www.nasa.gov/multimedia/imagegallery/image_feature_701.html">reflecting</a> off the brown dust. Other <a href="http://science.nasa.gov/astrophysics/focus-areas/how-do-stars-form-and-evolve/">stars</a> appear unusually red as their light barely peaks through a column of dark dust, with red the color that remains after the <a href="http://scijinks.jpl.nasa.gov/blue-sky/">blue is scattered</a> away. Yet other stars are behind <a href="http://apod.nasa.gov/apod/ap120722.html">dust pillars</a> so thick they are not visible here. Although <a href="https://s-media-cache-ak0.pinimg.com/236x/71/58/10/715810295b5ec04c5ba55d79cd195e42.jpg">appearing serene</a>, the scene is actually an ongoing loop of tumult and rebirth. This is because massive enough knots of gas and dust will <a href="https://www.youtube.com/watch?v=YbdwTwB8jtc">gravitationally collapse</a> to form new stars -- stars that both <a href="https://en.wikipedia.org/wiki/Cosmic_dust#Dust_grain_formation">create new dust</a> in their atmospheres and destroy old dust with their energetic light and <a href="https://en.wikipedia.org/wiki/Stellar_wind">winds</a>.
2016-03-13
Neon Saturn
Image Credit: <a href="http://wwwvims.lpl.arizona.edu/">VIMS Team</a>, <a href="http://pirlwww.lpl.arizona.edu/">U. Arizona</a>, <a href="http://www.esa.int/">ESA</a>, <a href="http://www.nasa.gov/">NASA</a>
If seen in the right light, Saturn glows like a neon sign. Although Saturn has comparatively little of the <a href="http://periodic.lanl.gov/10.shtml">element neon</a>, a <a href="http://photojournal.jpl.nasa.gov/catalog/PIA09212">composite image</a> false-colored in three bands of <a href="http://coolcosmos.ipac.caltech.edu/cosmic_classroom/ir_tutorial/discovery.html">infrared light</a> highlights features of the giant ringed planet like a <a href="http://en.wikipedia.org/wiki/Neon_sign">glowing sign</a>. At the most blue band of the infrared light featured, false-colored blue in the <a href="http://photojournal.jpl.nasa.gov/catalog/PIA09212">above image</a>, Saturn itself appears dark but <a href="http://apod.nasa.gov/apod/ap040712.html">Saturn's thin rings</a> brightly reflect light from our Sun. Conversely, <a href="http://pds-rings.seti.org/saturn/vgr2_iss/PIA02274.html">Saturn's B ring</a> is so thick that little reflected light makes it through, creating a dark band between <a href="http://en.wikipedia.org/wiki/Saturn's_rings">Saturn's A and C rings</a>. At the most red band of the infrared, false-colored red above, <a href="https://www.nasa.gov/saturn">Saturn</a> emits a surprisingly detailed <a href="http://apod.nasa.gov/apod/ap061012.html">thermal glow</a>, indicating planet-wide bands, huge hurricane-like storms, and a <a href="http://apod.nasa.gov/apod/ap070403.html">strange hexagon-shaped cloud system</a> around the <a href="http://apod.nasa.gov/apod/ap040919.html">North Pole</a>. In the middle infrared band, false-colored green, the sunlit side of Saturn's atmosphere reflects brightly. The <a href="http://photojournal.jpl.nasa.gov/catalog/PIA09212">above image</a> was obtained in 2007 by the <a href="http://www.nasa.gov/mission_pages/cassini/spacecraft/instruments.html">robotic Cassini spacecraft</a> orbiting about 1.6 million kilometers out from Saturn.
2016-03-12
The Flash Spectrum of the Sun
Image Credit & <a href="http://apod.nasa.gov/apod/lib/about_apod.html#srapply">Copyright</a>: <a href="mailto: lfulham [at] icloud [dot] com ">Len Fulham</a>
In a flash, the <a href="http://apod.nasa.gov/apod/ap131002.html">visible spectrum of the Sun</a> changed from <a href="http://csep10.phys.utk.edu/astr162/lect/light/ absorption.html">absorption to emission</a> on March 9 during the total solar eclipse. <a href="http://www.nasa.gov/topics/solarsystem/features/ eclipse/index.html">That fleeting moment</a>, at the beginning the total eclipse phase, is captured by telephoto lens and diffraction grating in this image from clearing skies over <a href="http://apod.nasa.gov/apod/ap160310.html">Ternate, Indonesia</a>. At left, the overwhelming light from the Sun is just blocked by the lunar disk. The normally dominant absorption spectrum of the solar photosphere is hidden. What remains, spread by the <a href="http://micro.magnet.fsu.edu/primer/lightandcolor/ diffractionhome.html">diffraction</a> grating into the spectrum of colors to the right of the eclipsed Sun, are individual eclipse images. The images appear at each wavelength of light emitted by atoms along the thin visible arc of <a href="http://apod.nasa.gov/apod/ap131108.html">the solar chromosphere</a> and in an enormous prominence extending beyond the Sun's upper limb. The brightest images, or strongest <a href="http://dailysolar.weebly.com/uploads/3/4/8/5/3485153/ 9810884.jpg">chromospheric emission lines</a>, are due to Hydrogen atoms that produce the red hydrogen alpha emission at the far right and blue hydrogen beta emission to the left. In between, the bright yellow emission image is caused by atoms of Helium, an element <a href="http://apod.nasa.gov/apod/ap980516.html">only first discovered</a> in the <a href="http://www.phys.vt.edu/~heremans/Astrolab1156/Readings/ flash.html">flash spectrum of the Sun</a>.
2016-03-11
Lunar Shadow Transit
Image Credit: <a href="http://www.nasa.gov/">NASA</a>, <a href="http://www.nesdis.noaa.gov/DSCOVR/">NOAA/DSCOVR</a>
This snapshot from deep space <a href="http://earthobservatory.nasa.gov/IOTD/ view.php?id=87675&eocn=home&eoci=iotd_image">captures planet Earth on March 9</a>. The shadow of its large moon is falling on the planet's sunlit hemisphere. Tracking toward the east (left to right) across the ocean-covered world the moon shadow moved quickly in the direction of the planet's rotation. <a href="http://blog.alaskaair.com/alaska-airlines/news/ eclipse-flight/">Of course</a>, denizens of Earth located close to the shadow track centerline saw this lunar <a href="http://apod.nasa.gov/apod/ap131102.html">shadow transit</a> as a brief, <a href="http://apod.nasa.gov/apod/ap160310.html">total eclipse of the Sun</a>. From a <a href="http://apod.nasa.gov/apod/ap150807.html">spacebased perspective</a> between Earth and Sun, the view of this shadow transit was provided by the <a href="http://www.nesdis.noaa.gov/DSCOVR/">Deep Space Climate Observatory</a> (DSCOVR) spacecraft's Earth Polychromatic Imaging Camera (EPIC).
2016-03-10
Dark Sun over Ternate
Image Credit & <a href="http://apod.nasa.gov/apod/lib/about_apod.html#srapply">Copyright</a>: <a href="http://www.twanight.org/tafreshi">Babak Tafreshi</a> (<a href="http://www.twanight.org/">TWAN</a>)
A dark Sun hangs in the clearing sky over a volcanic planet in this morning sea and skycape. It was taken during <a href="http://www.nasa.gov/topics/solarsystem/features/eclipse/ index.html">this week's total solar eclipse</a>, a dramatic snapshot from along the <a href="http://eclipse.gsfc.nasa.gov/SEgoogle/SEgoogle2001/ SE2016Mar09Tgoogle.html">narrow path of totality</a> in the dark shadow of a New Moon. Earth's Indonesian isle of Ternate, North Maluku lies in the foreground. The sky is still bright near the eastern horizon though, beyond the region's flattened volcanic peaks and outside the <a href="http://www.mreclipse.com/Special/SEprimer.html">Moon's umbral shadow.</a> In fact, near the equator the dark lunar <a href="http://blog.alaskaair.com/alaska-airlines/news/ eclipse-flight/">umbra is rushing eastward</a> across Earth's surface at about 1,700 kilometers (1,100 miles) per hour. Shining through the thin clouds, around the Sun's silhouette is the alluring glow of the solar corona, only easily seen during totality. <a href="http://apod.nasa.gov/apod/ap150331.html">An inspiring sight</a> for eclipse watchers, this solar corona is the tenuous, hot outer atmosphere of the Sun.
2016-03-09
Edge On Galaxy NGC 5866
Image Credit: <a href="http://www.nasa.gov/">NASA</a>, <a href="http://www.esa.int/">ESA</a>, <a href="http://hla.stsci.edu/">Hubble Legacy Archive</a>; Processed & Copyright: <a href="http://hwilson.zenfolio.com/f129011888">Hunter Wilson</a>
Why is this galaxy so thin? Many disk galaxies are actually just as thin as NGC 5866, <a href="http://hwilson.zenfolio.com/galaxies/h1de36ea2#h1de36ea2">pictured</a> <a href="http://hubblesite.org/newscenter/newsdesk/archive/releases/2006/24/">above</a>, but are not <a href="http://apod.nasa.gov/apod/ap010510.html">seen edge-on</a> from our vantage point. One galaxy that is situated edge-on is our own <a href="http://apod.nasa.gov/apod/ap100530.html">Milky Way Galaxy</a>. Classified as a <a href="http://apod.nasa.gov/apod/ap020408.html">lenticular galaxy</a>, NGC 5866 has numerous and complex dust lanes appearing dark and red, while many of the bright stars in the disk give it a more blue underlying hue. The blue disk of young stars can be seen extending past the <a href="http://apod.nasa.gov/apod/ap990919.html">dust</a> in the extremely thin galactic plane, while the bulge in the disk center appears tinged more orange from the older and redder stars that likely exist there. Although similar in mass to our <a href="http://www.nasa.gov/mission_pages/sunearth/news/gallery/galaxy-location.html">Milky Way Galaxy</a>, light takes about 60,000 <a href="http://starchild.gsfc.nasa.gov/docs/StarChild/questions/question19.html">years</a> to cross <a href="http://adsabs.harvard.edu/abs/2003AAS...20311002K">NGC 5866</a>, about 30 percent less than light takes to cross our own Galaxy. In general, many disk galaxies are very thin because the gas that <a href="http://en.wikipedia.org/wiki/Galaxy_formation_and_evolution">formed them</a> collided with itself as it rotated about the gravitational center. Galaxy <a href="http://en.wikipedia.org/wiki/NGC_5866">NGC 5866</a> lies about 50 million light years distant toward the constellation of the Dragon (<a href="http://www.ianridpath.com/startales/draco.htm">Draco</a>).
2016-03-08
Solar Eclipse Shoes in the Classroom
Image Credit & <a href="http://apod.nasa.gov/apod/lib/about_apod.html#srapply">Copyright</a>: Astronomie-AG, <a href="http://www.pgrosenfeld.de">Progymnasium Rosenfeld</a>, <a href="http://www.allthesky.com/">Till Credner</a>, AlltheSky.com
The total solar eclipse of March 8/9 will be the only total eclipse in 2016. Crossing the international date line, <a href="http://www.nasa.gov/feature/goddard/2016/ nasa-releases-march-8-total-solar-eclipse-visualizations">the New Moon's dark shadow</a> traces a limited, narrow path for viewing the total phase, making landfall in Indonesia and mostly tracking across the Pacific Ocean. <a href="http://earthsky.org/?p=230175">A much larger region will be witness</a> to a partially eclipsed Sun though, during morning hours on March 9 for southeast Asia and northeast Australia, and before sunset March 8 for Hawaii and Alaska. Safely viewing the eclipse can actually be very easy. One technique is demonstrated in this shoe group portrait from a classroom in Rosenfeld, Germany, taken during <a href="https://vimeo.com/157906051">March 2015's solar eclipse</a>. With blinds closed to darken the room, each threaded hole in the window blind creates a pinhole camera, projecting multiple images of the eclipsed sun that march across the floor. <a href="http://www.allthesky.de/movies/index.html?sofi20mar15.html">Other viewing alternatives</a> include eclipse glasses and a comfortable chair, but be sure to wear a fashionable <a href="http://apod.nasa.gov/apod/ap080802.html">eclipse shirt</a>.
2016-03-07
Mystery Feature Now Disappears in Titan Lake
Image Credit: <a href="http://saturn.jpl.nasa.gov/spacecraft/instruments-cassini-radar.cfm">Cassini Radar Mapper</a>, <a href="http://www.astro.cornell.edu/cassini.html">Cornell</a>, <a href="http://www.jpl.nasa.gov/">JPL</a>, <a href="http://www.esa.int/">ESA</a>, <a href="http://www.nasa.gov/">NASA</a>
What is that changing object in a cold <a href="http://en.wikipedia.org/wiki/Hydrocarbon">hydrocarbon</a> sea of Titan? Radar images from the robotic <a href="http://saturn.jpl.nasa.gov/spacecraft/overview/">Cassini spacecraft</a> orbiting <a href="http://solarsystem.nasa.gov/planets/profile.cfm?Object=Saturn">Saturn</a> have been recording the surface of the cloud-engulfed moon <a href="http://apod.nasa.gov/apod/ap150202.html">Titan</a> for years. When imaging the flat -- and hence radar dark -- surface of the <a href="http://scifun.chem.wisc.edu/chemweek/methane/methane.html">methane</a> and <a href="http://en.wikipedia.org/wiki/Ethane">ethane</a> lake called <a href="http://en.wikipedia.org/wiki/Ligeia_Mare">Ligeia Mare</a>, an object appeared in 2013 July just was not there in 2007. Subsequent observations in 2014 August found <a href="http://www.slate.com/blogs/bad_astronomy/2014/09/30/titan_lakes_changing_features_are_a_mystery.html">the object</a> remained -- but had changed. In an image released last week, the mystery object seems to have disappeared in 2015 January. The <a href="http://photojournal.jpl.nasa.gov/catalog/PIA20021">featured false-color image</a> shows how the 20-km long <a href="http://www.jpl.nasa.gov/news/news.php?release=2014-327">object</a> has come, changed, and gone. Current origin speculative explanations include <a href="https://www.youtube.com/watch?v=dtVQJCq2cCM">waves</a>, <a href="https://www.youtube.com/watch?v=NOpx4BWBCk0">bubbling foam</a> and <a href="http://apod.nasa.gov/apod/ap050223.html">floating solids</a>, but still no one is sure. <a href="https://pbs.twimg.com/media/Bq_2E8FIIAI83Po.jpg:large">Future observations</a>, in particular Cassini's <a href="http://saturn.jpl.nasa.gov/news/newsreleases/newsrelease20151221/">final</a> close <a href="http://saturn.jpl.nasa.gov/mission/saturntourdates/">flyby</a> of Titan in 2017 April, may either resolve the enigma or open up more speculation.
2016-03-06
A Solar Prominence Eruption from SDO
Video Credit: <a href="http://www.nasa.gov/">NASA</a>/<a href="http://www.nasa.gov/centers/goddard/home/index.html">Goddard</a>/<a href="http://sdo.gsfc.nasa.gov/">SDO AIA Team</a>
One of the most spectacular solar sights is an erupting prominence. In 2011, NASA's Sun-orbiting <a href="http://www.nasa.gov/mission_pages/sdo/main/index.html">Solar Dynamic Observatory</a> spacecraft imaged an <a href="http://apod.nasa.gov/apod/ap030223.html">impressively large prominence</a> erupting from the surface. The <a href="https://www.youtube.com/watch?v=PBL1RBj-P1g">dramatic explosion</a> was captured in ultraviolet light in the <a href="http://www.nasa.gov/mission_pages/sunearth/news/News022411-monsterprom.html">above time lapse video</a> covering 90 minutes, where a <a href="http://www.nasa.gov/mission_pages/sdo/multimedia/VC-1st-light.html">new frame</a> was taken every 24 seconds. The scale of the prominence is huge -- the entire <a href="http://antwrp.gsfc.nasa.gov/apod/image/0208/earthlights02_dmsp_big.jpg">Earth</a> would easily fit under the flowing <a href="http://apod.nasa.gov/apod/ap011203.html">curtain</a> of hot gas. A <a href="http://solar.physics.montana.edu/YPOP/Program/hfilament.html">solar prominence</a> is channeled and sometimes held above the Sun's surface by the Sun's <a href="http://solarscience.msfc.nasa.gov/the_key.shtml">magnetic field</a>. A quiescent <a href="http://csep10.phys.utk.edu/astr162/lect/sun/prominences.html">prominence typically</a> lasts about a month, and may erupt in a <a href="http://helios.gsfc.nasa.gov/cme.html">Coronal Mass Ejection</a> (CME) <a href="http://apod.nasa.gov/apod/ap010924.html">expelling hot gas</a> into the <a href="http://en.wikipedia.org/wiki/Sweden_Solar_System">Solar System</a>. The energy mechanism that creates a <a href="http://apod.nasa.gov/apod/ap030707.html">solar prominence</a> is still a topic of <a href="http://adsabs.harvard.edu/abs/2004ApJ...600.1043Z">research</a>. As the Sun passes <a href="http://en.wikipedia.org/wiki/Solar_maximum">Solar Maximum</a>, <a href="http://apod.nasa.gov/apod/ap071203.html">solar activity</a> like <a href="http://apod.nasa.gov/apod/ap100510.html">eruptive prominences</a> are expected to become less common over the next few years.
2016-03-05
Cities at Night
Image Credit: <a href="http://www.nasa.gov/">NASA</a>, <a href="http://www.nasa.gov/press-release/ update-on-nasa-astronaut-scott-kelly-s-return-to-houston">Scott Kelly</a>
Looking toward the south from an altitude of 400 kilometers, <a href="http://eol.jsc.nasa.gov/SearchPhotos/ photo.pl?mission=ISS044&roll=E&frame=22682">this stunning snapshot</a> from orbit finds bright lights of Tokyo and <a href="http://www.citiesatnight.org/">cities across</a> central and southern Japan, planet Earth shining upward through broken clouds. <a href="https://www.flickr.com/photos/nasa2explore/sets/ 72157648374058938/">The spacefaring perspective</a> was captured last July by astronaut Scott Kelly during his stay on board the International Space Station. Thin stripes of airglow follow the curve of the planet's dark limb, while beyond lie stars of the constellation Centaurus and the southern sky. Their solar panels extended, a docked Soyuz (bottom) and Progress spacecraft are posed in the foreground. <a href="https://www.nasa.gov/press-release/ nasa-astronaut-scott-kelly-returns-safely-to-earth-after-one-year-mission"> Kelly returned to planet Earth</a> this week after his <a href="http://www.nasa.gov/1ym">one-year mission in space</a>.

Statistics

Average successful run time: 3 minutes

Total run time: 4 days

Total cpu time used: about 5 hours

Total disk space used: 17.4 MB

History

  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • ...
  • Created on morph.io

Show complete history

Scraper code

Python

apod-scraper / scraper.py