Stanpol / ted_com_scraper

ted.com talks

Scrapes www.ted.com

TED Talks are influential videos from expert speakers on education, business, science, tech and creativity, with subtitles in 100+ languages. Ideas free to stream and download.


This is a scraper that runs on Morph.

It scrapes the latest talks from TED.com.

Contributors Stanpol

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-2.7.9  $ pip install -r requirements.txt  Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 6))  Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki  Collecting lxml==3.4.4 (from -r requirements.txt (line 8))  Downloading lxml-3.4.4.tar.gz (3.5MB)  Collecting bs4==0.0.1 (from -r requirements.txt (line 9))  Downloading bs4-0.0.1.tar.gz  Collecting cssselect==0.9.1 (from -r requirements.txt (line 10))  Downloading cssselect-0.9.1.tar.gz  Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 6))  Downloading dumptruck-0.1.6.tar.gz  Collecting requests (from scraperwiki->-r requirements.txt (line 6))  Downloading requests-2.12.3-py2.py3-none-any.whl (575kB)  Collecting beautifulsoup4 (from bs4==0.0.1->-r requirements.txt (line 9))  Downloading beautifulsoup4-4.5.1-py2-none-any.whl (83kB)  Installing collected packages: dumptruck, requests, scraperwiki, lxml, beautifulsoup4, bs4, cssselect  Running setup.py install for dumptruck: started  Running setup.py install for dumptruck: finished with status 'done'  Running setup.py develop for scraperwiki  Running setup.py install for lxml: started  Running setup.py install for lxml: still running...  Running setup.py install for lxml: finished with status 'done'  Running setup.py install for bs4: started  Running setup.py install for bs4: finished with status 'done'  Running setup.py install for cssselect: started  Running setup.py install for cssselect: finished with status 'done'  Successfully installed beautifulsoup4-4.5.1 bs4-0.0.1 cssselect-0.9.1 dumptruck-0.1.6 lxml-3.4.4 requests-2.12.3 scraperwiki   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Scraping the latest page Natalie Panek: Let's clean up the space junk orbiting Earth Tiq Milan and Kim Katrin Milan: A queer vision of love and marriage Kate Adams: 4 larger-than-life lessons from soap operas Ryan Gravel: How an old loop of railroads is changing the face of a city Danny Dorling: Maps that show us who we are (not just where we are) Alicia Garza, Patrisse Cullors, Opal Tometi and Mia Birdsong: An interview with the founders of Black Lives Matter Joe Lassiter: We need nuclear power to solve climate change Adam Galinsky: How to speak up for yourself Juan Enriquez: What will humans look like in 100 years? Sandi Toksvig: A political party for women's equality Roger Antonsen: Math is the hidden secret to understanding the world Victor Rios: Help for kids the education system ignores Steven Johnson: The playful wonderland behind great inventions Bettina Warburg: How the blockchain will radically transform the economy Kimberlé Crenshaw and Abby Dobson: The urgency of intersectionality Hector Garcia: We train soldiers for war. Let's train them to come home, too Fawn Qiu: Easy DIY projects for kid engineers Jonathan Haidt: Can a divided America heal? Suzanne Barakat: Islamophobia killed my brother. Let's end the hate Halla Tómasdóttir: It's time for women to run for office Amanda Palmer, Jherek Bischoff and Usman Riaz: "Space Oddity" Tim Leberecht: 4 ways to build a human company in the age of machines Mallory Soldner: Your company's data could help end world hunger Kandice Sumner: How America's public schools keep kids in poverty Christopher Soghoian: Your smartphone is a civil rights issue Kelli Jean Drinkwater: Enough with the fear of fat Silk Road Ensemble and Rhiannon Giddens: "St. James Infirmary Blues" Tasos Frantzolas: Everything you hear on film is a lie Alyssa Monks: How loss helped one artist find beauty in imperfection Ian Bremmer: How the US should use its superpower status Wanis Kabbaj: What a driverless world could look like Philippa Neave: The unexpected challenges of a country's first election Manwar Ali: Inside the mind of a former radical jihadist Zeynep Tufekci: Machine intelligence makes human morals more important Todd Coleman: A temporary tattoo that brings hospital care to the home Rachel Botsman: We've stopped trusting institutions and started trusting strangers

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.31 MB) Use the API

rows 10 / 2352

link_high link_medium link_low event_url talk_url event_name time date talk_name
/talks?q=TEDMED+2015
/talks/sarah_gray_how_my_son_s_short_life_made_a_lasting_difference
TEDMED 2015
10:17
May 2016
Sarah Gray: How my son's short life made a lasting difference
/talks?q=TED2016
/talks/riccardo_sabatini_how_to_read_the_genome_and_build_a_human_being
TED2016
15:28
Apr 2016
Riccardo Sabatini: How to read the genome and build a human being
/talks?q=TED%40State+Street+Boston
/talks/michael_metcalfe_a_provocative_way_to_finance_the_fight_against_climate_change
TED@State Street Boston
12:52
Apr 2016
Michael Metcalfe: A provocative way to finance the fight against climate change
/talks?q=TED2016
/talks/ameera_harouda_why_i_put_myself_in_danger_to_tell_the_stories_of_gaza
TED2016
8:38
Apr 2016
Ameera Harouda: Why I put myself in danger to tell the stories of Gaza
/talks?q=TED2016
/talks/r_luke_dubois_insightful_human_portraits_made_from_data
TED2016
12:43
Apr 2016
R. Luke DuBois: Insightful human portraits made from data
/talks?q=TED2016
/talks/shivani_siroya_a_smart_loan_for_people_with_no_credit_history_yet
TED2016
8:11
Apr 2016
Shivani Siroya: A smart loan for people with no credit history (yet)
/talks?q=TED2016
/talks/kenneth_lacovara_hunting_for_dinosaurs_showed_me_our_place_in_the_universe
TED2016
15:49
Apr 2016
Kenneth Lacovara: Hunting for dinosaurs showed me our place in the universe
/talks?q=TEDxGatewayWomen
/talks/aditi_gupta_a_taboo_free_way_to_talk_about_periods
TEDxGatewayWomen
11:10
Apr 2016
Aditi Gupta: A taboo-free way to talk about periods
/talks?q=TED+Talks+Live
/talks/juan_enriquez_we_can_reprogram_life_how_to_do_it_wisely
TED Talks Live
14:49
Apr 2016
Juan Enriquez: We can reprogram life. How to do it wisely
/talks?q=TEDActive+2015
/talks/joshua_prager_wisdom_from_great_writers_on_every_year_of_life
TEDActive 2015
6:01
Apr 2016
Joshua Prager: Wisdom from great writers on every year of life

Statistics

Average successful run time: 1 minute

Total run time: about 4 hours

Total cpu time used: 2 minutes

Total disk space used: 3.23 MB

History

  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    2 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    7 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 32 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    174 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 32 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 33 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    6 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    2 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    3 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    64 pages scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 31 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
    1 page scraped
  • Auto ran revision e7e4a317 and failed .
    nothing changed in the database
  • Auto ran revision e7e4a317 and failed .
    nothing changed in the database
  • Auto ran revision e7e4a317 and failed .
    nothing changed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 34 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 35 records removed in the database
  • Auto ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Manually ran revision e7e4a317 and completed successfully .
    36 records added, 36 records removed in the database
  • Manually ran revision bb478931 and completed successfully .
    36 records added, 36 records removed in the database
  • Manually ran revision 711698d5 and completed successfully .
    2235 records added, 13 records removed in the database
  • Manually ran revision dc398266 and failed .
    13 records added, 13 records removed in the database
  • Manually ran revision f4f2b7e3 and failed .
    13 records added in the database
  • Manually ran revision 01d32227 and failed .
    nothing changed in the database
  • Created on morph.io

Scraper code

Python

ted_com_scraper / scraper.py