You can do this by using a headless browser. On morph.io Google Chrome is pre-installed ready for you to use.
The Google Chrome binary is installed at
/usr/bin/google-chrome on every morph.io container
as part of the build process.
You can use Google Chrome directly by running it with
google-chrome --headless --disable-gpu or
you can control it via WebDriver with ChromeDriver
which is installed on morph.io at
gem 'capybara' gem 'selenium-webdriver'
Then in your scraper start a Capybara session using
require "capybara" require "selenium-webdriver" capybara = Capybara::Session.new(:selenium_chrome_headless) # Start scraping capybara.visit("https://morph.io/") puts capybara.find("#banner h2").text
Sometimes there's nothing like seeing a real-life example. If you have a scraper you would like to add to this list, please let us know.
morph.io, faye.morph.io, www.gravatar.com, and 6 others