jamezpolley / example_ruby_chrome_headless_scraper

Example scraper showing how to use Chrome headless from a ruby scraper

README.md

This is a simple scraper showing you how to use Chrome headless with Ruby. Here's what it does:

  1. Visits the morph.io home page
  2. Because the headless browser is a small window it now has to click the button to open the navigation menu so we can see the search box
  3. It enters a search for "planningalerts" into the search box and clicks the submit button
  4. After waiting for the results to appear (see the important gotcha in the code comments!) it outputs the full names of all the scrapers on the search results page

Any questions? Hit up the help forum.

Contributors mlandauer jamezpolley

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Installing bundler 1.17.3 [1G-----> Removing BUNDLED WITH version in the Gemfile.lock [1G-----> Compiling Ruby/Rack [1G Command: 'set -o pipefail; curl -L --fail --retry 5 --retry-delay 1 --connect-timeout 30 --max-time 180 https://heroku-buildpack-ruby.s3.us-east-1.amazonaws.com/heroku-24/amd64/ruby-2.6.2.tgz -s -o - | tar zxf - --strip 0' failed on attempt 1 of 3. [1G Command: 'set -o pipefail; curl -L --fail --retry 5 --retry-delay 1 --connect-timeout 30 --max-time 180 https://heroku-buildpack-ruby.s3.us-east-1.amazonaws.com/heroku-24/amd64/ruby-2.6.2.tgz -s -o - | tar zxf - --strip 0' failed on attempt 2 of 3. [1G [1m[31m [1G ! [1G ! The Ruby version you are trying to install does not exist: ruby-2.6.2 [1G ! [1G ! Heroku recommends you use the latest supported Ruby version listed here: [1G ! https://devcenter.heroku.com/articles/ruby-support#supported-runtimes [1G ! [1G ! For more information on syntax for declaring a Ruby version see: [1G ! https://devcenter.heroku.com/articles/ruby-versions [1G ![0m

Statistics

Average successful run time: less than 20 seconds

Total run time: about 8 hours

Total cpu time used: about 1 hour

Total disk space used: 35.9 KB

History

  • Auto ran revision f26447c1 and failed .
    nothing changed in the database
  • Auto ran revision f26447c1 and failed .
    nothing changed in the database
  • Auto ran revision f26447c1 and failed .
    nothing changed in the database
  • Auto ran revision f26447c1 and failed .
    nothing changed in the database
  • Auto ran revision f26447c1 and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history