This is a simple scraper showing you how to use PhantomJS with Ruby. Here's what it does:

  1. Visits the morph.io home page
  2. Because the headless browser is a small window it now has to click the button to open the navigation menu so we can see the search box
  3. It enters a search for "planningalerts" into the search box and clicks the submit button
  4. After waiting for the results to appear (see the important gotcha in the code comments!) it outputs the full names of all the scrapers on the search results page

Any questions? Hit up the help forum.

Contributors henare

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... -----> Ruby app detected -----> Compiling Ruby/Rack -----> Using Ruby version: ruby-2.0.0 -----> Installing dependencies using 1.7.12  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/.........  Fetching additional metadata from https://rubygems.org/..  Installing mini_portile 0.6.2  Installing mime-types 2.6.1  Installing cliver 0.3.2  Installing rack 1.6.4  Using bundler 1.7.12  Installing multi_json 1.11.2  Installing websocket-extensions 0.1.2  Installing rack-test 0.6.3  Installing websocket-driver 0.6.1  Installing nokogiri 1.6.6.2  Installing xpath 2.0.0  Installing capybara 2.4.4  Installing poltergeist 1.6.0  Your bundle is complete!  Gems in the groups development and test were not installed.  It was installed into ./vendor/bundle  Bundle completed (31.15s)  Cleaning up the bundler cache.  -----> Discovering process types  Procfile declares types -> scraper  Default process types for Ruby -> rake, console, web Error removing intermediate container 21a73db67f8b: Driver aufs failed to remove root filesystem 21a73db67f8b0029ac8a7cfbd4abd713f2698c3837935fe9b3a4d3cbbd122a13: rename /var/lib/docker/aufs/diff/21a73db67f8b0029ac8a7cfbd4abd713f2698c3837935fe9b3a4d3cbbd122a13 /var/lib/docker/aufs/diff/21a73db67f8b0029ac8a7cfbd4abd713f2698c3837935fe9b3a4d3cbbd122a13-removing: device or resource busy Injecting scraper and running... henare/planningalerts_scraperwiki_scrapers planningalerts-scrapers/kiama CloCkWeRX/sorell planningalerts-scrapers/sorell handelaar/ie_planningalerts_corkcity planningalerts-scrapers/moreland planningalerts-scrapers/muswellbrook planningalerts-scrapers/nillumbik planningalerts-scrapers/blue-mountains planningalerts-scrapers/Bega_Valley_Applications_on_Exhibition

Statistics

Average successful run time: 1 minute

Total run time: 4 minutes

Total cpu time used: less than 20 seconds

Total disk space used: 23.1 KB

History

  • Manually ran revision 84f98f06 and completed successfully .
    nothing changed in the database
  • Manually ran revision 84f98f06 and completed successfully .
    nothing changed in the database
    16 pages scraped
  • Manually ran revision f8c1dde9 and completed successfully .
    nothing changed in the database
    16 pages scraped
  • Created on morph.io