blimpage / sash_windows

Sash windows for sale on renovaterestorerecycle.com.au


Oh hi! This is the code for a scraper that runs daily on Morph.io. It finds all of the sash windows available for sale at https://renovaterestorerecycle.com.au/ and https://www.renovatorsparadise.com.au/, and sends an email (via Sendgrid) to a nominated email address if any new windows are found.

If that sounds incredibly niche and not very interesting, that's because it is!

Contributors blimpage

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby/Rack -----> Using Ruby version: ruby-2.6.2 -----> Installing dependencies using bundler version 1.17.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/...........  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.17.2  Fetching unf_ext 0.0.7.1  Fetching httpclient 2.6.0.1  Fetching mime-types 2.5  Installing mime-types 2.5  Installing httpclient 2.6.0.1  Installing unf_ext 0.0.7.1 with native extensions  Fetching net-http-digest_auth 1.4  Fetching net-http-persistent 2.9.4  Installing net-http-persistent 2.9.4  Fetching mini_portile2 2.4.0  Installing mini_portile2 2.4.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Fetching webrobots 0.1.1  Installing webrobots 0.1.1  Fetching mustermann 1.0.3  Installing net-http-digest_auth 1.4  Installing mustermann 1.0.3  Fetching rack 2.0.7  Installing rack 2.0.7  Fetching ruby_http_client 3.3.0  Installing ruby_http_client 3.3.0  Fetching sqlite3 1.3.10  Installing sqlite3 1.3.10 with native extensions  Fetching tilt 2.0.9  Installing tilt 2.0.9  Fetching nokogiri 1.10.8  Installing nokogiri 1.10.8 with native extensions  Fetching rack-protection 2.0.5  Installing rack-protection 2.0.5  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching sinatra 2.0.5  Installing sinatra 2.0.5  Fetching domain_name 0.5.24  Installing domain_name 0.5.24  Fetching sendgrid-ruby 5.3.0  Installing sendgrid-ruby 5.3.0  Fetching http-cookie 1.0.2  Installing http-cookie 1.0.2  Fetching sqlite_magic 0.0.3  Installing sqlite_magic 0.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.3  Installing mechanize 2.7.3  Bundle complete! 3 Gemfile dependencies, 24 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into `./vendor/bundle`  Removing bundler (1.15.2)  Bundle completed (23.10s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Scraping renovaterestorerecycle.com.au: Scraping page: https://www.renovaterestorerecycle.com.au/category/sash-windows/?post_type=rrr_stock 12 products found. Scraping page: https://www.renovaterestorerecycle.com.au/category/sash-windows/page/2/?post_type=rrr_stock 12 products found. Scraping page: https://www.renovaterestorerecycle.com.au/category/sash-windows/page/3/?post_type=rrr_stock 6 products found. 3 pages scraped. 30 total products found. 3 available product(s) found. Scraping renovatorsparadise.com.au: Finding pages to scrape. /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/net/protocol.rb:44:in `connect_nonblock': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (certificate has expired) (OpenSSL::SSL::SSLError) from /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect' from /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/net/http.rb:996:in `connect' from /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/net/http.rb:930:in `do_start' from /app/vendor/ruby-2.6.2/lib/ruby/2.6.0/net/http.rb:925:in `start' from /app/vendor/bundle/ruby/2.6.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start' from /app/vendor/bundle/ruby/2.6.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:631:in `connection_for' from /app/vendor/bundle/ruby/2.6.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request' from /app/vendor/bundle/ruby/2.6.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:259:in `fetch' from /app/vendor/bundle/ruby/2.6.0/gems/mechanize-2.7.3/lib/mechanize.rb:440:in `get' from /app/scrape_rp.rb:13:in `get_products' from scraper.rb:40:in `<main>'

Data

Downloaded 2 times by blimpage

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (102 KB) Use the API

rows 10 / 419

name description price url
Pair of Victorian Single Sash Windows.
1185 wide x 2060 high.
$1090
Pair of Victorian Sidelight Sash Windows.
2 @ 2220 wide x 2145 high. Both centre sashes + smaller sidelight sashes open + close on each window. All the architrave ( internal + external sides ) is also included.
$3500
Obscure Etched Glass Single Sash Window.
1110 wide x 1935 high.
$490
House Lot of 3 Single Sash Windows.
3 @ 930 wide x 2145 high.
$690 each
Victorian Federation Single Sash Window.
900 wide x 1340 high.
$290
Sandblasted Obscure Glass Single Sash Window.
1025 wide x 1510 high.
$490
8 Light Colonial Single Sash Window.
1040 wide x 1550 high.
$240
Colonial Single Sash Window with Opening Fanlight.
995 wide x 2440 high.
$1250
Cricket Bat Victorian Single Sash Window.
1110 wide x 1995 high.
$550
4 Light Colonial Single Sash Window.
1100 wide x 1400 high.
$240

Statistics

Average successful run time: 2 minutes

Total run time: 1 day

Total cpu time used: 26 minutes

Total disk space used: 171 KB

History

  • Auto ran revision 476d08bb and failed .
    nothing changed in the database
  • Auto ran revision 476d08bb and failed .
    nothing changed in the database
  • Auto ran revision 476d08bb and failed .
    nothing changed in the database
  • Auto ran revision 476d08bb and failed .
    nothing changed in the database
  • Auto ran revision 476d08bb and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

sash_windows / scraper.rb