blimpage / sash_windows

Sash windows for sale on renovaterestorerecycle.com.au


Oh hi! This is the code for a scraper that runs daily on Morph.io. It finds all of the sash windows available for sale at https://renovaterestorerecycle.com.au/ and https://www.renovatorsparadise.com.au/, and sends an email (via Sendgrid) to a nominated email address if any new windows are found.

If that sounds incredibly niche and not very interesting, that's because it is!

Contributors blimpage

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby/Rack -----> Using Ruby version: ruby-2.6.2 -----> Installing dependencies using bundler version 1.17.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/...........  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.17.2  Fetching unf_ext 0.0.7.1  Fetching httpclient 2.6.0.1  Fetching mime-types 2.5  Installing mime-types 2.5  Fetching net-http-digest_auth 1.4  Installing unf_ext 0.0.7.1 with native extensions  Installing httpclient 2.6.0.1  Installing net-http-digest_auth 1.4  Fetching net-http-persistent 2.9.4  Installing net-http-persistent 2.9.4  Fetching mini_portile2 2.4.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Installing mini_portile2 2.4.0  Fetching webrobots 0.1.1  Fetching mustermann 1.0.3  Installing mustermann 1.0.3  Installing webrobots 0.1.1  Fetching rack 2.0.7  Fetching ruby_http_client 3.3.0  Installing rack 2.0.7  Installing ruby_http_client 3.3.0  Fetching sqlite3 1.3.10  Installing sqlite3 1.3.10 with native extensions  Fetching tilt 2.0.9  Installing tilt 2.0.9  Fetching nokogiri 1.10.8  Installing nokogiri 1.10.8 with native extensions  Fetching rack-protection 2.0.5  Installing rack-protection 2.0.5  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching sinatra 2.0.5  Installing sinatra 2.0.5  Fetching domain_name 0.5.24  Installing domain_name 0.5.24  Fetching sendgrid-ruby 5.3.0  Installing sendgrid-ruby 5.3.0  Fetching http-cookie 1.0.2  Installing http-cookie 1.0.2  Fetching sqlite_magic 0.0.3  Installing sqlite_magic 0.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.3  Installing mechanize 2.7.3  Bundle complete! 3 Gemfile dependencies, 24 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into `./vendor/bundle`  Removing bundler (1.15.2)  Bundle completed (29.04s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Scraping renovaterestorerecycle.com.au: Scraping page: https://www.renovaterestorerecycle.com.au/category/sash-windows/?post_type=rrr_stock 12 products found. Scraping page: https://www.renovaterestorerecycle.com.au/category/sash-windows/page/2/?post_type=rrr_stock 12 products found. Scraping page: https://www.renovaterestorerecycle.com.au/category/sash-windows/page/3/?post_type=rrr_stock 7 products found. 3 pages scraped. 31 total products found. 2 available product(s) found. Scraping renovatorsparadise.com.au: Finding pages to scrape. 7 page(s) of interest found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/single-sash-window/interwar-sash/interwar-0-1395-mm-high/ 26 product(s) found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/single-sash-window/interwar-sash/interwar-1400-1595-high/ 25 product(s) found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/single-sash-window/interwar-sash/interwar-1600-1895-high/ 9 product(s) found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/single-sash-window/interwar-sash/interwar-1900-mm-high/ 16 product(s) found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/single-sash-window/leadlight-sash/ 14 product(s) found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/sash-window/ 28 product(s) found. Scraping page: https://www.renovatorsparadise.com.au/product-category/windows/triple-quadruple-sash/ 39 product(s) found. 7 pages scraped. 157 total products found. No new products found. Oh well. Seeya.

Data

Downloaded 2 times by blimpage

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (126 KB) Use the API

rows 10 / 512

name description price url
Pair of Victorian Single Sash Windows.
1185 wide x 2060 high.
$1090
Pair of Victorian Sidelight Sash Windows.
2 @ 2220 wide x 2145 high. Both centre sashes + smaller sidelight sashes open + close on each window. All the architrave ( internal + external sides ) is also included.
$3500
Obscure Etched Glass Single Sash Window.
1110 wide x 1935 high.
$490
House Lot of 3 Single Sash Windows.
3 @ 930 wide x 2145 high.
$690 each
Victorian Federation Single Sash Window.
900 wide x 1340 high.
$290
Sandblasted Obscure Glass Single Sash Window.
1025 wide x 1510 high.
$490
8 Light Colonial Single Sash Window.
1040 wide x 1550 high.
$240
Colonial Single Sash Window with Opening Fanlight.
995 wide x 2440 high.
$1250
Cricket Bat Victorian Single Sash Window.
1110 wide x 1995 high.
$550
4 Light Colonial Single Sash Window.
1100 wide x 1400 high.
$240

Statistics

Average successful run time: 2 minutes

Total run time: 1 day

Total cpu time used: 30 minutes

Total disk space used: 199 KB

History

  • Auto ran revision 476d08bb and completed successfully .
    nothing changed in the database
  • Auto ran revision 476d08bb and completed successfully .
    nothing changed in the database
  • Auto ran revision 476d08bb and completed successfully .
    5 records added in the database
  • Auto ran revision 476d08bb and completed successfully .
    nothing changed in the database
  • Auto ran revision 476d08bb and completed successfully .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

sash_windows / scraper.rb