planningalerts-scrapers / rockingham

City of Rockingham Town planning advertising and submissions Scraper

README.md

City of Rockingham - Town planning advertising and submissions

  • Cookie tracking - No
  • Pagnation - none obvious
  • JavaScript - No
  • Clearly defined data within a row - Clear enough in HTML dom, with additional address details in free text in details page
  • System - custom

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Expected output

Getting listing page
  Pausing 4.057s
  Fetching detail page: https://rockingham.wa.gov.au/planning-and-building/local-planning/town-planning-advertising-and-submissions/proposed-oilseed-processing-plant-patterson-road,-
Saving record Proposed Oilseed Processing Plant Patterson Road - - Lot 9008 Patterson Road, East Rockingham, WA
...
  Pausing 3.815s
  Fetching detail page: https://rockingham.wa.gov.au/planning-and-building/local-planning/town-planning-advertising-and-submissions/proposed-holiday-house-emerald-court,-singleton
Saving record Proposed Holiday House Emerald Court Singleton - 38 Emerald Court, Singleton, WA
Deleting 0 applications scraped between  and 2025-12-28
  Running VACUUM to reclaim space...
Finished! Added 3 applications, and skipped 0 unprocessable applications.

Execution time: ~ 30 seconds

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors ianheggie-oaf benrfairless

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting listing page /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.14.0/lib/mechanize/http/agent.rb:340:in `fetch': 404 => Net::HTTPNotFound for https://rockingham.wa.gov.au/planning-and-building/local-planning/town-planning-advertising-and-submissions -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.14.0/lib/mechanize.rb:483:in `get' from scraper.rb:136:in `block in run' from scraper.rb:33:in `throttle_block' from scraper.rb:134:in `run' from scraper.rb:198:in `<main>'

Data

Downloaded 354 times by mlandauer dcalde Manasaanand 538digital Joel-Tan_valocity guptara123

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (12 KB) Use the API

rows 0 / 0

Statistics

Average successful run time: half a minute

Total run time: 37 minutes

Total cpu time used: 3 minutes

Total disk space used: 58.8 KB

History

  • Auto ran revision cc0b87d7 and failed .
    nothing changed in the database
  • Auto ran revision cc0b87d7 and failed .
    nothing changed in the database
  • Auto ran revision cc0b87d7 and failed .
    nothing changed in the database
  • Auto ran revision cc0b87d7 and failed .
    nothing changed in the database
  • Auto ran revision cc0b87d7 and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

rockingham / scraper.rb