ianheggie-oaf / joondalup

City of Joondalup Open Development Applications


City of Joondalup - Open Development Applications Scraper

  • Cookie tracking - No
  • Pagnation - yes, via a flag in the HTML returned in the JSON data
  • JavaScript - Yes, but we don't need to execute it, instead call the api directly
  • Clearly defined data within a row - No, data is in HTML in a JSON record, and reference in the details page
  • System - custom

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Expected output

Getting initial page
  Pausing 1.023s
Getting page 1
/home/ianh/.local/share/mise/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated.
  Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/31b-strathyre-drive,-duncraig-–-single-house-(additions)
  Extracted DA25/0869 from detail page
Saving record DA25/0869 - 31B Strathyre Drive, Duncraig, WA
...
  Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/51a-conidae-drive,-heathridge-–-grouped-dwelling-(new-dwelling)
  Extracted DA25/0774 from detail page
Saving record DA25/0774 - 51A Conidae Drive, Heathridge, WA
Deleting 0 applications scraped between  and 2025-12-28
  Running VACUUM to reclaim space...
Finished! Added 12 applications, and skipped 0 unprocessable applications from 1 pages.

Execution time: ~ 11 seconds

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors ianheggie-oaf benrfairless

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting initial page Getting page 1 Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/18-viewcrest-way%2C-sorrento-%E2%80%93-single-house-%28balcony-addition%29 Extracted DA26/0811 from detail page Saving record DA26/0811 - 18 Viewcrest Way, Sorrento, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/28-tenerife-boulevard%2C-hillarys-%E2%80%93-unhosted-short-term-rental-accommodation-%28change-of-use%29 Extracted DA26/0702 from detail page Saving record DA26/0702 - 28 Tenerife Boulevard, Hillarys, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/48-vista-close%2C-edgewater-%E2%80%93-un-hosted-short-term-rental-%28change-of-use%29 Extracted DA26/0616 from detail page Saving record DA26/0616 - 48 Vista Close, Edgewater, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/13-cavendish-avenue%2C-burns-beach-%E2%80%93-single-house-%28new-two-storey-dwelling%29 Extracted DA26/0750 from detail page Saving record DA26/0750 - 13 Cavendish Avenue, Burns Beach, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/19-injune-way%2C-joondalup-%E2%80%93-warehouse-storage-%28two-new-warehouses%29 Extracted DA26/0630 from detail page Saving record DA26/0630 - 19 Injune Way, Joondalup, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/33-willow-road%2C-warwick-%E2%80%93-grouped-dwelling-%28two-new-dwellings%29 Extracted DA26/0801 from detail page Saving record DA26/0801 - 33 Willow Road, Warwick, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/2-the-plaza%2C-sorrento-%28change-of-use-from-restaurant-cafe-to-tavern-and-alfresco-addition%29 Extracted DA26/0785 from detail page Warning - Unable to parse address and description from: 2 The Plaza, Sorrento (change of use from restaurant/café to tavern and alfresco addition) (skipped) Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/7-macbean-place%2C-duncraig-%E2%80%93-single-house-%28additions%29 Extracted DA26/0602 from detail page Saving record DA26/0602 - 7 MacBean Place, Duncraig, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/46a-craigie-drive%2C-craigie-%E2%80%93-grouped-dwelling-%28new-dwelling%29 Extracted DA26/0716 from detail page Saving record DA26/0716 - 46A Craigie Drive, Craigie, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/54-daytona-drive%2C-iluka-%E2%80%93-single-house-%28privacy-screen-%E2%80%93-retrospective%29 Extracted DA26/0790 from detail page Saving record DA26/0790 - 54 Daytona Drive, Iluka, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/168-beachside-drive%2C-burns-beach-%E2%80%93-single-house-%28new-two-storey-dwelling%29 Extracted DA26/0692 from detail page Saving record DA26/0692 - 168 Beachside Drive, Burns Beach, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/10-harrier-way%2C-beldon-%E2%80%93-single-house-%28additions%29 Extracted DA26/0736 from detail page Saving record DA26/0736 - 10 Harrier Way, Beldon, WA Deleting 2 applications scraped between 2026-10-22 and 2026-10-23 Finished! Added 11 applications, and skipped 1 unprocessable applications from 1 pages.

Data

Downloaded 26333 times by ianheggie-oaf tkrikke

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (28 KB) Use the API

rows 10 / 29

council_reference address description info_url date_scraped on_notice_from on_notice_to
DA26/0534
4 Roe Court, Padbury, WA
grouped dwelling (two new dwellings)
2026-10-26
2026-10-12
2026-10-26
DA26/0373
10 Idaho Place, Craigie, WA
grouped dwelling (four new two storey dwellings)
2026-10-26
2026-10-12
2026-10-26
DA26/0538
58 Sporing Way, Hillarys, WA
grouped dwelling (four new two storey dwellings)
2026-10-26
2026-10-12
2026-10-26
DA26/0658
19 Kutcharo Crescent, Joondalup, WA
grouped dwelling (new dwelling)
2026-10-27
2026-10-13
2026-10-27
DA26/0663
98 Camberwarra Drive, Craigie, WA
grouped dwelling (two new two storey dwellings)
2026-10-27
2026-10-13
2026-10-27
DA26/0418
27 Garners Way, Burns Beach, WA
home business (hairdresser)
2026-10-29
2026-10-15
2026-10-29
DA26/0603
94 Caridean Street, Heathridge, WA
Multiple Dwellings (8 New Dwellings)
2026-11-01
2026-10-05
2026-11-02
DA26/0460
7 Springfield Court, Kallaroo, WA
single house (additions)
2026-11-03
2026-10-21
2026-11-04
DA14/0565.03
31 Chadlington Drive, Padbury, WA
Child care premises (modifications to previously approved operating hours)
2026-11-05
2026-10-22
2026-11-05
DA25/0673.01
4 Martinique Mews Hillarys, WA
un-hosted short term rental accommodation
2026-11-09
2026-10-27
2026-11-10

Statistics

Average successful run time: less than 20 seconds

Total run time: about 1 hour

Total cpu time used: 5 minutes

Total disk space used: 2.18 MB

History

  • Auto ran revision c56eb95c and completed successfully .
    11 records added, 13 records removed in the database
  • Auto ran revision c56eb95c and completed successfully .
    11 records added, 11 records removed in the database
  • Auto ran revision c56eb95c and completed successfully .
    11 records added, 13 records removed in the database
  • Auto ran revision c56eb95c and completed successfully .
    11 records added, 11 records removed in the database
  • Auto ran revision c56eb95c and completed successfully .
    11 records added, 9 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

joondalup / scraper.rb