ianheggie-oaf / joondalup

City of Joondalup Open Development Applications


City of Joondalup - Open Development Applications Scraper

  • Cookie tracking - No
  • Pagnation - yes, via a flag in the HTML returned in the JSON data
  • JavaScript - Yes, but we don't need to execute it, instead call the api directly
  • Clearly defined data within a row - No, data is in HTML in a JSON record, and reference in the details page
  • System - custom

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Expected output

Getting initial page
  Pausing 1.023s
Getting page 1
/home/ianh/.local/share/mise/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated.
  Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/31b-strathyre-drive,-duncraig-–-single-house-(additions)
  Extracted DA25/0869 from detail page
Saving record DA25/0869 - 31B Strathyre Drive, Duncraig, WA
...
  Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/51a-conidae-drive,-heathridge-–-grouped-dwelling-(new-dwelling)
  Extracted DA25/0774 from detail page
Saving record DA25/0774 - 51A Conidae Drive, Heathridge, WA
Deleting 0 applications scraped between  and 2025-12-28
  Running VACUUM to reclaim space...
Finished! Added 12 applications, and skipped 0 unprocessable applications from 1 pages.

Execution time: ~ 11 seconds

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors ianheggie-oaf

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting initial page Pausing 2.093s Getting page 1 /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/58-bernard-manning-drive,-duncraig-–-grouped-dwelling-(two-new-dwellings) Extracted DA26/0214 from detail page Saving record DA26/0214 - 58 Bernard Manning Drive, Duncraig, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/8-laurel-street,-mullaloo-–-single-house-(additions) Extracted DA26/0171 from detail page Saving record DA26/0171 - 8 Laurel Street, Mullaloo, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/1-beach-road,-marmion-–-single-house-(additions) Extracted DA26/0287 from detail page Saving record DA26/0287 - 1 Beach Road, Marmion, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/25c-glenmere-road,-warwick-–-unhosted-short-term-accommodation-(change-of-use) Extracted DA26/0351 from detail page Saving record DA26/0351 - 25C Glenmere Road, Warwick, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/41-amalfi-drive-hillarys-–-unhosted-short-term-rental-accommodation-(change-of-use) Extracted DA26/0284 from detail page Saving record DA26/0284 - 41 Amalfi Drive Hillarys, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/23-achilles-loop,-iluka-–-single-house-(siteworks) Extracted DA26/0369 from detail page Saving record DA26/0369 - 23 Achilles Loop, Iluka, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/32-woodlake-retreat-kingsley-–-unhosted-short-term-rental-accommodation-(change-of-use) Extracted DA26/0290 from detail page Saving record DA26/0290 - 32 Woodlake Retreat Kingsley, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/14-manhattan-avenue,-iluka-–-single-house-(outbuilding-addition) Extracted DA26/0200 from detail page Saving record DA26/0200 - 14 Manhattan Avenue, Iluka, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/7-ayton-way,-duncraig-–-single-house-(additions-modification-to-previous-approved-da24-0598) Extracted DA24/0589.01 from detail page Saving record DA24/0589.01 - 7 Ayton Way, Duncraig, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/2-drysdale-road,-craigie-–-grouped-dwelling-(three-new-two-storey-dwellings) Extracted DA26/0167 from detail page Saving record DA26/0167 - 2 Drysdale Road, Craigie, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/6-gregory-ave,-padbury-–-grouped-dwelling-(new-two-storey-dwelling) Extracted DA26/0365 from detail page Saving record DA26/0365 - 6 Gregory Avenue, Padbury, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/30-fleetwing-heights,-ocean-reef-–-single-house-(additions) Extracted DA26/0220 from detail page Saving record DA26/0220 - 30 Fleetwing Heights, Ocean Reef, WA Finished! Added 12 applications, and skipped 0 unprocessable applications from 1 pages.

Data

Downloaded 10250 times by ianheggie-oaf

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (32 KB) Use the API

rows 10 / 36

council_reference address description info_url date_scraped on_notice_from on_notice_to
DA25/0249.01
Bar Ole, 98 O’Mara Boulevard, Iluka, WA
Small bar (additions) State Administrative Tribunal – Section 31 Reconsideration
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/bar-ole,-98-o’mara-boulevard,-iluka-small-bar-(additions)-state-administrative-tribunal-–-section
2026-05-05
2026-04-21
2026-05-05
DA26/0050
56 Stillwater Way, Edgewater, WA
Single house (new two storey dwelling)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/56-stillwater-way,-edgewater-–-single-house-(new-two-storey-dwelling)
2026-05-06
2026-04-24
2026-05-08
DA26/0042
63 Davallia Road, Duncraig, WA
Single house (additions and site works)
2026-05-06
2026-04-22
2026-05-06
DA25/0949
67 Mooring Crescent, Ocean Reef, WA
Single House (Additions)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/67-mooring-crescent,-ocean-reef-–-single-house-(additions)
2026-05-07
2026-05-01
2026-05-15
DA26/0165
2 Snowbird Gardens, Joondalup, WA
Holiday accommodation (change of use)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/2-snowbird-gardens,-joondalup-–-holiday-accommodation-(change-of-use)
2026-05-14
2026-05-01
2026-05-15
DA26/0148
9 Milson Court, Kallaroo, WA
Single House (Privacy Screen – Retrospective)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/9-milson-court,-kallaroo-–-single-house-(privacy-screen-–-retrospective)
2026-05-14
2026-05-01
2026-05-15
DA26/0029
1 Janthina Crescent, Heathridge, WA
Grouped dwelling (three new dwellings)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/1-janthina-crescent,-heathridge-–-grouped-dwelling-(three-new-dwellings)
2026-05-14
2026-05-01
2026-05-15
DA26/0262
27A Baxter Way, Padbury, WA
Grouped dwelling (new two storey dwelling)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/27a-baxter-way,-padbury-–-grouped-dwelling-(new-two-storey-dwelling)
2026-05-17
2026-05-04
2026-05-18
DA26/0021
27 Mykonos View, Iluka, WA
Unhosted short term rental accommodation (change of use – retrospective)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/27-mykonos-view,-iluka-unhosted-short-term-rental-accommodation-(change-of-use-–-retrospective)
2026-05-18
2026-05-04
2026-05-18
DA26/0013
16 Cambria Street, Kallaroo, WA
Grouped dwelling (three new two storey dwellings)
2026-05-18
2026-05-04
2026-05-18

Statistics

Average successful run time: less than 20 seconds

Total run time: 16 minutes

Total cpu time used: 1 minute

Total disk space used: 66.9 KB

History

  • Auto ran revision de14a479 and completed successfully .
    12 records added, 12 records removed in the database
  • Auto ran revision de14a479 and completed successfully .
    12 records added, 14 records removed in the database
  • Auto ran revision de14a479 and completed successfully .
    12 records added, 9 records removed in the database
  • Auto ran revision de14a479 and completed successfully .
    12 records added, 12 records removed in the database
  • Auto ran revision de14a479 and completed successfully .
    12 records added, 15 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

joondalup / scraper.rb