planningalerts-scrapers / joondalup

City of Joondalup Open Development Applications

README.md

City of Joondalup - Open Development Applications Scraper

  • Cookie tracking - No
  • Pagnation - yes, via a flag in the HTML returned in the JSON data
  • JavaScript - Yes, but we don't need to execute it, instead call the api directly
  • Clearly defined data within a row - No, data is in HTML in a JSON record, and reference in the details page
  • System - custom

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Expected output

Getting initial page
  Pausing 1.023s
Getting page 1
/home/ianh/.local/share/mise/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated.
  Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/31b-strathyre-drive,-duncraig-–-single-house-(additions)
  Extracted DA25/0869 from detail page
Saving record DA25/0869 - 31B Strathyre Drive, Duncraig, WA
...
  Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/51a-conidae-drive,-heathridge-–-grouped-dwelling-(new-dwelling)
  Extracted DA25/0774 from detail page
Saving record DA25/0774 - 51A Conidae Drive, Heathridge, WA
Deleting 0 applications scraped between  and 2025-12-28
  Running VACUUM to reclaim space...
Finished! Added 12 applications, and skipped 0 unprocessable applications from 1 pages.

Execution time: ~ 11 seconds

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors ianheggie-oaf

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting initial page Pausing 1.955s Getting page 1 /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/23-st-james-approach,-iluka-–-unhosted-short-term-rental-accommodation-(change-of-use) Extracted DA25/1001 from detail page Saving record DA25/1001 - 23 St James Approach, Iluka, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/10-doveridge-drive,-duncraig-–-grouped-dwelling-(two-new-dwellings) Extracted DA25/0890 from detail page Saving record DA25/0890 - 10 Doveridge Drive, Duncraig, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/19-lakeside-drive,-joondalup-–-grouped-dwelling-(two-new-two-storey-dwellings) Extracted DA25/0924 from detail page Saving record DA25/0924 - 19 Lakeside Drive, Joondalup, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/14-tipperary-rise,-padbury-grouped-dwelling-(two-new-two-storey-dwellings) Extracted DA25/0773 from detail page Saving record DA25/0773 - 14 Tipperary Rise, Padbury, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/27-hunter-way,-padbury-–-grouped-dwelling-(two-new-dwellings) Unable to extract reference from: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/27-hunter-way,-padbury-–-grouped-dwelling-(two-new-dwellings) Saving record 27 Hunter Way Padbury Grouped dwelling two new dw- - 27 Hunter Way, Padbury, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/12-tenerife-boulevard,-hillarys-–-un-hosted-short-term-accommodation-(change-of-use) Extracted DA25/0982 from detail page Saving record DA25/0982 - 12 Tenerife Boulevard, Hillarys, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/15a-marine-terrace,-sorrento-–-single-house-(new-two-storey-dwelling) Extracted DA25/0902 from detail page Saving record DA25/0902 - 15a Marine Terrace, Sorrento, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/9-tottenham-road,-joondalup-–-multiple-dwellings-(five-new-dwellings) Extracted DA26/0001 from detail page Saving record DA26/0001 - 9 Tottenham Road, Joondalup, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/1-delcomyn-place,-craigie-–-grouped-dwelling-(two-new-two-storey-dwellings-one-new-dwelling) Extracted DA25/0928 from detail page Saving record DA25/0928 - 1 Delcomyn Place, Craigie, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/50-laurel-street,-mullaloo-–-single-house-(new-two-storey-dwelling) Extracted DA25/0995 from detail page Saving record DA25/0995 - 50 Laurel Street, Mullaloo, WA Fetching detail page: https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/26-hawkins-avenue,-sorrento-–-grouped-dwelling-(new-two-storey-dwelling-with-undercroft) Extracted DA25/0868 from detail page Saving record DA25/0868 - 26 Hawkins Avenue, Sorrento, WA Finished! Added 11 applications, and skipped 0 unprocessable applications from 1 pages.

Data

Downloaded 29 times by dcalde Manasaanand mlandauer

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (12 KB) Use the API

rows 10 / 11

council_reference address description info_url date_scraped on_notice_from on_notice_to
DA25/1001
23 St James Approach, Iluka, WA
Unhosted short-term rental accommodation (change of use)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/23-st-james-approach,-iluka-–-unhosted-short-term-rental-accommodation-(change-of-use)
2026-02-16
2026-02-16
2026-03-02
DA25/0890
10 Doveridge Drive, Duncraig, WA
Grouped dwelling (two new dwellings)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/10-doveridge-drive,-duncraig-–-grouped-dwelling-(two-new-dwellings)
2026-02-16
2026-02-13
2026-02-27
DA25/0924
19 Lakeside Drive, Joondalup, WA
Grouped dwelling (two new two storey dwellings)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/19-lakeside-drive,-joondalup-–-grouped-dwelling-(two-new-two-storey-dwellings)
2026-02-16
2026-02-13
2026-02-27
DA25/0773
14 Tipperary Rise, Padbury, WA
Grouped dwelling (two new two storey dwellings)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/14-tipperary-rise,-padbury-grouped-dwelling-(two-new-two-storey-dwellings)
2026-02-16
2026-02-10
2026-02-24
27 Hunter Way Padbury Grouped dwelling two new dw-
27 Hunter Way, Padbury, WA
Grouped dwelling (two new dwellings)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/27-hunter-way,-padbury-–-grouped-dwelling-(two-new-dwellings)
2026-02-16
2026-02-10
2026-02-24
DA25/0982
12 Tenerife Boulevard, Hillarys, WA
Un-hosted short-term accommodation (change of use)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/12-tenerife-boulevard,-hillarys-–-un-hosted-short-term-accommodation-(change-of-use)
2026-02-16
2026-02-09
2026-02-23
DA25/0902
15a Marine Terrace, Sorrento, WA
single house (new two storey dwelling)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/15a-marine-terrace,-sorrento-–-single-house-(new-two-storey-dwelling)
2026-02-16
2026-02-09
2026-02-23
DA26/0001
9 Tottenham Road, Joondalup, WA
Multiple dwellings (five new dwellings)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/9-tottenham-road,-joondalup-–-multiple-dwellings-(five-new-dwellings)
2026-02-16
2026-02-09
2026-02-23
DA25/0928
1 Delcomyn Place, Craigie, WA
Grouped dwelling (two new two storey dwellings and one new dwelling)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/1-delcomyn-place,-craigie-–-grouped-dwelling-(two-new-two-storey-dwellings-one-new-dwelling)
2026-02-16
2026-02-02
2026-02-16
DA25/0995
50 Laurel Street, Mullaloo, WA
Single house (new two storey dwelling)
https://www.joondalup.wa.gov.au/community-and-spaces/community-consultation/50-laurel-street,-mullaloo-–-single-house-(new-two-storey-dwelling)
2026-02-16
2026-02-16
2026-03-02

Statistics

Average successful run time: less than 20 seconds

Total run time: less than 20 seconds

Total cpu time used: less than 5 seconds

Total disk space used: 43.8 KB

History

  • Manually ran revision de14a479 and completed successfully .
    11 records added in the database
  • Created on morph.io

Scraper code

Ruby

joondalup / scraper.rb