ianheggie-oaf / multiple_nsw_planning_lgas
NSW planning portal LGA DA Applications
NSW Planning LGAs
This scraper runs on Morph to scrape DA Applications for LGA's that use it.
As Date range filtering is currently broken, it initially searches for All types, but if it finds more than 5 pages it then filters by "DA Status = On Ehxhibition".
When LGA is not set, it currently shows 11,770 results with 1308 pages of 9 cards per page, so its early days so far.
To get started see the documentation
To run the scraper
bundle exec ruby scraper.rb
To run the tests
bundle exec rake
To run style and coding checks
bundle exec rubocop
Contributors
Last run completed successfully
.
Console output of last run
Injecting configuration and compiling...
Injecting scraper and running...
Scraping authorities: gunnedah
Collecting feed data for gunnedah, attempt: 1...
INITIAL URL: https://www.planningportal.nsw.gov.au/daexhibitions?field_daex_status_value=All&field_local_government_area_value=GUNNEDAH&field_application_type_value=All&field_panel_reference_number_value=&field_council_unique_number_value=&title=&field_exhibition_start_date_value=&field_exhibition_end_date_value=
Scraping page 1 for GUNNEDAH - All...
Saving record gunnedah - 93-95 Marquis Street, Gunnedah NSW 2380
Saving record gunnedah - 9 Eighth Division Memorial Avenue, Gunnedah NSW 2380
Saving record gunnedah - 8521 Oxley Highway, Gunnedah NSW 2380
Saving record gunnedah - 1693 Blue Vale Rd, Blue Vale NSW
Saving record gunnedah - View Street, Gunnedah NSW 2380
Saving record gunnedah - 29-31 Breeza St, Carroll NSW 2340
Saving record gunnedah - 107 Barber Street, Gunnedah NSW 2380
Saving record gunnedah - 107 George Street, Gunnedah NSW 2380
Saving record gunnedah - 42 Bindea Place, Gunnedah NSW 2380
NEXT URL: https://www.planningportal.nsw.gov.au/daexhibitions?field_daex_status_value=All&field_local_government_area_value=GUNNEDAH&field_application_type_value=All&field_panel_reference_number_value=&field_council_unique_number_value=&title=&field_exhibition_start_date_value=&field_exhibition_end_date_value=&page=1
Scraping page 2 for GUNNEDAH - All...
Saving record gunnedah - 73 Maitland Street, Gunnedah NSW 2380
Saving record gunnedah - 23 Palmer Crescent, Gunnedah NSW 2380
Saving record gunnedah - 21 George Street, Gunnedah NSW 2380
Saving record gunnedah - 2-42 Kamilaroi Highway, Gunnedah NSW 2380
Saving record gunnedah - 12 Parkview Drive, Gunnedah NSW 2380
Saving record gunnedah - 160 Wandobah Road, Gunnedah NSW 2380
Saving record gunnedah - 100B Beulah Street, Gunnedah NSW 2380
Saving record gunnedah - 1258 The Dip Rd, Curlewis NSW
Saving record gunnedah - 131 Quia Road, Gunnedah NSW 2380
NEXT URL: https://www.planningportal.nsw.gov.au/daexhibitions?field_daex_status_value=All&field_local_government_area_value=GUNNEDAH&field_application_type_value=All&field_panel_reference_number_value=&field_council_unique_number_value=&title=&field_exhibition_start_date_value=&field_exhibition_end_date_value=&page=2
Scraping page 3 for GUNNEDAH - All...
Saving record gunnedah - 277 Conadilly Street, Gunnedah NSW 2380
Saving record gunnedah - 25 Booloocooroo Road, Gunnedah NSW 2380
Saving record gunnedah - 170 Bushs Lane, Gunnedah NSW 2380
Saving record gunnedah - 309-329 Conadilly Street, Gunnedah NSW 2380
Saving record gunnedah - 49-75 Finch Street, Gunnedah NSW 2380
Saving record gunnedah - 44 Booloocooroo Road, Gunnedah NSW 2380
Saving record gunnedah - 34-36 Silversmith Place, Gunnedah NSW 2380
Saving record gunnedah - 64 Kerry Elizabeth Drive, Gunnedah NSW 2380
Saving record gunnedah - 8 Goran St, Curlewis NSW
NEXT URL: https://www.planningportal.nsw.gov.au/daexhibitions?field_daex_status_value=All&field_local_government_area_value=GUNNEDAH&field_application_type_value=All&field_panel_reference_number_value=&field_council_unique_number_value=&title=&field_exhibition_start_date_value=&field_exhibition_end_date_value=&page=3
Scraping page 4 for GUNNEDAH - All...
Saving record gunnedah - 76A Beulah Street, Gunnedah NSW 2380
Saving record gunnedah - 18-22 Kamilaroi Road, Gunnedah NSW 2380
Saving record gunnedah - 150 Little Barber Street, Gunnedah NSW 2380
Saving record gunnedah - 418 Quia Road, Gunnedah NSW 2380
Saving record gunnedah - 6B Parkview Drive, Gunnedah NSW 2380
Saving record gunnedah - 27 Booloocooroo Road, Gunnedah NSW 2380
Saving record gunnedah - 37 Wattle Street, Gunnedah NSW 2380
Saving record gunnedah - 35 Wentworth Street, Gunnedah NSW 2380
Saving record gunnedah - 30 Galway Avenue, Gunnedah NSW 2380
NEXT URL: https://www.planningportal.nsw.gov.au/daexhibitions?field_daex_status_value=All&field_local_government_area_value=GUNNEDAH&field_application_type_value=All&field_panel_reference_number_value=&field_council_unique_number_value=&title=&field_exhibition_start_date_value=&field_exhibition_end_date_value=&page=4
Scraping page 5 for GUNNEDAH - All...
Saving record gunnedah - 617 Beeson Road, Gunnedah NSW 2380
Saving record gunnedah - 341-347 Conadilly Street, Gunnedah NSW 2380
Saving record gunnedah - 24-42 Goodwin Road, Gunnedah NSW 2380
Saving record gunnedah - 334 Pownall Rd, Mullaley NSW 2379
Saving record gunnedah - 2 Vera Close, Gunnedah NSW 2380
Saving record gunnedah - 54 Hunter Street, Gunnedah NSW 2380
Saving record gunnedah - 90 Marquis Street, Gunnedah NSW 2380
Saving record gunnedah - 809 Oakey Creek Rd, Piallaway NSW 2342
Saving record gunnedah - Modification of Consent NSW
NEXT URL: https://www.planningportal.nsw.gov.au/daexhibitions?field_daex_status_value=All&field_local_government_area_value=GUNNEDAH&field_application_type_value=All&field_panel_reference_number_value=&field_council_unique_number_value=&title=&field_exhibition_start_date_value=&field_exhibition_end_date_value=&page=5
Scraping page 6 for GUNNEDAH - All...
Saving record gunnedah - 12 Allgayer Drive, Gunnedah NSW 2380
MORPH_EXPECT_BAD=
Scraping Summary:
Authority OK Bad Exception
-------------------- ------ ------ --------------------------------------------------
gunnedah 46 0 -
Exiting with OK status!
Data
Downloaded
2340 times
by
To download data sign in with GitHub
rows 10 / 46
| council_reference | address | description | info_url | date_scraped | authority_label |
|---|---|---|---|---|---|
|
2024/031.002
|
93-95 Marquis Street, Gunnedah NSW 2380
|
24/7 Gym Modification
|
2025-05-31
|
gunnedah
|
|
|
2025.028
|
9 Eighth Division Memorial Avenue, Gunnedah NSW 2380
|
Shed
|
2025-05-31
|
gunnedah
|
|
|
2025.025
|
8521 Oxley Highway, Gunnedah NSW 2380
|
Dwelling
|
2025-05-31
|
gunnedah
|
|
|
2025.019
|
1693 Blue Vale Rd, Blue Vale NSW
|
Temporary Event Infrastructure
|
2025-05-31
|
gunnedah
|
|
|
2025.011
|
View Street, Gunnedah NSW 2380
|
Circus
|
2025-05-31
|
gunnedah
|
|
|
2024/098
|
29-31 Breeza St, Carroll NSW 2340
|
Shed
|
2025-05-31
|
gunnedah
|
|
|
202/2017.002
|
107 Barber Street, Gunnedah NSW 2380
|
Mod to Stage Development
|
2025-05-31
|
gunnedah
|
|
|
2024.095
|
107 George Street, Gunnedah NSW 2380
|
Dual Occupancy & Subdivision
|
2025-05-31
|
gunnedah
|
|
|
2024.097
|
42 Bindea Place, Gunnedah NSW 2380
|
Dwelling, Garage & Pool
|
2025-05-31
|
gunnedah
|
|
|
2025.012
|
73 Maitland Street, Gunnedah NSW 2380
|
Dwelling
|
2025-05-31
|
gunnedah
|
To download data sign in with GitHub
rows 10 / 26
| run_at | attempt | authority_label | records_saved | unprocessable_records | status | error_message | error_class | error_backtrace |
|---|---|---|---|---|---|---|---|---|
|
2025-05-06T21:03:06+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-07T23:26:12+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-08T20:29:10+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-09T03:30:10+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-10T11:24:15+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-11T02:33:11+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-12T02:03:12+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-13T23:44:11+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-14T17:13:42+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
|
2025-05-15T19:12:41+00:00
|
1
|
gunnedah
|
45
|
0
|
successful
|
|
|
|
To download data sign in with GitHub
rows 10 / 26
| run_at | attempt | duration | successful | failed | interrupted | successful_count | interrupted_count | failed_count | public_ip |
|---|---|---|---|---|---|---|---|---|---|
|
2025-05-06T21:03:06+00:00
|
1
|
8.9
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-07T23:26:12+00:00
|
1
|
8.6
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-08T20:29:10+00:00
|
1
|
7.9
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-09T03:30:10+00:00
|
1
|
7.7
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-10T11:24:15+00:00
|
1
|
8.7
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-11T02:33:11+00:00
|
1
|
8.3
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-12T02:03:12+00:00
|
1
|
8.7
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-13T23:44:11+00:00
|
1
|
7.9
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-14T17:13:42+00:00
|
1
|
8.0
|
gunnedah
|
|
|
1
|
0
|
0
|
|
|
2025-05-15T19:12:41+00:00
|
1
|
8.1
|
gunnedah
|
|
|
1
|
0
|
0
|
|
Statistics
Average successful run time: less than 20 seconds
Total run time: 8 minutes
Total cpu time used: 1 minute
Total disk space used: 495 KB
History
-
Auto
ran
revision
24915c2b
and
completed successfully
.
48 records added, 46 records removed in the database
-
Auto
ran
revision
24915c2b
and
completed successfully
.
48 records added, 46 records removed in the database
-
Auto
ran
revision
24915c2b
and
completed successfully
.
48 records added, 46 records removed in the database
-
Auto
ran
revision
24915c2b
and
completed successfully
.
48 records added, 46 records removed in the database
-
Auto
ran
revision
24915c2b
and
completed successfully
.
48 records added, 46 records removed in the database
- ...
- Created on morph.io