ianheggie-oaf / spear

Streamlined Planning through Electronic Applications and Referrals Victoria


SPEAR - Streamlined Planning through Electronic Applications and Referrals, Victoria Scraper

  • Cookie tracking - No
  • Pagnation - yes, start row based via API
  • Javascript - No
  • Clearly defined data within a row - Partial - some data is found via details page

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Optionally set MORPH_AUTHORITIES to a comma seperated list of Authorities to examine to speed up development.

Scraper pauses between requests to be nice to the server.

Expected output

Injecting configuration and compiling...
Injecting scraper and running...
Getting list of Authorities...
Authenticating with SPEAR API...
Found 81 authorities...

Getting applications for Alpine Shire Council - Bright Office...
Getting applications for Bass Coast Shire Council...
Saving S262573E - 16 HAZELWOOD ROAD, SAN REMO VIC 3925 ...
Saving S262397S - 45 RUTTLE LANE, INVERLOCH VIC 3996 ...
Saving S262141V - HESLOP ROAD, NORTH WONTHAGGI VIC 3995 ...
Saving S262019P - 7 MARION COURT, INVERLOCH VIC 3996 ...
Saving S261931S - 15 STANLEY ROAD, GRANTVILLE VIC 3984 ...
Getting applications for Baw Baw Shire Council...
Saving S262533H - 409 COPELANDS ROAD, WARRAGUL VIC 3820 ...
Saving S261836C - 65 ROLLO STREET, YARRAGON VIC 3823 ...
Getting applications for Benalla Rural City Council...
WARNING: nothing found between 28 and 60 ago! previous SPEAR record dated 2025-11-26
...

Count  Authority
-----  --------------------------------------
    0  Alpine Shire Council - Bright Office
    5  Bass Coast Shire Council
    2  Baw Baw Shire Council
       Benalla Rural City Council [Not in use since 2025-11-26]
    ...

Finished!

Execution time: 20 to 30 minutes

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors mlandauer ianheggie-oaf benrfairless archoo CloCkWeRX henare jamasalbertastash

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting list of Authorities... Authenticating with SPEAR API... scraper.rb:196:in `run': undefined method `size' for nil:NilClass (NoMethodError) puts "Found #{authorities['data'].size} authorities..." ^^^^^ from scraper.rb:270:in `<main>'

Data

Downloaded 9918 times by ianheggie-oaf

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (448 KB) Use the API

rows 10 / 1353

council_reference address description info_url date_scraped date_received
S266439T
309 GREENHALGHS ROAD, DELACOMBE VIC 3356
The application proposes the subdivision of the land into four residential lots within an established urban area and removal of trees.
2026-04-28
2026-03-31
S266392T
2 POPLAR STREET, WENDOUREE VIC 3355
Certification of a Plan under the Subdivision Act. Dealing Type Section 22 (Subdivision) (PS938413T) 2 lots
2026-04-28
2026-03-31
S266410A
14 DOUGLAS STREET, ROSANNA VIC 3084
4 lot subdivision in accordance with development permit (P552/2025) and its associated endorsed plans.
2026-04-28
2026-03-31
S266405H
21 ROBINHOOD ROAD, IVANHOE EAST VIC 3079
2 lot subdivision
2026-04-28
2026-03-31
S266446B
40 GRAHAM STREET, WONTHAGGI VIC 3995
Certification of a Plan under the Subdivision Act. Dealing Type Section 22 (Consolidation) (PC383194H) 1 lot
2026-04-28
2026-03-31
S265562H
15 PEVERILL CRESCENT, WONTHAGGI VIC 3995
Certification of a Plan under the Subdivision Act. Dealing Type Section 37 (Subdivision - Staged) (PS934206V/S2) 2 lots
2026-04-28
2026-03-31
S265443B
30 BREEZE COURT, SAN REMO VIC 3925
Removal of Easements: E-3, E-4 & E-9 Variation of Easement: E-10
2026-04-28
2026-03-31
S266411T
8 LOUISE STREET, BRIGHTON EAST VIC 3187
2 lot building subdivision
2026-04-28
2026-03-31
S266153J
1/189 POINT LONSDALE ROAD, POINT LONSDALE VIC 3225
Certification of a Plan under the Subdivision Act. Dealing Type Section 32A (Subdivision) (PS938713F) 2 lots
2026-04-28
2026-03-31
S266301J
20 JEFFREY STREET, HAMPTON PARK VIC 3976
2 lot land subdivision. Client would like to complete subdivision prior to construction of the new unit at the rear of the site through the use of 173 agreements with Council. Please ensure subdivision permit has notation to deferring works through use of 173 agreements where applicable.
2026-04-28
2026-03-31

Statistics

Average successful run time: 43 minutes

Total run time: 2 days

Total cpu time used: 8 minutes

Total disk space used: 516 KB

History

  • Auto ran revision 7c8b34f7 and failed .
    nothing changed in the database
  • Auto ran revision 7c8b34f7 and completed successfully .
    603 records added, 595 records removed in the database
  • Auto ran revision 7c8b34f7 and completed successfully .
    601 records added, 582 records removed in the database
  • Auto ran revision 7c8b34f7 and completed successfully .
    606 records added, 557 records removed in the database
  • Auto ran revision 7c8b34f7 and completed successfully .
    582 records added, 607 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

spear / scraper.rb