jamasalbertastash / multiple-civica

README.md

This is a scraper that runs on Morph. To get started see the documentation

To run the scraper

bundle exec ruby scraper.rb

To run the tests

bundle exec rake

To run style and coding checks

bundle exec rubocop

Contributors mlandauer tseyiokorodudu jamasalbertastash

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Scraping authorities: nambucca, cairns, orange, woollahra, lane_cove Collecting feed data for nambucca... Saving record 2/2023, 11A Albert Drive DONNELLYVILLE NSW 2447 Saving record 5/2023, 11A Albert Drive DONNELLYVILLE NSW 2447 Saving record 148/2021, 10 Banyandah Road HYLAND PARK NSW 2448 Saving record 381/2022, 27 Charles Place NAMBUCCA HEADS NSW 2448 Saving record 385/2022, 2 Giidany Close MACKSVILLE NSW 2447 Saving record 3/2023, 3 Nelson Street NAMBUCCA HEADS NSW 2448 Saving record 84/2022, 231 Rhones Creek Road TALARM NSW 2447 Saving record 2/2020, 31 River Street MACKSVILLE NSW 2447 Saving record 10/2023, Smith's Lane CONGARINNI NORTH NSW 2447 Saving record 6/2023, 21 Woodbell Street NAMBUCCA HEADS NSW 2448 Collecting feed data for cairns... Saving record 12114/2023, 23-51 Anderson Road WOREE QLD 4868 Saving record 12098/2023, 13 Bel-Air Drive WHITFIELD QLD 4870 Saving record 12105/2023, 20 Blue Water Lane TRINITY BEACH QLD 4879 Saving record 12101/2023, 593-619 Bruce Highway WOREE QLD 4868 Saving record 12109/2023, 59-61 Carruthers Street EDMONTON QLD 4869 Saving record 12103/2023, 20-24 Comport Street PORTSMITH QLD 4870 Saving record 12095/2023, 61-79 Cook Street PORTSMITH QLD 4870 Saving record 12100/2023, 28 Dillon Street BUNGALOW QLD 4870 Saving record 12090/2023, Ducrot Road BABINDA QLD 4861 Saving record 12108/2023, 34 East Parkridge Drive BRINSMEAD QLD 4870 Saving record 12099/2023, 1077R Kamerunga Road CARAVONICA QLD 4878 Saving record 12111/2023, 181-183 Kamerunga Road FRESHWATER QLD 4870 Saving record 12107/2023, 43 Laurel Avenue MOUNT SHERIDAN QLD 4868 Saving record 12112/2023, 18 Mango Drive EARLVILLE QLD 4870 Saving record 12106/2023, 41 Redden Street PORTSMITH QLD 4870 Saving record 12093/2023, 46 Redden Street PORTSMITH QLD 4870 Saving record 12097/2023, 142 Reed Road TRINITY PARK QLD 4879 Saving record 12096/2023, 2 Riverstone Road GORDONVALE QLD 4865 Saving record 12110/2023, 14 Sunbird Drive WOREE QLD 4868 Saving record 12094/2023, 34 Wiseman Road East EDMONTON QLD 4869 Collecting feed data for orange... Saving record 22/2023, 22 Alkira Way ORANGE NSW 2800 Saving record 29/2023, 8 Aloe Vera Place ORANGE NSW 2800 Saving record 12/2023, 225-243 Anson Street ORANGE NSW 2800 Saving record 9/2023, 121 Autumn Street ORANGE NSW 2800 Saving record 17/2023, 415 Beasley Road SPRING HILL NSW 2800 Saving record 25/2023, 190 Berrilee Road CANOBOLAS NSW 2800 Saving record 16/2023, 82 Botanic Way ORANGE NSW 2800 Saving record 11/2023, 14 Courallie Drive ORANGE NSW 2800 Saving record 148/2022, 97 Gardiner Road ORANGE NSW 2800 Saving record 18/2023, 153 Hill Street ORANGE NSW 2800 Saving record 24/2023, 12 Hilldale Crescent ORANGE NSW 2800 Saving record 289/2022, 11 Joseph Drive ORANGE NSW 2800 Saving record 288/2022, 13 Joseph Drive ORANGE NSW 2800 Saving record 19/2023, 1 Kanzi Close ORANGE NSW 2800 Saving record 2/2023, 8 Kearneys Drive ORANGE NSW 2800 Saving record 10/2023, 5 Kurim Avenue ORANGE NSW 2800 Saving record 27/2023, Leeds Parade ORANGE NSW 2800 Saving record 179/2021, 91 March Street ORANGE NSW 2800 Saving record 20/2023, 141 Matthews Avenue ORANGE NSW 2800 Saving record 14/2023, 7 Mclachlan Street ORANGE NSW 2800 Saving record 26/2023, 4536 Mitchell Highway LUCKNOW NSW 2800 Saving record 28/2023, 24 Newport Street ORANGE NSW 2800 Saving record 3/2023, 53 Orchard Grove Road ORANGE NSW 2800 Saving record 15/2023, 16 Platinum Parade ORANGE NSW 2800 Saving record 437/2021, 130 Ploughmans Lane ORANGE NSW 2800 Saving record 240/2019, 130 Ploughmans Lane ORANGE NSW 2800 Saving record 8/2023, 64 Prince Street ORANGE NSW 2800 Saving record 1/2023, 222-224 Summer Street ORANGE NSW 2800 Saving record 13/2023, 64 Valencia Drive ORANGE NSW 2800 Saving record 21/2023, 44 Wentworth Lane ORANGE NSW 2800 Saving record 4/2023, 41 Westerly Drive ORANGE NSW 2800 Saving record 5/2023, 43 Westerly Drive ORANGE NSW 2800 Saving record 6/2023, 45 Westerly Drive ORANGE NSW 2800 Saving record 7/2023, 47 Westerly Drive ORANGE NSW 2800 Saving record 23/2023, 176 Woodward Street ORANGE NSW 2800 Collecting feed data for woollahra... Saving record 531/2022, 8 Albert Street EDGECLIFF NSW 2027 Saving record 541/2022, 5 Arthur Street BELLEVUE HILL NSW 2023 Saving record 539/2022, 6 Arthur Street BELLEVUE HILL NSW 2023 Saving record 580/2022, 6 Bayview Hill Road ROSE BAY NSW 2029 Saving record 572/2022, 89 Bellevue Road BELLEVUE HILL NSW 2023 Saving record 536/2022, 1 Beresford Crescent BELLEVUE HILL NSW 2023 Saving record 545/2022, 78 Beresford Road BELLEVUE HILL NSW 2023 Saving record 542/2022, Unit 7 4 Birriga Road BELLEVUE HILL NSW 2023 Saving record 528/2022, 5 Boambillee Avenue VAUCLUSE NSW 2030 Saving record 399/2021, 18 Boronia Road BELLEVUE HILL NSW 2023 Saving record 544/2022, 10A Bradley Avenue BELLEVUE HILL NSW 2023 Saving record 583/2022, 1-3 Brodie Street PADDINGTON NSW 2021 Saving record 586/2022, 1-3 Brodie Street PADDINGTON NSW 2021 Saving record 588/2022, Unit 1 6 Buckhurst Avenue POINT PIPER NSW 2027 Saving record 578/2022, 73 Bulkara Road BELLEVUE HILL NSW 2023 Saving record 2/2023, 30 Bundarra Road BELLEVUE HILL NSW 2023 Saving record 190/2021, 32 Bundarra Road BELLEVUE HILL NSW 2023 Saving record 547/2022, 9 Bunyula Road BELLEVUE HILL NSW 2023 Saving record 575/2022, Unit 4 43 Cambridge Avenue VAUCLUSE NSW 2030 Saving record 142/2022, 48 Cameron Street EDGECLIFF NSW 2027 Saving record 537/2022, 64 Cameron Street EDGECLIFF NSW 2027 Saving record 530/2022, 9 Captain Pipers Road VAUCLUSE NSW 2030 Saving record 260/2021, 45 Cascade Street PADDINGTON NSW 2021 Saving record 245/2022, 87 Cascade Street PADDINGTON NSW 2021 Saving record 378/2022, 49 Chamberlain Avenue ROSE BAY NSW 2029 Saving record 285/2021, 18 Court Road DOUBLE BAY NSW 2028 Saving record 467/2022, 12 Cross Street DOUBLE BAY NSW 2028 Saving record 321/2020, 19-27 Cross Street DOUBLE BAY NSW 2028 Saving record 321/2020, 19-27 Cross Street DOUBLE BAY NSW 2028 Saving record 617/2017, 28-34 Cross Street DOUBLE BAY NSW 2028 Saving record 598/2022, 17 Dover Road ROSE BAY NSW 2029 Saving record 557/2022, 9 Dumaresq Road ROSE BAY NSW 2029 Saving record 439/2021, 6 Dunbar Street WATSONS BAY NSW 2030 Saving record 540/2022, 23 Eastbourne Road DARLING POINT NSW 2027 Saving record 605/2022, 262 Edgecliff Road WOOLLAHRA NSW 2025 Saving record 573/2022, 265 Edgecliff Road WOOLLAHRA NSW 2025 Saving record 78/2019, 337 Edgecliff Road WOOLLAHRA NSW 2025 Saving record 534/2022, 3 Fairfax Road BELLEVUE HILL NSW 2023 Saving record 559/2022, 24 Faraday Avenue ROSE BAY NSW 2029 Saving record 604/2022, 4 Fisher Avenue VAUCLUSE NSW 2030 Saving record 548/2022, 34A Fitzwilliam Road VAUCLUSE NSW 2030 Saving record 550/2022, 37 Fitzwilliam Road VAUCLUSE NSW 2030 Saving record 551/2022, 37 Fitzwilliam Road VAUCLUSE NSW 2030 Saving record 552/2022, 37 Fitzwilliam Road VAUCLUSE NSW 2030 Saving record 579/2022, 37 Fitzwilliam Road VAUCLUSE NSW 2030 Saving record 564/2022, 9 Gap Road WATSONS BAY NSW 2030 Saving record 593/2022, 14 Ginahgulla Road BELLEVUE HILL NSW 2023 Saving record 570/2022, 47 Glenview Street PADDINGTON NSW 2021 Saving record 567/2022, 12 Greenoaks Avenue DARLING POINT NSW 2027 Saving record 589/2022, 42 Grosvenor Street WOOLLAHRA NSW 2025 Saving record 553/2022, 14 Hargrave Lane PADDINGTON NSW 2021 Saving record 595/2022, 86A Hopetoun Avenue VAUCLUSE NSW 2030 Saving record 585/2022, 1 Jesmond Avenue VAUCLUSE NSW 2030 Saving record 527/2022, 86 John Street WOOLLAHRA NSW 2025 Saving record 275/2021, 36 Kambala Road BELLEVUE HILL NSW 2023 Saving record 219/2018, 34 Kent Road ROSE BAY NSW 2029 Saving record 3/2023, 31-37 Knox Street DOUBLE BAY NSW 2028 Saving record 323/2021, 9 Magney Street WOOLLAHRA NSW 2025 Saving record 602/2022, 9 Magney Street WOOLLAHRA NSW 2025 Saving record 2/2017, 9 Mona Road DARLING POINT NSW 2027 Saving record 603/2022, 6 Nelson Street WOOLLAHRA NSW 2025 Saving record 498/2016, 1 New Beach Road DARLING POINT NSW 2027 Saving record 546/2022, 549 New South Head Road BELLEVUE HILL NSW 2023 Saving record 13/2019, 590 New South Head Road POINT PIPER NSW 2027 Saving record 569/2022, 835 New South Head Road ROSE BAY NSW 2029 Saving record 404/2017, 63 New South Head Road VAUCLUSE NSW 2030 Saving record 492/2019, 13 Nulla Street VAUCLUSE NSW 2030 Saving record 592/2022, 186-192 Old South Head Road BELLEVUE HILL NSW 2023 Saving record 566/2022, 52 Ormond Street PADDINGTON NSW 2021 Saving record 558/2022, 22-36 Oxford Street WOOLLAHRA NSW 2025 Saving record 607/2021, 60 Oxford Street WOOLLAHRA NSW 2025 Saving record 591/2022, 7 Pacific Street WATSONS BAY NSW 2030 Saving record 590/2022, 175 Paddington Street PADDINGTON NSW 2021 Saving record 565/2022, 9 Quambi Place EDGECLIFF NSW 2027 Saving record 596/2022, 23 Queen Street WOOLLAHRA NSW 2025 Saving record 607/2022, 4 Riddell Street BELLEVUE HILL NSW 2023 Saving record 5/2023, 14 Rosemont Avenue WOOLLAHRA NSW 2025 Saving record 93/2022, 8 Rupertswood Avenue BELLEVUE HILL NSW 2023 Saving record 587/2022, 30A Russell Street VAUCLUSE NSW 2030 Saving record 4/2023, 62 Salisbury Road BELLEVUE HILL NSW 2023 Saving record 523/2022, 26 Salisbury Road ROSE BAY NSW 2029 Saving record 529/2022, 14 Serpentine Parade VAUCLUSE NSW 2030 Saving record 581/2022, 8-10 Small Street WOOLLAHRA NSW 2025 Saving record 543/2022, 25 Small Street WOOLLAHRA NSW 2025 Saving record 561/2022, 25 Spicer Street WOOLLAHRA NSW 2025 Saving record 538/2022, 54 Streatfield Road BELLEVUE HILL NSW 2023 Saving record 78/2021, 35 Sutherland Avenue PADDINGTON NSW 2021 Saving record 1/2023, 64 Sutherland Street PADDINGTON NSW 2021 Saving record 532/2022, 4 Tivoli Avenue ROSE BAY NSW 2029 Saving record 582/2022, 175 Underwood Street PADDINGTON NSW 2021 Saving record 574/2022, 25 Vaucluse Road VAUCLUSE NSW 2030 Saving record 584/2022, 32A Vaucluse Road VAUCLUSE NSW 2030 Saving record 533/2022, 42 Vaucluse Road VAUCLUSE NSW 2030 Saving record 562/2022, 27 Victoria Road BELLEVUE HILL NSW 2023 Saving record 606/2022, 73 Victoria Road BELLEVUE HILL NSW 2023 Saving record 171/2021, 145 Victoria Road BELLEVUE HILL NSW 2023 Saving record 535/2022, 176 Victoria Road BELLEVUE HILL NSW 2023 Saving record 568/2022, 35 View Street WOOLLAHRA NSW 2025 Saving record 306/2021, 48 Village High Road VAUCLUSE NSW 2030 Saving record 600/2022, 3 Village Lower Road VAUCLUSE NSW 2030 Saving record 599/2022, 2 Wallis Street WOOLLAHRA NSW 2025 Saving record 597/2022, 3 Wentworth Place POINT PIPER NSW 2027 Saving record 555/2022, 64 Wilberforce Avenue ROSE BAY NSW 2029 Saving record 498/2022, 26 Windsor Street PADDINGTON NSW 2021 Saving record 556/2022, 1 Wingadal Place POINT PIPER NSW 2027 Saving record 601/2022, 16-18 Wolseley Road POINT PIPER NSW 2027 Saving record 610/2021, 16 Wunulla Road POINT PIPER NSW 2027 Saving record 239/2022, 29A Wunulla Road POINT PIPER NSW 2027 Saving record 577/2022, 30 Wyuna Road POINT PIPER NSW 2027 Saving record 594/2022, 11 Yarranabbe Road DARLING POINT NSW 2027 Saving record 168/2022, Unit 1 21 Yarranabbe Road DARLING POINT NSW 2027 Collecting feed data for lane_cove... Saving record 3/2023, 51 Arabella Street LONGUEVILLE NSW 2066 Saving record 131/2015, 4 Bay Street GREENWICH NSW 2065 Saving record 97/2018, 34C College Rd South RIVERVIEW NSW 2066 Saving record 2/2023, 41 Epping Road LANE COVE NORTH NSW 2066 Saving record 72/2019, 10 Fourth Avenue LANE COVE NSW 2066 Saving record 6/2023, 72 Helen Street LANE COVE NORTH NSW 2066 Saving record 4/2023, 21 Murralah Place LANE COVE NSW 2066 Saving record 1/2023, 23 Norfolk Road LONGUEVILLE NSW 2066 Saving record 8/2023, 113 Northwood Road NORTHWOOD NSW 2066 Saving record 5/2023, 21 Romani Avenue RIVERVIEW NSW 2066 Saving record 98/2021, 16 Tantallon Road LANE COVE NORTH NSW 2066 Saving record 7/2023, 16 Trouve Street LANE COVE NSW 2066

Data

Downloaded 1 time by dcalde

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (184 KB) Use the API

rows 10 / 545

council_reference address description info_url date_received date_scraped authority_label on_notice_from on_notice_to
12113/2023
800L Orient Bend SMITHFIELD QLD 4878
Operational Works - Landscaping Smithfield Village Stage 29
https://eservices.cairns.qld.gov.au/eservice/daEnquiryInit.do?nodeNum=227
2023-01-24
2023-01-25
cairns
204/2023
2 Abbott Close EGLINTON WA 6034
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-19
2023-01-25
wanneroo
3980/2022
3 Abbott Close EGLINTON WA 6034
AMENDMENT - Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-13
2023-01-25
wanneroo
65/2023
79 Aduro Street EGLINTON WA 6034
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-10
2023-01-25
wanneroo
167/2023
31 Alabaster Approach JINDALEE WA 6036
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-18
2023-01-25
wanneroo
99/2023
33 Alabaster Approach JINDALEE WA 6036
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-12
2023-01-25
wanneroo
143/2023
59 Alabaster Approach JINDALEE WA 6036
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-17
2023-01-25
wanneroo
82/2023
39 Althorpe Grove EGLINTON WA 6034
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-11
2023-01-25
wanneroo
182/2023
40 Althorpe Grove EGLINTON WA 6034
Single Dwelling - Detached
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-18
2023-01-25
wanneroo
155/2023
24 Alveley Loop BUTLER WA 6036
Swimming Pool Safety Fencing
https://eservice.wanneroo.wa.gov.au/eservice/daEnquiry.do?nodeNum=8047
2023-01-17
2023-01-25
wanneroo

Statistics

Average successful run time: 38 minutes

Total run time: about 5 hours

Total cpu time used: half a minute

Total disk space used: 4.2 MB

History

  • Manually ran revision f171e8fb and completed successfully .
    187 records added, 186 records removed in the database
  • Manually ran revision 5efec1e0 and failed .
    186 records added, 142 records removed in the database
  • Manually ran revision 1f74bf59 and failed .
    142 records added, 125 records removed in the database
  • Manually ran revision a089ffcc and failed .
    125 records added, 124 records removed in the database
  • Manually ran revision d1762d44 and failed .
    124 records added, 103 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

multiple-civica / scraper.rb