Brisbane Council scraper

  • Accept Terms - Yes
  • Cookie tracking - Yes
  • Request actual DA page for data - Yes
  • Clearly defined data within a row - No but reasonable

Setup MORPH_PERIOD for data recovery, available options are

  • not set = past 14 days (default)
  • thismonth
  • lastmonth

Contributors mlandauer equivalentideas henare ljnicol LoveMyData MichaelBone

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.4.1 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching connection_pool 2.2.2  Fetching unf_ext 0.0.7.5  Fetching httpclient 2.8.3  Installing connection_pool 2.2.2  Fetching mime-types-data 3.2018.0812  Installing httpclient 2.8.3  Installing unf_ext 0.0.7.5 with native extensions  Installing mime-types-data 3.2018.0812  Fetching net-http-digest_auth 1.4.1  Installing net-http-digest_auth 1.4.1  Fetching mini_portile2 2.3.0  Installing mini_portile2 2.3.0  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Fetching webrobots 0.1.2  Fetching sqlite3 1.3.13  Installing webrobots 0.1.2  Installing sqlite3 1.3.13 with native extensions  Fetching net-http-persistent 3.0.0  Installing net-http-persistent 3.0.0  Fetching mime-types 3.2.2  Installing mime-types 3.2.2  Fetching nokogiri 1.8.4  Installing nokogiri 1.8.4 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.20180417  Installing domain_name 0.5.20180417  Fetching http-cookie 1.0.3  Installing http-cookie 1.0.3  Fetching sqlite_magic 0.0.6  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.6  Installing mechanize 2.7.6  Bundle complete! 2 Gemfile dependencies, 19 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (26.73s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Collecting data from 02/10/2018&2=16/10/2018 Skipping already saved record A005033637 Skipping already saved record A005033705 Skipping already saved record A005033596 Skipping already saved record A005033731 Skipping already saved record A005033907 Skipping already saved record A005033534 Skipping already saved record A005033577 Skipping already saved record A005034125 Skipping already saved record A005034243 Skipping already saved record A005034509 Skipping already saved record A005034408 Skipping already saved record A005034453 Skipping already saved record A005034198 Skipping already saved record A005034536 Skipping already saved record A005034700 Skipping already saved record A005034790 Skipping already saved record A005034865 Skipping already saved record A005034905 Skipping already saved record A005034940 Skipping already saved record A005035041 Skipping already saved record A005035119 Skipping already saved record A005034889 Skipping already saved record A005035005 Skipping already saved record A005035344 Skipping already saved record A005035450 Skipping already saved record A005035465 Skipping already saved record A005038140 Skipping already saved record A005037860 Skipping already saved record A005038247 Skipping already saved record A005037895 Skipping already saved record A005038521 Skipping already saved record A005038635 Skipping already saved record A005038712 Skipping already saved record A005038744 Skipping already saved record A005038596 Skipping already saved record A005038694 Skipping already saved record A005039367 Skipping already saved record A005039688 Skipping already saved record A005039698 Skipping already saved record A005039811 Skipping already saved record A005039920 Skipping already saved record A005043067 Skipping already saved record A005039702 Skipping already saved record A005039942 Skipping already saved record A005040390 Skipping already saved record A005040191 Skipping already saved record A005040182 Skipping already saved record A005040220 Skipping already saved record A005040275 Skipping already saved record A005040478 Skipping already saved record A005040496 - Skipping already saved record A005040501 Skipping already saved record A005040340 Skipping already saved record A005040442 Skipping already saved record A005040560 Skipping already saved record A005040391 Skipping already saved record A005043302 Skipping already saved record A005043171 Skipping already saved record A005043246 Skipping already saved record A005043218 Saving record A005043650 - 53 TARINGA PDE INDOOROOPILLY QLD 4068 Saving record A005043716 - L 5 62 CROCKFORD 62 CROCKFORD ST NORTHGATE QLD 4013 Saving record A005043815 - 32 ADA ST WINDSOR QLD 4030 Saving record A005043877 - - 97 STANNARD RD MANLY WEST QLD 4179 Skipping already saved record A005033741 Skipping already saved record A005033805 Skipping already saved record A005033707 Skipping already saved record A005033639 Skipping already saved record A005033853 Skipping already saved record A005033688 Skipping already saved record A005033503 Skipping already saved record A005033786 Skipping already saved record A005033577 Skipping already saved record A005034466 Skipping already saved record A005035064 Skipping already saved record A005035127 Skipping already saved record A005034790 Skipping already saved record A005034810 Skipping already saved record A005034911 Skipping already saved record A005035006 Skipping already saved record A005035144 Skipping already saved record A005034713 Skipping already saved record A005035404 Skipping already saved record A005035455 Skipping already saved record A005035465 Skipping already saved record A005038172 Skipping already saved record A005037895 Skipping already saved record A005038837 Skipping already saved record A005038493 Skipping already saved record A005038688 Skipping already saved record A005039210 Skipping already saved record A005039279 Skipping already saved record A005039367 Skipping already saved record A005039503 Skipping already saved record A005039245 Skipping already saved record A005039875 Skipping already saved record A005043067 Skipping already saved record A005039817 Skipping already saved record A005039870 Skipping already saved record A005039909 Skipping already saved record A005039836 Skipping already saved record A005039794 Skipping already saved record A005039813 Skipping already saved record A005040275 Skipping already saved record A005040478 Skipping already saved record A005040496 - Skipping already saved record A005040501 Skipping already saved record A005040367 Skipping already saved record A005040440 Saving record A005043786 - 57 CAMBRIDGE ST CARINA HEIGHTS QLD 4152 Skipping already saved record A005040369 Skipping already saved record A005043345 Skipping already saved record A005043106 Skipping already saved record A005043302 Skipping already saved record A005043041 Skipping already saved record A005043211 Skipping already saved record A005043716 Saving record A005043847 - 21 THOMAS ST CAMP HILL QLD 4152 Skipping already saved record A005033707 Skipping already saved record A005033810 Skipping already saved record A005033853 Skipping already saved record A005033556 Skipping already saved record A005033577 Skipping already saved record A005034478 Skipping already saved record A005034333 Skipping already saved record A005034238 Skipping already saved record A005034269 Skipping already saved record A005034366 Skipping already saved record A005034466 Skipping already saved record A005034536 Skipping already saved record A005034999 Skipping already saved record A005034865 Skipping already saved record A005035130 Skipping already saved record A005034911 Skipping already saved record A005035061 Skipping already saved record A005035442 Skipping already saved record A005035555 Skipping already saved record A005035465 Skipping already saved record A005035522 Skipping already saved record A005037937 Skipping already saved record A005037883 Skipping already saved record A005038016 Skipping already saved record A005038038 Skipping already saved record A005038603 Skipping already saved record A005038550 Skipping already saved record A005038652 Skipping already saved record A005039465 Skipping already saved record A005039528 Skipping already saved record A005039965 Skipping already saved record A005040052 Skipping already saved record A005039734 Skipping already saved record A005039847 Skipping already saved record A005039975 Skipping already saved record A005040544 Skipping already saved record A005040247 Skipping already saved record A005040326 Skipping already saved record A005040530 Skipping already saved record A005040275 Skipping already saved record A005040440 Skipping already saved record A005043302 Skipping already saved record A005042872 Skipping already saved record A005043262 Skipping already saved record A005043068 Skipping already saved record A005043361 Skipping already saved record A005043202 Saving record A005043586 - 140 QUEENS RD EVERTON PARK QLD 4053 Saving record A005043655 - 38 MUNCE PL CANNON HILL QLD 4170 Skipping already saved record A005043716 Skipping already saved record A005033671 Skipping already saved record A005033856 Skipping already saved record A005034305 Skipping already saved record A005034373 Skipping already saved record A005034200 Skipping already saved record A005034551 Skipping already saved record A005034502 Skipping already saved record A005034839 Skipping already saved record A005034949 Skipping already saved record A005035404 Skipping already saved record A005035442 Skipping already saved record A005035395 Skipping already saved record A005035360 Skipping already saved record A005035545 Skipping already saved record A005035465 Skipping already saved record A005035592 Skipping already saved record A005035622 Skipping already saved record A005038260 Skipping already saved record A005038279 Skipping already saved record A005038247 Skipping already saved record A005037974 Skipping already saved record A005038635 Skipping already saved record A005038594 Skipping already saved record A005038606 Skipping already saved record A005038744 Skipping already saved record A005038724 Skipping already saved record A005038639 Skipping already saved record A005039161 Skipping already saved record A005039213 Skipping already saved record A005039389 Skipping already saved record A005039367 Skipping already saved record A005039503 Skipping already saved record A005039683 Skipping already saved record A005039708 Skipping already saved record A005039909 Skipping already saved record A005039836 Skipping already saved record A005040067 Skipping already saved record A005040372 Skipping already saved record A005040494 Skipping already saved record A005040478 Skipping already saved record A005040496 - Skipping already saved record A005040501 Skipping already saved record A005040196 Skipping already saved record A005040317 Skipping already saved record A005040328 Skipping already saved record A005043239 Skipping already saved record A005043304 Skipping already saved record A005042918 Skipping already saved record A005043052 Skipping already saved record A005043068 Skipping already saved record A005043141 Skipping already saved record A005043171 Skipping already saved record A005043383 Skipping already saved record A005043192 Skipping already saved record A005043242 Skipping already saved record A005043267 Skipping already saved record A005043313 Skipping already saved record A005043319 Saving record A005043862 - 12 BAY TCE WYNNUM QLD 4178 Saving record A005043801 - 284 GARDNER RD ROCHEDALE QLD 4123 Skipping already saved record A005033935 Skipping already saved record A005033560 Skipping already saved record A005033810 Skipping already saved record A005033657 Skipping already saved record A005033886 Skipping already saved record A005033577 Skipping already saved record A005034425 Skipping already saved record A005035158 Skipping already saved record A005034700 Skipping already saved record A005034776 Skipping already saved record A005034835 Skipping already saved record A005035095 Skipping already saved record A005035442 Skipping already saved record A005035389 Skipping already saved record A005035700 Skipping already saved record A005035545 Skipping already saved record A005035465 Skipping already saved record A005035497 Skipping already saved record A005035558 Skipping already saved record A005035679 Skipping already saved record A005035681 Skipping already saved record A005037966 Skipping already saved record A005038077 Skipping already saved record A005037841 Skipping already saved record A005037952 Skipping already saved record A005038084 Skipping already saved record A005037895 Skipping already saved record A005038737 Skipping already saved record A005038540 Skipping already saved record A005039328 Skipping already saved record A005039503 Skipping already saved record A005039528 Skipping already saved record A005039710 Skipping already saved record A005040052 Skipping already saved record A005040008 Skipping already saved record A005039660 Skipping already saved record A005039811 Skipping already saved record A005043067 Skipping already saved record A005039847 Skipping already saved record A005040163 Skipping already saved record A005040275 Skipping already saved record A005040478 Skipping already saved record A005040496 - Skipping already saved record A005040440 Skipping already saved record A005043000 Skipping already saved record A005043115 Skipping already saved record A005043239 Skipping already saved record A005043302 Skipping already saved record A005043203 Skipping already saved record A005043361 Skipping already saved record A005043300 Skipping already saved record A005043020 Saving record A005043764 - 6 BEET ST COORPAROO QLD 4151 Skipping already saved record A005033877 Skipping already saved record A005033894 Skipping already saved record A005033707 Skipping already saved record A005033561 Skipping already saved record A005033744 Skipping already saved record A005033838 Skipping already saved record A005033873 Skipping already saved record A005034154 Skipping already saved record A005034518 Skipping already saved record A005034417 Skipping already saved record A005034366 Skipping already saved record A005035032 Skipping already saved record A005034863 Skipping already saved record A005034992 Skipping already saved record A005034700 Skipping already saved record A005034839 Skipping already saved record A005035059 Skipping already saved record A005035061 Skipping already saved record A005035005 Skipping already saved record A005035442 Skipping already saved record A005035525 Skipping already saved record A005035619 Skipping already saved record A005035430 Skipping already saved record A005035465 Skipping already saved record A005035558 Skipping already saved record A005035592 Skipping already saved record A005035410 Skipping already saved record A005038202 Skipping already saved record A005038012 Skipping already saved record A005038782 Skipping already saved record A005038568 Skipping already saved record A005038724 Skipping already saved record A005039445 Skipping already saved record A005039193 Skipping already saved record A005039503 Skipping already saved record A005040008 Skipping already saved record A005039741 Skipping already saved record A005039986 Skipping already saved record A005043067 Skipping already saved record A005039765 Skipping already saved record A005039847 Skipping already saved record A005040539 Skipping already saved record A005040478 Skipping already saved record A005040440 Skipping already saved record A005042829 Skipping already saved record A005043117 Skipping already saved record A005043352 Skipping already saved record A005043014 Skipping already saved record A005043105 Skipping already saved record A005043141 Skipping already saved record A005043171 Skipping already saved record A005043361 Skipping already saved record A005043161 Skipping already saved record A005043249 Skipping already saved record A005043305 Skipping already saved record A005043093 Saving record A005043843 - 12 VIOLET ST WYNNUM QLD 4178 Saving record A005043754 - 54 FLAME TREE CRES CARINDALE QLD 4152 Skipping already saved record A005043862 Saving record A005043756 - 89 VICTORIA AVE CHELMER QLD 4068 Skipping already saved record A005033723 Skipping already saved record A005033731 Skipping already saved record A005034320 Skipping already saved record A005034341 Skipping already saved record A005034142 Skipping already saved record A005034417 Skipping already saved record A005034534 Skipping already saved record A005034502 Skipping already saved record A005034536 Skipping already saved record A005034865 Skipping already saved record A005034905 Skipping already saved record A005035023 Skipping already saved record A005034703 Skipping already saved record A005034884 Skipping already saved record A005034984 Skipping already saved record A005034911 Skipping already saved record A005035404 Skipping already saved record A005035450 Skipping already saved record A005035514 Skipping already saved record A005035499 Skipping already saved record A005035545 Skipping already saved record A005035465 Skipping already saved record A005035681 Skipping already saved record A005038035 Skipping already saved record A005038304 Skipping already saved record A005038247 Skipping already saved record A005037952 Skipping already saved record A005037974 Skipping already saved record A005038264 Skipping already saved record A005038806 Skipping already saved record A005038682 Skipping already saved record A005038782 Skipping already saved record A005038688 Skipping already saved record A005038611 Skipping already saved record A005038733 Skipping already saved record A005038784 Skipping already saved record A005039213 Skipping already saved record A005039328 Skipping already saved record A005039458 Skipping already saved record A005039503 Skipping already saved record A005039189 Skipping already saved record A005039392 Skipping already saved record A005039062 Skipping already saved record A005039631 Skipping already saved record A005039986 Skipping already saved record A005039847 Skipping already saved record A005040418 Skipping already saved record A005040275 Skipping already saved record A005040538 Skipping already saved record A005040317 Skipping already saved record A005043171 Skipping already saved record A005043246 Skipping already saved record A005043221 Skipping already saved record A005043252 Skipping already saved record A005043287 Skipping already saved record A005033767 Skipping already saved record A005034240 Skipping already saved record A005034419 Skipping already saved record A005034417 Skipping already saved record A005034540 Skipping already saved record A005034495 Skipping already saved record A005034366 Skipping already saved record A005034466 Skipping already saved record A005034502 Skipping already saved record A005034979 Skipping already saved record A005034790 Skipping already saved record A005035023 Skipping already saved record A005034810 Skipping already saved record A005035153 Skipping already saved record A005034911 Skipping already saved record A005035005 Skipping already saved record A005035700 Skipping already saved record A005035545 Skipping already saved record A005035430 Skipping already saved record A005035465 Skipping already saved record A005038097 Skipping already saved record A005038205 Skipping already saved record A005037841 Skipping already saved record A005038806 Skipping already saved record A005038840 Skipping already saved record A005038854 Skipping already saved record A005038474 Skipping already saved record A005038594 Skipping already saved record A005039389 Skipping already saved record A005039449 Skipping already saved record A005039503 Skipping already saved record A005039965 Skipping already saved record A005040008 Skipping already saved record A005043067 Skipping already saved record A005039847 Skipping already saved record A005040191 Skipping already saved record A005040467 Skipping already saved record A005040275 Skipping already saved record A005040478 Skipping already saved record A005043057 Skipping already saved record A005043239 Skipping already saved record A005043120 Skipping already saved record A005043302 Skipping already saved record A005042989 Skipping already saved record A005043141 Skipping already saved record A005043145 Skipping already saved record A005043260 Skipping already saved record A005043311 Saving record A005043783 - 62 BERTHA ST KALINGA QLD 4030 Saving record A005043578 - 863B WATERWORKS RD THE GAP QLD 4061 Saving record A005043647 - 231 FORD RD BURBANK QLD 4156 Saving record A005043710 - 16 CROCKFORD ST NORTHGATE QLD 4013 Skipping already saved record A005043801

Data

Downloaded 7555 times by slav123 openaustralia LoveMyData TheZepto jesse-a-reilly spatialsteve stevebrencorp yourneighbourhood srsmonty ljnicol dsccramb peteclowes primin2 MikeRalphson n1k0p0l ShawInnes newmana thinkpropertyjason bigsmaug jamezpolley rimonhm darylrae

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.08 MB) Use the API

rows 10 / 4030

info_url council_reference date_received description address date_scraped comment_url lot property_description
A004868622
2018-03-01
Signs and Line Marking for Minor Roads
106 CONSIDINE ST ELLEN GROVE QLD 4078
2018-03-09
A004868759
2018-03-01
Multiple Dwelling, Extension, Multiple Dwelling, Demolition
14 FIFTH AVE SANDGATE QLD 4017
2018-03-09
A004871923
2018-03-01
Standard Format Plan
79 BROOKBENT RD PALLARA QLD 4110
2018-03-09
A004869372
2018-03-01
Standard Format Plan
7 HOOLAN ST STAFFORD QLD 4053
2018-03-09
A004872047
2018-03-01
Standard Format Plan
261 GOODERHAM RD WILLAWONG QLD 4110
2018-03-09
A004868803
2018-03-01
Outdoor Sport and Recreation, Extension
338 GOODERHAM RD N WILLAWONG QLD 4110
2018-03-09
A004868586
2018-03-01
Food and Drink Outlet, Office, Food and Drink Outlet, Office
22 ROSS ST NEWSTEAD QLD 4006
2018-03-09
A004869102
2018-03-02
Dwelling House, Extension, Dwelling House, Extension, Dwelling House, Demolition
66 PRIMROSE ST GRANGE QLD 4051
2018-03-09
A004869404
2018-03-02
Educational Establishment, Extension
109 NORTH RD WYNNUM WEST QLD 4178
2018-03-09
A004869449
2018-03-02
Filling and/or Excavation
59 DENHAM ST BRACKEN RIDGE QLD 4017
2018-03-09

Statistics

Average successful run time: 4 minutes

Total run time: 14 days

Total cpu time used: about 1 hour

Total disk space used: 2.13 MB

History

  • Auto ran revision 412275ea and completed successfully .
    18 records added in the database
    945 pages scraped
  • Auto ran revision 412275ea and completed successfully .
    49 records added in the database
    899 pages scraped
  • Auto ran revision 412275ea and completed successfully .
    nothing changed in the database
  • Auto ran revision 412275ea and completed successfully .
    nothing changed in the database
    758 pages scraped
  • Auto ran revision 412275ea and completed successfully .
    30 records added in the database
    872 pages scraped
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

brisbane / scraper.rb