gooch / city_of_busselton_policy_manual

Busselton Policy Documents

Scrapes www.busselton.wa.gov.au

City of Busselton


This is a scraper that runs on Morph. To get started see the documentation

Contributors gooch

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.3.1 -----> Installing dependencies using bundler 1.15.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Fetching gem metadata from https://rubygems.org/........  Fetching version metadata from https://rubygems.org/.  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Using bundler 1.15.2  Fetching unf_ext 0.0.7.1  Fetching httpclient 2.6.0.1  Fetching mime-types 2.5  Installing mime-types 2.5  Installing httpclient 2.6.0.1  Installing unf_ext 0.0.7.1 with native extensions  Fetching net-http-digest_auth 1.4  Fetching net-http-persistent 2.9.4  Installing net-http-digest_auth 1.4  Installing net-http-persistent 2.9.4  Fetching mini_portile 0.6.2  Fetching ntlm-http 0.1.1  Installing ntlm-http 0.1.1  Fetching webrobots 0.1.1  Installing webrobots 0.1.1  Fetching sqlite3 1.3.10  Installing mini_portile 0.6.2  Installing sqlite3 1.3.10 with native extensions  Fetching nokogiri 1.6.6.2  Installing nokogiri 1.6.6.2 with native extensions  Fetching unf 0.1.4  Installing unf 0.1.4  Fetching domain_name 0.5.24  Installing domain_name 0.5.24  Fetching http-cookie 1.0.2  Installing http-cookie 1.0.2  Fetching sqlite_magic 0.0.3  Installing sqlite_magic 0.0.3  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Fetching mechanize 2.7.3  Installing mechanize 2.7.3  Bundle complete! 2 Gemfile dependencies, 17 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (31.55s)  Cleaning up the bundler cache. -----> Detecting rake tasks   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... I, [2018-01-07T07:47:44.064623 #17] INFO -- : Could not find date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp004_busselton_jetty_memorials_pand_l_180517.pdf using (?-mix:Last.updated.(\d{2})\/(\d{2})\/(\d{4})) I, [2018-01-07T07:47:44.091210 #17] INFO -- : Could not find date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp004_busselton_jetty_memorials_pand_l_180517.pdf using (?-mix:Implemented (\d{2})\/(\d{2})\/(\d{4})) I, [2018-01-07T07:47:44.123394 #17] INFO -- : Could not find date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp004_busselton_jetty_memorials_pand_l_180517.pdf using (?-mix:Last.updated.(.*)) E, [2018-01-07T07:47:44.123507 #17] ERROR -- : Could not find a date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp004_busselton_jetty_memorials_pand_l_180517.pdf using any pattern I, [2018-01-07T07:47:44.123552 #17] INFO -- : {"title"=>"Council Policy : 004 - Busselton Jetty - Placement of Memorial Plaques (PDF, 26KB)", "document_url"=>"http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp004_busselton_jetty_memorials_pand_l_180517.pdf", "id"=>"cp004"} Skipping already saved record cp004 I, [2018-01-07T07:47:46.016141 #17] INFO -- : Could not find date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp006_mobile_vendors_jetty_pandl_180517cleanversion.pdf using (?-mix:Last.updated.(\d{2})\/(\d{2})\/(\d{4})) I, [2018-01-07T07:47:46.038843 #17] INFO -- : Could not find date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp006_mobile_vendors_jetty_pandl_180517cleanversion.pdf using (?-mix:Implemented (\d{2})\/(\d{2})\/(\d{4})) I, [2018-01-07T07:47:46.069617 #17] INFO -- : Could not find date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp006_mobile_vendors_jetty_pandl_180517cleanversion.pdf using (?-mix:Last.updated.(.*)) E, [2018-01-07T07:47:46.069778 #17] ERROR -- : Could not find a date in http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp006_mobile_vendors_jetty_pandl_180517cleanversion.pdf using any pattern I, [2018-01-07T07:47:46.069869 #17] INFO -- : {"title"=>"Council Policy : 006 - Mobile Vendors on the Busselton Jetty (PDF, 108KB)", "document_url"=>"http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp006_mobile_vendors_jetty_pandl_180517cleanversion.pdf", "id"=>"cp006"} I, [2018-01-07T07:47:46.078558 #17] INFO -- : Stored {"title"=>"Council Policy : 006 - Mobile Vendors on the Busselton Jetty (PDF, 108KB)", "document_url"=>"http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/review_of_cp006_mobile_vendors_jetty_pandl_180517cleanversion.pdf", "id"=>"cp006"} I, [2018-01-07T07:47:48.130480 #17] INFO -- : {"title"=>"Council Policy : 010 - Asbestos Management  (PDF, 360KB)", "document_url"=>"http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/cp010_asbestos_managementv2_-_adopted_13042016.pdf", "id"=>"cp010", "last_updated"=>"2016-04-13"} Skipping already saved record cp010 I, [2018-01-07T07:47:50.988857 #17] INFO -- : {"title"=>"Council Policy : 011 - Rails to Trails (PDF, 347KB)", "document_url"=>"http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/cp011_rails_to_trailsv3_-_adopted_13042016.pdf", "id"=>"cp011", "last_updated"=>"2016-04-13"} Skipping already saved record cp011 E, [2018-01-07T07:47:50.989629 #17] ERROR -- : #<URI::HTTP http://www.busselton.wa.gov.au/files/sharedassets/public/ecm/ceogov/communitypolicies/policy_017_urban_art_2.pdf>

Data

Downloaded 2 times by gooch

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (21 KB) Use the API

rows 10 / 64

title document_url id last_updated
Council Policy - Access and Inclusion (PDF, 132KB)
cp235
2015-05-13
Council Policy - Applications for Exploration or Mining Licences for Coal within the City of Busselton  (PDF, 147KB)
cp043
2016-03-09
Council Policy - Asbestos Management  (PDF, 360KB)
cp010
2016-04-13
Council Policy - Building and Development Lists (PDF, 27KB)
cp039
2010-07-14
Council Policy - Busselton Jetty - Placement of Memorial Plaques (PDF, 46KB)
cp004
2012-02-22
Council Policy - Busselton Regional Airport - Statement of Intent  (PDF, 136KB)
cp124
2014-11-12
Council Policy - Closed Circuit Television Policy (CCTV) Busselton Jetty (PDF, 100KB)
cp028
Council Policy - Community Engagement and Consultation (PDF, 144KB)
cp023
2014-03-12
Council Policy - Community Facilities Bookings (PDF, 165KB)
cp027
2014-08-13
Council Policy - Council Property - Granting of Access to Public Venues (PDF, 28KB)
cp065

Statistics

Average successful run time: 1 minute

Total run time: 3 minutes

Total cpu time used: less than 10 seconds

Total disk space used: 46.9 KB

History

  • Manually ran revision fe0ec7c5 and completed successfully .
    1 record added in the database
    6 pages scraped
  • Manually ran revision fe0ec7c5 and completed successfully .
    57 records added in the database
    65 pages scraped
  • Manually ran revision 547604b3 and failed .
    6 records added in the database
    8 pages scraped
  • Created on morph.io