adrianshort / suttoncouncilaudio

Scrapes www.sutton.gov.uk

Government, politics and public administration, Sutton Council Home, Sutton Council


Contributors adrianshort

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Error removing intermediate container 1a98a393466c: Driver aufs failed to remove root filesystem 1a98a393466c8445fd1008e25899b759e13f99c92de3905113dc73ea91b73e3a: rename /var/lib/docker/aufs/mnt/b57d84689c4d061c8c8cad5e09af106985177756d3a2deab416df3fc151f28ff /var/lib/docker/aufs/mnt/b57d84689c4d061c8c8cad5e09af106985177756d3a2deab416df3fc151f28ff-removing: device or resource busy Error removing intermediate container 1a98a393466c: No such container: 1a98a393466c8445fd1008e25899b759e13f99c92de3905113dc73ea91b73e3a Error removing intermediate container 1a98a393466c: No such container: 1a98a393466c8445fd1008e25899b759e13f99c92de3905113dc73ea91b73e3a Error removing intermediate container 1a98a393466c: No such container: 1a98a393466c8445fd1008e25899b759e13f99c92de3905113dc73ea91b73e3a  -----> Ruby app detected -----> Compiling Ruby -----> Using Ruby version: ruby-2.2.4 -----> Installing dependencies using bundler 1.11.2  Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment  Warning: the running version of Bundler is older than the version that created the lockfile. We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.  Fetching gem metadata from https://rubygems.org/.........  Fetching version metadata from https://rubygems.org/..  Fetching https://github.com/openaustralia/scraperwiki-ruby.git  Installing mime-types 2.99.2  Installing unf_ext 0.0.7.2 with native extensions  Installing httpclient 2.8.0  Installing net-http-digest_auth 1.4  Installing net-http-persistent 2.9.4  Installing ntlm-http 0.1.1  Installing mini_portile2 2.0.0  Installing webrobots 0.1.2  Installing sqlite3 1.3.11 with native extensions  Using bundler 1.11.2  Installing nokogiri 1.6.7.2 with native extensions  Installing unf 0.1.4  Installing domain_name 0.5.20160310  Installing http-cookie 1.0.2  Installing sqlite_magic 0.0.6  Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176)  Installing mechanize 2.7.4  Bundle complete! 2 Gemfile dependencies, 17 gems now installed.  Gems in the groups development and test were not installed.  Bundled gems are installed into ./vendor/bundle.  Bundle completed (25.00s)  Cleaning up the bundler cache.   -----> Discovering process types  Procfile declares types -> scraper Error removing intermediate container 1a98a393466c: No such container: 1a98a393466c8445fd1008e25899b759e13f99c92de3905113dc73ea91b73e3a Injecting scraper and running... {:slug=>"council", :name=>"Council", :url=> "http://www.sutton.gov.uk/downloads/download/494/council_audio_recordings"} {:slug=>"strategy-and-resources", :name=>"Strategy and Resources Committee", :url=> "http://www.sutton.gov.uk/downloads/download/495/strategy_and_resources_committee_audio_recordings"} {:slug=>"adult-social-services-and-health", :name=>"Adult Social Services and Health Committee", :url=> "http://www.sutton.gov.uk/downloads/download/496/adult_social_services_and_health_committee_audio_recordings"} {:slug=>"children-family-and-education", :name=>"Children, Family and Education Committee", :url=> "http://www.sutton.gov.uk/downloads/download/497/children_family_and_education_committee_audio_recordings"} {:slug=>"environment-and-neighbourhood", :name=>"Environment and Neighbourhood Committee", :url=> "http://www.sutton.gov.uk/downloads/download/498/environment_and_neighbourhood_committee_audio_recordings"} {:slug=>"housing-economy-and-business", :name=>"Housing, Economy and Business Committee", :url=> "http://www.sutton.gov.uk/downloads/download/499/housing_economy_and_business_committee_audio_recordings"} {:slug=>"health-and-wellbeing", :name=>"Health and Wellbeing Board", :url=> "http://www.sutton.gov.uk/downloads/download/500/health_and_wellbeing_board_audio_recordings"} {:slug=>"planning", :name=>"Planning Committee", :url=> "http://www.sutton.gov.uk/downloads/download/501/planning_committee_audio_recordings"} {:slug=>"beddington-and-wallington-local", :name=>"Beddington and Wallington Local Committee", :url=> "http://www.sutton.gov.uk/downloads/download/502/beddington_and_wallington_local_committee_audio_recordings"} {:slug=>"carshalton-and-clockhouse-local", :name=>"Carshalton and Clockhouse Local Committee", :url=> "http://www.sutton.gov.uk/downloads/download/492/carshalton_and_clockhouse_local_committee_audio_recordings"} {:slug=>"cheam-north-and-worcester-park-local", :name=>"Cheam North and Worcester Park Local Committee", :url=> "http://www.sutton.gov.uk/downloads/download/503/cheam_north_and_worcester_park_local_committee_audio_recordings"} {:slug=>"st-helier-the-wrythe-and-wandle-valley-local", :name=>"St Helier, The Wrythe and Wandle Valley Local Committee", :url=> "http://www.sutton.gov.uk/downloads/download/504/st_helier_the_wrythe_and_wandle_valley_local_committee_audio_recordings"} {:slug=>"sutton-local", :name=>"Sutton Local Committee", :url=> "http://www.sutton.gov.uk/downloads/download/505/sutton_local_committee_audio_recordings"} {:slug=>"sutton-south-cheam-and-belmont-local", :name=>"Sutton South, Cheam and Belmont Local Committee", :url=> "http://www.sutton.gov.uk/downloads/download/506/sutton_south_cheam_and_belmont_local_committee_audio_recordings"} {:slug=>"opportunity-sutton-ltd-sub", :name=>"Opportunity Sutton Ltd Sub Committee", :url=> "https://www.sutton.gov.uk/downloads/download/624/opportunity_sutton_ltd_sub_committee"} {:slug=>"standards", :name=>"Standards Committee", :url=> "https://www.sutton.gov.uk/downloads/download/644/standards_committee_audio_recordings"} {:slug=>"scrutiny", :name=>"Scrutiny Committee", :url=>"https://www.sutton.gov.uk/downloads/download/701/scrutiny_committee"} {:slug=>"sutton-shareholdings", :name=>"Sutton Shareholdings Board", :url=> "https://www.sutton.gov.uk/downloads/download/705/sutton_shareholdings_board"} Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning. 23rd May 2016 - Part 1 (MP3) 23rd May 2016 - Part 2 (MP3) 1st August 2016 (MP3) 12th September 2016 (MP3) 14th November 2016 (MP3) 30th January 2017 (MP3) 18/7/2016 (MP3) 31/10/2016 (MP3) 16/01/2017 (MP3) 20/02/2017 (MP3) 13th September 2016 (MP3) 20th December 2016 (MP3) 16/06/2016 (MP3) 29/09/2016 (MP3) 15/12/2016 (MP3) 27th June 2016 (MP3) 15th September 2016 (MP3) 26th January 2017 (MP3) 27th September 2016 (MP3) 6th December 2016 (MP3) 31st January 2017 (MP3) 20th June 2016 (MP3) 26th September 2016 (MP3) 5th December 2016 (MP3) 20th July 2016 (MP3) 17th August 2016 (MP3) 7th September 2016 (MP3) 28th September 2016 (MP3) 19th October 2016 (MP3) 9th November 2016 (MP3) 30th November 2016 (MP3) 21st December 2016 (MP3) 11th January 2017 (MP3) 22nd February 2017 (MP3) 28th June 2016 (MP3) 11th October 2016 (MP3) 10th January 2017 (MP3) 12th April 2016 (MP3) 18th October 2016 (MP3) 17th January 2017 (MP3) 14th July 2016 (MP3) 6th October 2016 (MP3) 12th January 2017 (MP3) 21st April 2016 (MP3) 7th July 2016 (MP3) 13th October 2016 (MP3) 19th January 2017 (MP3) 15th March 2016 (MP3) 7th June 2016 (MP3) 6th September 2016 (MP3) 13th December 2016 (MP3) 9th June 2016 (MP3) 8th September 2016 (MP3) 5th January 2017 (MP3) 16th November 2015 (MP3) 22nd February 2016 - Part 1 (MP3) 22nd February 2016 - Part 2 (MP3) 22nd June 2016 (MP3) 21st July 2016 (MP3) 14th December 2016 (12.2016.MP3) 13th July 2016 (MP3) 2nd November 2016 (MP3) 8th February 2017 (MP3) 4th January 2017 (MP3)

Data

Downloaded 1 time by adrianshort

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (78 KB) Use the API

rows 10 / 16

slug name url
council
Council
strategy-and-resources
Strategy and Resources Committee
adult-social-services-and-health
Adult Social Services and Health Committee
children-family-and-education
Children, Family and Education Committee
environment-and-neighbourhood
Environment and Neighbourhood Committee
housing-economy-and-business
Housing, Economy and Business Committee
health-and-wellbeing
Health and Wellbeing Board
planning
Planning Committee
beddington-and-wallington-local
Beddington and Wallington Local Committee
carshalton-and-clockhouse-local
Carshalton and Clockhouse Local Committee

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (78 KB) Use the API

rows 10 / 139

committee_slug d href title
council
2015-05-18T00:00:00
18th May 2015 - Part 1
council
2015-05-18T00:00:00
18th May 2015 - Part 2
council
2015-07-13T00:00:00
13th July 2015
council
2015-09-28T00:00:00
28th September 2015
council
2015-11-02T00:00:00
2nd November 2015
council
2016-01-18T00:00:00
18th January 2016
council
2016-03-07T00:00:00
7th March 2016
council
2016-05-23T00:00:00
23rd May 2016 - Part 1
council
2016-05-23T00:00:00
23rd May 2016 - Part 2
strategy-and-resources
2015-09-28T00:00:00
28/9/2015 - Part 1

Statistics

Average successful run time: 4 minutes

Total run time: 9 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 102 KB

History

  • Manually ran revision adfac30b and completed successfully .
    nothing changed in the database
    33 pages scraped
  • Manually ran revision 298a3fb2 and failed .
    nothing changed in the database
  • Manually ran revision 298a3fb2 and failed .
    nothing changed in the database
  • Manually ran revision 298a3fb2 and failed .
    nothing changed in the database
  • Manually ran revision 298a3fb2 and completed successfully .
    155 records added in the database
  • Created on morph.io

Scraper code

Ruby

suttoncouncilaudio / scraper.rb