Note that as of November 2015, the government API is no longer maintained and all the resources related to the UK Government Business Plans have been archived. This is a result of the government's decision to build a new system for managing departmental priorities after the 2015 election.
As a result, this tool is obsolete and does not update but the historical data can still be used.
The code in this repository collects and analyses data on the UK Government's progress against its own Structural Reform Plan actions. This was used in several Whitehall Monitor bulletins and Whitehall Monitor 2013 and 2014 annual reports published by the Institute for Government.
Latest data and analysis from the Institute for Government using this dataset: http://www.instituteforgovernment.org.uk/performance
Code by @petrbouchal.
This repository consists of
a script that collects data from the UK government's Business Plans API and makes basic calculations, e.g. on overdue and missed actions
a script that takes the resulting data and produces overall and departmental analytics reports
Currently, only the basic data collection and per-subaction analytics are done.
The data comes in JSON format and is nested as follows:
Department
Priority
Action
Subaction
Only subactions consistently have information attached to them on dates, comments, etc.
It loops through departments, priorities, actions and subactions, then calculates time overdue etc. The key variable that it produces is subaction_[start|end]_status
.
The code is hosted on Morph. It is scheduled to run every day but programmed to only scrape & save data every week.
Columns used for analysis in bold.
| Column name | Description | Unprocessed data |
| --------------- | ------------ | -------- |
| dept_abb | Department abbreviation | N
| deptname | Department name | Y
| deptid | Department ID in API | Y
| depturl | Department URL | Y
| prioritybody | Name of priority | Y
| priorityid | Priority ID | Y
| prioritystrapline | Priority strapline. May contain pointless HTML | Y
| actionid | Action ID | Y
| actionbody | Name of action | Y
| actionnotes | Notes on action | Y
| schedulestartdate | Action scheduled start date. Ignore. | N
| scheduleenddate | Action scheduled end date. Ignore. | N
| actualstartdate | Action actual start date. Ignore. | N
| actualenddate | Action actual end date. Ignore. | N
| **subactionid** | Subaction ID. | N
| subaction_body | Name of subaction. | N
| subaction_notes | Typically contains explanations of missed due dates etc. | N
| subactionschedulestart_date | Schedule start date for subaction. | N
| subactionscheduleend_date | Scheduled end date for subaction | N
| actstart | Action actual start date. Ignore. | N
| actend | Action actual end date. Ignore. | N
| schedstartendmonth | Date at end of month in which subaction is due to start. Created to introduce some leeway into overdue calculations. | N
| schedendendmonth | Date at end of month in which subaction is due to end. Created to introduce some leeway into overdue calculations. | N
| started | Did subaction start? | N
| ended | Did subaction end? | N
| start_status | Start status of subaction. | N
| end_status | End status of subaction. | N
| startearlyby | How long ahead of schedule did subaction start? Treat with caution/ignore. | N
| startedlateby | How long behind schedule did subaction start? Treat with caution/ignore. | N
| startoverdueby | Treat with caution/ignore. | N
| endearlyby | How long ahead of schedule did subaction end? Treat with caution/ignore. | N
| endedlateby | How much overdue did subaction end? Treat with caution/ignore. | N
| endedoverdueby | Treat with caution/ignore. | N
| carriedover | If no start date, start date will be 'Carried over' and this column will=1 | N
| subactionschedulestartorig | Unprocessed scheduled start date of subaction. | Y
| subactionscheduleendorig | Unprocessed scheduled end date of subaction. | Y
| subactionactualstartorig | Unprocessed actual end date of subaction. | Y
| subactionactualendorig | Unprocessed actual end date of subaction. | Y
| datetime | Date and time of data collection. Use this, not date
- to get entries from one run of the scraper rather than whole day | N
| date | Date of data collection. | N
The data can be downloaded from Morph as CSV or SQLite in one piece or as a subset in JSON, XML or CSV via the Morph API, which requires writing a bit of SQL.
To get data out of Morph, you need to login using a Github account.
This is a very suboptimally structured piece of code - but it works.
Ideally, it would not loop around every action, but first collect the data (with the option of saving it as a raw text file), then save it into a table, and then, optionally, do some analytics on the whole table at once, ideally saving into another table to preserve the distinction between data collection and data processing.
To download data sign in with GitHub
rows 10 / 89950
dept_abb | dept_name | dept_id | dept_url | priority_body | priority_id | priority_strapline | action_id | action_body | action_notes | schedule_start_date | schedule_end_date | actual_start_date | actual_end_date | subaction_id | subaction_body | subaction_notes | subaction_schedule_start_date | subaction_schedule_end_date | act_start | act_end | sched_start_endmonth | sched_end_endmonth | started | ended | start_status | end_status | startearlyby | startedlateby | startoverdueby | endearlyby | endedlateby | endedoverdueby | carriedover | subaction_schedule_start_orig | subaction_schedule_end_orig | subaction_actual_start_orig | subaction_actual_end_orig | datetime | date |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CO
|
Cabinet Office
|
1
|
1. Promote UK Growth
|
5
|
Promote UK growth through leading the drive to improve Government efficiency and effectiveness, reforming government procurement, transforming public services, introducing new commercial models across the public sector, getting out of surplus state-owned property, and being a world leader in social investment and the exploitation of open data
|
1175
|
<B>1.11 Drive Cyber Growth</B> <br /><br />
|
|
|
|
|
|
1177
|
i. Work with other Government departments and UK Trade and Investment (UKTI) to stimulate the growth of the UK cyber security sector, and with GCHQ to more directly harness its expertise and intellectual property to support economic growth<br /><br />
|
|
Started
|
May 2015
|
NA
|
NA
|
NA
|
2015-05-31
|
1
|
0
|
Carried over
|
Not due
|
NA
|
NA
|
NA
|
NA
|
NA
|
NA
|
1
|
Started
|
May 2015
|
|
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
993
|
2.1 Drive efficiency savings to save taxpayers’ money and protect frontline services
|
|
|
|
|
|
1035
|
2.1.i. Support Whitehall departments in delivering £20 billion of savings per year by 2014/15
|
The latest update on progress was published on 10 June 2014 (at https://www.gov.uk/government/news/government-unveils-143-billion-of-savings-for-2013-to-2014), reporting that the Cabinet Office’s Efficiency and Reform Group worked with departments across government to realise savings of £14.3billion for 2013/14. This continues ERG’s work to deliver savings of £20 billion by 2015.
|
Started
|
Mar 2015
|
NA
|
NA
|
NA
|
2015-03-31
|
1
|
0
|
Carried over
|
Not due
|
NA
|
NA
|
NA
|
NA
|
NA
|
NA
|
1
|
Started
|
Mar 2015
|
|
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
995
|
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
|
|
|
|
|
|
1037
|
2.2.i. Publish report on the work of the Mutuals Taskforce, the first annual report on the wider mutuals programme and a ‘one-year on’ report
|
Started
|
Oct 2013
|
NA
|
2013-07-04
|
NA
|
2013-10-31
|
1
|
1
|
Carried over
|
Early
|
NA
|
NA
|
NA
|
89
|
NA
|
NA
|
1
|
Started
|
Oct 2013
|
|
2013-07-04 00:00:00
|
2014-08-30 12:46:06
|
2014-08-30
|
||
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
995
|
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
|
|
|
|
|
|
1039
|
2.2.ii. Support public sector spin-outs including 50 mutuals in local authority control and further spin-outs in central government
|
There are now over 70 mutuals in operation:
https://www.gov.uk/government/news/nudge-unit-forming-mutual-joint-venture
|
Started
|
Apr 2014
|
NA
|
2014-02-28
|
NA
|
2014-04-30
|
1
|
1
|
Carried over
|
Early
|
NA
|
NA
|
NA
|
32
|
NA
|
NA
|
1
|
Started
|
Apr 2014
|
|
2014-02-28 00:00:00
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
995
|
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
|
|
|
|
|
|
1041
|
2.2.iii. Launch a new commercial entity for the provision of shared services, as part of the Next Generation Shared Services strategy and in line with the Civil Service Reform Plan
|
On 1 November 2013 it was announced that Steria Limited has been appointed as the private sector partner in a deal that will save taxpayers money by consolidating the government’s back office functions. As partner in a new Joint Venture company, Shared Services Connected Limited (SSCL), Steria will run the second of two new Independent Shared Services Centres (ISSC2). The Joint Venture will harness commercial expertise to deliver government back office functions more efficiently and is a key example of driving innovation in government commercial models – both key parts of the Civil Service Reform Plan.
https://www.gov.uk/government/news/shared-service-strategy-to-save-taxpayer-money
|
Started
|
Mar 2014
|
NA
|
2013-11-30
|
NA
|
2014-03-31
|
1
|
1
|
Carried over
|
Early
|
NA
|
NA
|
NA
|
91
|
NA
|
NA
|
1
|
Started
|
Mar 2014
|
|
2013-11-30 00:00:00
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
995
|
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
|
|
|
|
|
|
1043
|
2.2.iv. Commence three procurements with the aim of transforming the business models of Government services
|
The three procurements are: Steria Ltd taken on shared services work, Behavioural Insights Team forming a joint venture with Nesta and Capita forming a joint venture on government intellectual property: https://www.gov.uk/government/news/new-deal-will-market-government-professional-qualifications, https://www.gov.uk/government/news/nudge-unit-forming-mutual-joint-venture
https://www.gov.uk/government/news/shared-service-strategy-to-save-taxpayer-money
|
Started
|
Mar 2014
|
NA
|
2014-03-31
|
NA
|
2014-03-31
|
1
|
1
|
Carried over
|
On time
|
NA
|
NA
|
NA
|
NA
|
NA
|
NA
|
1
|
Started
|
Mar 2014
|
|
2014-03-31 00:00:00
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
995
|
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
|
|
|
|
|
|
3657
|
2.2.v. With the Department of Communities and Local Government source professional support for 10 pioneering local authorities to consider and implement new delivery models (including VCSE, mutuals, joint ventures etc.) for their services and disseminate learning through the Delivering Differently programme
|
|
Started
|
Mar 2015
|
NA
|
NA
|
NA
|
2015-03-31
|
1
|
0
|
Carried over
|
Not due
|
NA
|
NA
|
NA
|
NA
|
NA
|
NA
|
1
|
Started
|
Mar 2015
|
|
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
997
|
2.3 Introduce greater choice in and control of public services for individuals and communities through Open Public Services
|
|
|
|
|
|
1047
|
2.3.i. Appoint National Advisor for What Works, to support the development of What Works Centres across £200 billion of public spending
|
Started
|
Jul 2013
|
2013-07-02
|
2013-07-02
|
NA
|
2013-07-31
|
1
|
1
|
Carried over
|
On time
|
NA
|
NA
|
NA
|
NA
|
NA
|
NA
|
1
|
Started
|
Jul 2013
|
2013-07-02 00:00:00
|
2013-07-02 00:00:00
|
2014-08-30 12:46:06
|
2014-08-30
|
||
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
997
|
2.3 Introduce greater choice in and control of public services for individuals and communities through Open Public Services
|
|
|
|
|
|
1049
|
2.3.ii. Publish Open Public Services 2014, the annual account of progress and next steps in reforming public services
|
The Government published the 2014 progress report on 14 March 2014. Here is a link to the publication:
https://www.gov.uk/government/publications/open-public-services-2014-progress-report
|
April 2014
|
Apr 2014
|
2014-03-17
|
2014-03-14
|
2014-04-30
|
2014-04-30
|
1
|
1
|
Early
|
Early
|
15
|
NA
|
NA
|
18
|
NA
|
NA
|
0
|
April 2014
|
Apr 2014
|
2014-03-17 00:00:00
|
2014-03-14 00:00:00
|
2014-08-30 12:46:06
|
2014-08-30
|
|
CO
|
Cabinet Office
|
1
|
2. Drive efficiency and effectiveness in government
|
7
|
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
|
999
|
2.4 Reform procurement to support growth and save the taxpayer money
|
|
|
|
|
|
1051
|
2.4.i. Transfer all common goods and services spend to centralised arrangements
|
The transfer to a centralised arrangement has been planned and agreed. The first phase of transition is underway and will take place in stages across 2014-15.
|
Started
|
Dec 2013
|
NA
|
NA
|
NA
|
2013-12-31
|
1
|
0
|
Carried over
|
Overdue
|
NA
|
NA
|
NA
|
NA
|
NA
|
242
|
1
|
Started
|
Dec 2013
|
|
|
2014-08-30 12:46:06
|
2014-08-30
|
Average successful run time: about 3 hours
Total run time: about 1 month
Total cpu time used: 16 minutes
Total disk space used: 128 MB