petrbouchal / GovBusinessPlans

Pull data from UK Government's Business Plans API and build snapshot and time-series analytical reports based on it.

README.md

UK Government Business Plan data retrieval & analytics

The code in this repository collects and analyses data on the UK Government's progress against its own Structural Reform Plan actions. This was used in several Whitehall Monitor bulletins and Whitehall Monitor 2013 and 2014 annual reports published by the Institute for Government.

Latest data and analysis from the Institute for Government using this dataset: http://www.instituteforgovernment.org.uk/publication/measuring-performance

Code by @petrbouchal.

This repository consists of

  1. a script that collects data from the UK government's Business Plans API and makes basic calculations, e.g. on overdue and missed actions

  2. a script that takes the resulting data and produces overall and departmental analytics reports

Currently, only the basic data collection and per-subaction analytics are done.

Where the data come from

  • API base URL: http://transparency.number10.gov.uk/api/
  • API documentation: http://transparency.number10.gov.uk/developers
  • Number 10 front end: http://transparency.number10.gov.uk/

The data comes in JSON format and is nested as follows:

Department Priority Action Subaction Only subactions consistently have information attached to them on dates, comments, etc.

What it does

It loops through departments, priorities, actions and subactions, then calculates time overdue etc. The key variable that it produces is subaction_[start|end]_status.

The code is hosted on Morph. It is scheduled to run every day but programmed to only scrape & save data every week.

Codebook:

Columns used for analysis in bold.

| Column name | Description | Unprocessed data | | --------------- | ------------ | -------- | | dept_abb | Department abbreviation | N | deptname | Department name | Y | deptid | Department ID in API | Y | depturl | Department URL | Y | prioritybody | Name of priority | Y | priorityid | Priority ID | Y | prioritystrapline | Priority strapline. May contain pointless HTML | Y | actionid | Action ID | Y | actionbody | Name of action | Y | actionnotes | Notes on action | Y | schedulestartdate | Action scheduled start date. Ignore. | N | scheduleenddate | Action scheduled end date. Ignore. | N | actualstartdate | Action actual start date. Ignore. | N | actualenddate | Action actual end date. Ignore. | N | **subactionid** | Subaction ID. | N | subaction_body | Name of subaction. | N | subaction_notes | Typically contains explanations of missed due dates etc. | N | subactionschedulestart_date | Schedule start date for subaction. | N | subactionscheduleend_date | Scheduled end date for subaction | N | actstart | Action actual start date. Ignore. | N | actend | Action actual end date. Ignore. | N | schedstartendmonth | Date at end of month in which subaction is due to start. Created to introduce some leeway into overdue calculations. | N | schedendendmonth | Date at end of month in which subaction is due to end. Created to introduce some leeway into overdue calculations. | N | started | Did subaction start? | N | ended | Did subaction end? | N | start_status | Start status of subaction. | N | end_status | End status of subaction. | N | startearlyby | How long ahead of schedule did subaction start? Treat with caution/ignore. | N | startedlateby | How long behind schedule did subaction start? Treat with caution/ignore. | N | startoverdueby | Treat with caution/ignore. | N | endearlyby | How long ahead of schedule did subaction end? Treat with caution/ignore. | N | endedlateby | How much overdue did subaction end? Treat with caution/ignore. | N | endedoverdueby | Treat with caution/ignore. | N | carriedover | If no start date, start date will be 'Carried over' and this column will=1 | N | subactionschedulestartorig | Unprocessed scheduled start date of subaction. | Y | subactionscheduleendorig | Unprocessed scheduled end date of subaction. | Y | subactionactualstartorig | Unprocessed actual end date of subaction. | Y | subactionactualendorig | Unprocessed actual end date of subaction. | Y | datetime | Date and time of data collection. Use this, not date - to get entries from one run of the scraper rather than whole day | N | date | Date of data collection. | N

How to get the data out

The data can be downloaded from Morph as CSV or SQLite in one piece or as a subset in JSON, XML or CSV via the Morph API, which requires writing a bit of SQL.

To get data out of Morph, you need to login using a Github account.

Caveats

This is a very suboptimally structured piece of code - but it works.

Ideally, it would not loop around every action, but first collect the data (with the option of saving it as a raw text file), then save it into a table, and then, optionally, do some analytics on the whole table at once, ideally saving into another table to preserve the distinction between data collection and data processing.

Contributors petrbouchal

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... It's not Friday, and I only scrape weekly. Exiting.

Data

Downloaded 4 times by emilyishness MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (121 MB) Use the API

rows 10 / 89950

dept_abb dept_name dept_id dept_url priority_body priority_id priority_strapline action_id action_body action_notes schedule_start_date schedule_end_date actual_start_date actual_end_date subaction_id subaction_body subaction_notes subaction_schedule_start_date subaction_schedule_end_date act_start act_end sched_start_endmonth sched_end_endmonth started ended start_status end_status startearlyby startedlateby startoverdueby endearlyby endedlateby endedoverdueby carriedover subaction_schedule_start_orig subaction_schedule_end_orig subaction_actual_start_orig subaction_actual_end_orig datetime date
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
1. Promote UK Growth
5
Promote UK growth through leading the drive to improve Government efficiency and effectiveness, reforming government procurement, transforming public services, introducing new commercial models across the public sector, getting out of surplus state-owned property, and being a world leader in social investment and the exploitation of open data
1175
<B>1.11 Drive Cyber Growth</B> <br /><br />
1177
i. Work with other Government departments and UK Trade and Investment (UKTI) to stimulate the growth of the UK cyber security sector, and with GCHQ to more directly harness its expertise and intellectual property to support economic growth<br /><br />
Started
May 2015
NA
NA
NA
2015-05-31
1
0
Carried over
Not due
NA
NA
NA
NA
NA
NA
1
Started
May 2015
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
993
2.1 Drive efficiency savings to save taxpayers’ money and protect frontline services
1035
2.1.i. Support Whitehall departments in delivering £20 billion of savings per year by 2014/15
The latest update on progress was published on 10 June 2014 (at https://www.gov.uk/government/news/government-unveils-143-billion-of-savings-for-2013-to-2014), reporting that the Cabinet Office’s Efficiency and Reform Group worked with departments across government to realise savings of £14.3billion for 2013/14. This continues ERG’s work to deliver savings of £20 billion by 2015.
Started
Mar 2015
NA
NA
NA
2015-03-31
1
0
Carried over
Not due
NA
NA
NA
NA
NA
NA
1
Started
Mar 2015
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
995
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
1037
2.2.i. Publish report on the work of the Mutuals Taskforce, the first annual report on the wider mutuals programme and a ‘one-year on’ report
http://mutuals.cabinetoffice.gov.uk/mutuals-taskforce-launch-one-year-update
Started
Oct 2013
NA
2013-07-04
NA
2013-10-31
1
1
Carried over
Early
NA
NA
NA
89
NA
NA
1
Started
Oct 2013
2013-07-04 00:00:00
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
995
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
1039
2.2.ii. Support public sector spin-outs including 50 mutuals in local authority control and further spin-outs in central government
There are now over 70 mutuals in operation: https://www.gov.uk/government/news/nudge-unit-forming-mutual-joint-venture
Started
Apr 2014
NA
2014-02-28
NA
2014-04-30
1
1
Carried over
Early
NA
NA
NA
32
NA
NA
1
Started
Apr 2014
2014-02-28 00:00:00
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
995
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
1041
2.2.iii. Launch a new commercial entity for the provision of shared services, as part of the Next Generation Shared Services strategy and in line with the Civil Service Reform Plan
On 1 November 2013 it was announced that Steria Limited has been appointed as the private sector partner in a deal that will save taxpayers money by consolidating the government’s back office functions. As partner in a new Joint Venture company, Shared Services Connected Limited (SSCL), Steria will run the second of two new Independent Shared Services Centres (ISSC2). The Joint Venture will harness commercial expertise to deliver government back office functions more efficiently and is a key example of driving innovation in government commercial models – both key parts of the Civil Service Reform Plan. https://www.gov.uk/government/news/shared-service-strategy-to-save-taxpayer-money
Started
Mar 2014
NA
2013-11-30
NA
2014-03-31
1
1
Carried over
Early
NA
NA
NA
91
NA
NA
1
Started
Mar 2014
2013-11-30 00:00:00
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
995
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
1043
2.2.iv. Commence three procurements with the aim of transforming the business models of Government services
The three procurements are: Steria Ltd taken on shared services work, Behavioural Insights Team forming a joint venture with Nesta and Capita forming a joint venture on government intellectual property: https://www.gov.uk/government/news/new-deal-will-market-government-professional-qualifications, https://www.gov.uk/government/news/nudge-unit-forming-mutual-joint-venture https://www.gov.uk/government/news/shared-service-strategy-to-save-taxpayer-money
Started
Mar 2014
NA
2014-03-31
NA
2014-03-31
1
1
Carried over
On time
NA
NA
NA
NA
NA
NA
1
Started
Mar 2014
2014-03-31 00:00:00
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
995
2.2 Support new forms of provision in the public sector, including mutuals, co-operatives, joint ventures and other commercial models
3657
2.2.v. With the Department of Communities and Local Government source professional support for 10 pioneering local authorities to consider and implement new delivery models (including VCSE, mutuals, joint ventures etc.) for their services and disseminate learning through the Delivering Differently programme
Started
Mar 2015
NA
NA
NA
2015-03-31
1
0
Carried over
Not due
NA
NA
NA
NA
NA
NA
1
Started
Mar 2015
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
997
2.3 Introduce greater choice in and control of public services for individuals and communities through Open Public Services
1047
2.3.i. Appoint National Advisor for What Works, to support the development of What Works Centres across £200 billion of public spending
https://www.gov.uk/government/news/david-halpern-appointed-as-what-works-national-adviser
Started
Jul 2013
2013-07-02
2013-07-02
NA
2013-07-31
1
1
Carried over
On time
NA
NA
NA
NA
NA
NA
1
Started
Jul 2013
2013-07-02 00:00:00
2013-07-02 00:00:00
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
997
2.3 Introduce greater choice in and control of public services for individuals and communities through Open Public Services
1049
2.3.ii. Publish Open Public Services 2014, the annual account of progress and next steps in reforming public services
The Government published the 2014 progress report on 14 March 2014. Here is a link to the publication: https://www.gov.uk/government/publications/open-public-services-2014-progress-report
April 2014
Apr 2014
2014-03-17
2014-03-14
2014-04-30
2014-04-30
1
1
Early
Early
15
NA
NA
18
NA
NA
0
April 2014
Apr 2014
2014-03-17 00:00:00
2014-03-14 00:00:00
2014-08-30 12:46:06
2014-08-30
CO
Cabinet Office
1
http://www.cabinetoffice.gov.uk
2. Drive efficiency and effectiveness in government
7
Help tackle the deficit by spending less taxpayers’ money and spending it more wisely, collecting consistent management information to improve the efficiency of Government, addressing fraud, error and uncollected debt, and contributing to the reform of Government to improve its effectiveness. Reform the Civil Service including to make it more unified and open, sharpen accountability, address skills gaps and improve delivery and implementation (Lead Officials: Stephen Kelly, Will Cavendish (Actions 2.3 and 2.11); Sue Gray (Action 2.7); Katherine Kerswell (Actions 2.9 and 2.10))
999
2.4 Reform procurement to support growth and save the taxpayer money
1051
2.4.i. Transfer all common goods and services spend to centralised arrangements
The transfer to a centralised arrangement has been planned and agreed. The first phase of transition is underway and will take place in stages across 2014-15.
Started
Dec 2013
NA
NA
NA
2013-12-31
1
0
Carried over
Overdue
NA
NA
NA
NA
NA
242
1
Started
Dec 2013
2014-08-30 12:46:06
2014-08-30

Statistics

Average successful run time: about 3 hours

Total run time: about 1 month

Total cpu time used: 16 minutes

Total disk space used: 131 MB

History

  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Manually ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and failed .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    98 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    106 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    107 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    49 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    43 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    4 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    2 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
    105 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Manually ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and failed .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and failed .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
    4 pages scraped
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Manually ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    1384 records added in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 9949ecb3 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and failed .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    1384 records added in the database
  • Auto ran revision 36ceaa32 and completed successfully .
    nothing changed in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1385 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1385 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1385 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1385 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1385 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1384 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1382 records added in the database
  • Manually ran revision 4b4fc83b and completed successfully .
    1371 records added in the database
  • Manually ran revision ae14ec13 and failed .
    nothing changed in the database
  • Manually ran revision eea3c078 and failed .
    nothing changed in the database
  • Manually ran revision 5a5e870d and failed .
    nothing changed in the database
  • Manually ran revision c0a3cf32 and failed .
    nothing changed in the database
  • Manually ran revision 54371a61 and failed .
    nothing changed in the database
  • Manually ran revision 54371a61 and failed .
    nothing changed in the database
  • Manually ran revision 54371a61 and failed .
    nothing changed in the database
  • Manually ran revision 54371a61 and failed .
    nothing changed in the database
  • Manually ran revision 69905816 and failed .
    nothing changed in the database
  • Manually ran revision 50ff160a and failed .
    nothing changed in the database
  • Manually ran revision e3ae89c8 and failed .
  • Manually ran revision e3ae89c8 and failed .
  • Created on morph.io

Scraper code

Python

GovBusinessPlans / scraper.py