codeforIATI / dac-sector-groups

A morph.io scraper to identify the higher-level groups from DAC CRS sector codes


Contributors markbrough

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-3.6.2 -----> Installing pip -----> Installing requirements with pip  Collecting lxml==4.6.1  Downloading lxml-4.6.1-cp36-cp36m-manylinux1_x86_64.whl (5.5 MB)  Collecting requests==2.24.0  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)  Collecting scraperwiki==0.5.1  Downloading scraperwiki-0.5.1.tar.gz (7.7 kB)  Collecting idna<3,>=2.5  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)  Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)  Collecting chardet<4,>=3.0.2  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)  Collecting six  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)  Collecting sqlalchemy  Downloading SQLAlchemy-1.3.20-cp36-cp36m-manylinux2010_x86_64.whl (1.3 MB)  Collecting alembic  Downloading alembic-1.4.3-py2.py3-none-any.whl (159 kB)  Collecting python-dateutil  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)  Collecting Mako  Downloading Mako-1.1.3-py2.py3-none-any.whl (75 kB)  Collecting python-editor>=0.3  Downloading python_editor-1.0.4-py3-none-any.whl (4.9 kB)  Collecting MarkupSafe>=0.9.2  Downloading MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl (27 kB)  Building wheels for collected packages: scraperwiki  Building wheel for scraperwiki (setup.py): started  Building wheel for scraperwiki (setup.py): finished with status 'done'  Created wheel for scraperwiki: filename=scraperwiki-0.5.1-py3-none-any.whl size=6545 sha256=ae4356a60f4b89a92b6cdfc091de3bf40df9bc7175c744e4fd210702cb583dae  Stored in directory: /tmp/pip-ephem-wheel-cache-7_k5yg0h/wheels/cd/f8/ac/cd66eb1c557ab40d35c1ed852da3e9b37baa3e21b61906a5cf  Successfully built scraperwiki  Installing collected packages: lxml, idna, certifi, urllib3, chardet, requests, six, sqlalchemy, python-dateutil, MarkupSafe, Mako, python-editor, alembic, scraperwiki  Successfully installed Mako-1.1.3 MarkupSafe-1.1.1 alembic-1.4.3 certifi-2020.11.8 chardet-3.0.4 idna-2.10 lxml-4.6.1 python-dateutil-2.8.1 python-editor-1.0.4 requests-2.24.0 scraperwiki-0.5.1 six-1.15.0 sqlalchemy-1.3.20 urllib3-1.25.11   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running...

Data

Downloaded 316 times by markbrough pacoDevelop

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (29 KB) Use the API

rows 10 / 227

group_code group_name category_code category_name code name
110
Education
111
Education, Level Unspecified
11110
Education policy and administrative management
110
Education
111
Education, Level Unspecified
11120
Education facilities and training
110
Education
111
Education, Level Unspecified
11130
Teacher training
110
Education
111
Education, Level Unspecified
11182
Educational research
110
Education
112
Basic Education
11220
Primary education
110
Education
112
Basic Education
11230
Basic life skills for youth and adults
110
Education
112
Basic Education
11240
Early childhood education
110
Education
112
Basic Education
11250
School feeding
110
Education
113
Secondary Education
11320
Secondary education
110
Education
113
Secondary Education
11330
Vocational training

Statistics

Average successful run time: 1 minute

Total run time: 5 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 53.4 KB

History

  • Manually ran revision 02605ccd and completed successfully .
    227 records added in the database
  • Manually ran revision a3bb9bdc and failed .
  • Manually ran revision be13f897 and failed .
  • Created on morph.io

Scraper code

Python

dac-sector-groups / scraper.py