codeforIATI / humanitarian-emergency-codelists

Humanitarian emergency codelists scraper


Humantarian emergency codelists

This repository contains a scraper which collects reference codelists from the following sources:

The scraper runs every night and the latest data is available to download on morph.io. You'll need to create a free account to download the data.

Alternatively, you can also run this code yourself.

Contributors markbrough andylolz

Last run completed successfully .

Console output of last run

Injecting configuration and compiling...  -----> Python app detected -----> Installing python-3.6.2 -----> Installing pip -----> Installing requirements with pip  Collecting lxml  Downloading lxml-4.8.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.4 MB)  Collecting requests  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)  Collecting scraperwiki==0.5.1  Downloading scraperwiki-0.5.1.tar.gz (7.7 kB)  Preparing metadata (setup.py): started  Preparing metadata (setup.py): finished with status 'done'  Collecting six  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)  Collecting sqlalchemy  Downloading SQLAlchemy-1.4.36-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)  Collecting alembic  Downloading alembic-1.7.7-py3-none-any.whl (210 kB)  Collecting charset-normalizer~=2.0.0  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)  Collecting idna<4,>=2.5  Downloading idna-3.3-py3-none-any.whl (61 kB)  Collecting urllib3<1.27,>=1.21.1  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)  Collecting certifi>=2017.4.17  Downloading certifi-2022.5.18.1-py3-none-any.whl (155 kB)  Collecting importlib-metadata  Downloading importlib_metadata-4.8.3-py3-none-any.whl (17 kB)  Collecting importlib-resources  Downloading importlib_resources-5.4.0-py3-none-any.whl (28 kB)  Collecting Mako  Downloading Mako-1.1.6-py2.py3-none-any.whl (75 kB)  Collecting greenlet!=0.4.17  Downloading greenlet-1.1.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (147 kB)  Collecting zipp>=0.5  Downloading zipp-3.6.0-py3-none-any.whl (5.3 kB)  Collecting typing-extensions>=3.6.4  Downloading typing_extensions-4.1.1-py3-none-any.whl (26 kB)  Collecting MarkupSafe>=0.9.2  Downloading MarkupSafe-2.0.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)  Building wheels for collected packages: scraperwiki  Building wheel for scraperwiki (setup.py): started  Building wheel for scraperwiki (setup.py): finished with status 'done'  Created wheel for scraperwiki: filename=scraperwiki-0.5.1-py3-none-any.whl size=6545 sha256=f9688069b2272a821609a57e82dadb99b49ab459f56e26fbb49923c60806b838  Stored in directory: /tmp/pip-ephem-wheel-cache-dtvtxp_i/wheels/cd/f8/ac/cd66eb1c557ab40d35c1ed852da3e9b37baa3e21b61906a5cf  Successfully built scraperwiki  Installing collected packages: zipp, typing-extensions, MarkupSafe, importlib-metadata, greenlet, urllib3, sqlalchemy, Mako, importlib-resources, idna, charset-normalizer, certifi, six, requests, alembic, scraperwiki, lxml  Successfully installed Mako-1.1.6 MarkupSafe-2.0.1 alembic-1.7.7 certifi-2022.5.18.1 charset-normalizer-2.0.12 greenlet-1.1.2 idna-3.3 importlib-metadata-4.8.3 importlib-resources-5.4.0 lxml-4.8.0 requests-2.27.1 scraperwiki-0.5.1 six-1.16.0 sqlalchemy-1.4.36 typing-extensions-4.1.1 urllib3-1.26.9 zipp-3.6.0   -----> Discovering process types  Procfile declares types -> scraper Injecting scraper and running... Opening GLIDEnumber.net Requesting list of GLIDE numbers, this may take a moment... Found 7539 entries Irregular column width, skipping Irregular column width, skipping /app/.heroku/python/lib/python3.6/site-packages/scraperwiki/sql.py:75: SAWarning: SQLite version (3, 7, 9) is older than 3.7.16, and will not support right nested joins, as are sometimes used in more complex ORM scenarios. SQLAlchemy 1.4 and above no longer tries to rewrite these joins. connect_args={'timeout': DATABASE_TIMEOUT})

Data

Downloaded 1 time by andylolz

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (3.65 MB) Use the API

rows 10 / 7539

GLIDE_number Event Country Date Event_Code Country_Code Glide_Serial Comments URL
EP-2021-000048-KEN
Epidemic
Kenya
2021-04-28
EP
KEN
2021-000048
According to the county department of health in Mombasa the first dengue cases were reported in early March 2021 with 24 cases testing positive out of 47 (51% positivity rate). In April another 305 cases tested positive out of 315 (97% positivity rate). The adjacent Lamu county, has also reported a total of 224 positive cases from different health facilities where 59 are children under 5 years old. Cumulatively, 529 cases have been reported within the past 2 months of March and April. No deaths have been reported so far within the two counties.
OT-2022-000166-SYR
Other
Syrian Arab Rep
2022-02-10
OT
SYR
2022-000166
On 20 January 2022, an attack by ISg's on Sina'a prison in Al-Hasakeh city resulted in rapidly escalating hostilities with gunfire and explosions reported from the area. Civilian casualties were reported, and up to 75,000 people (15,000 families) were forced to move out from the neighbourhoods or between houses where the main hostilities and fighting took place. Most of the displaced people have sought safety with family and friends in Al Hasakeh city and its surrounding neighbourhoods, while others have been hosted at seven temporary shelters. The pre-crises conditions in the affected neighbourhoods and host communities were already critical, with low resilience among the population to withstand any additional shocks. The harsh winter conditions exacerbated by the lack of electricity, fuel and transportation, combined with economic crises that have sent food and basic items prices sky-rocketing, impact from COVID-19 had already depleted the coping capacities among the host communities. Families depending on daily labour for their livelihoods lost their income due to the lockdown and curfews not only in Al-Hasakeh city but also in the governorate at large. While the lockdown has been lifted, there is still a partial curfew in effect and movements are strictly controlled.
FL-2022-000217-BGD
Flood
Bangladesh
2022-05-22
FL
BGD
2022-000217
Heavy rainfall continues to cause widespread and severe flooding and landslides across parts of Bangladesh and north-eastern India, resulting in increased human impact. In Bangladesh, at least 10 people died and over 2 million were affected.
FL-2022-000217-IND
Flood
India
2022-05-22
FL
IND
2022-000217
Heavy rainfall continues to cause widespread and severe flooding and landslides across parts of Bangladesh and north-eastern India, resulting in increased human impact. According to media reports in India, at least 33 people died in Bihar State due to floods, thunderstorms and lightning, while floods and landslides in Assam State caused at least 14 fatalities and affected about 1.7 Million people in 29 districts.
CE-2022-000216-BFA
Complex Emergency
Burkina Faso
2022-05-12
CE
BFA
2022-000216
Burkina Faso is currently experiencing a multi-faceted crisis characterised by armed conflicts and inter-community violence, leading to the desertion of arable land and a large flow of people, which in turn puts pressure on the production and consumption structure of the host communities, creating a snowball effect of the food crisis from the northern part of the country to other localities further south. Localized deficits in cereal production coupled with reduced cross-border trade and a general increase in high food prices linked to the country's macroeconomic challenges, the closure of some European input and food supply routes (with the war in Ukraine) as well as the socio-economic impact of the COVID-19 pandemic have also largely contributed to the growing food deficit. Projections show 12 provinces in minimum phase, 17 provinces in pressure phase, 11 provinces in crisis and 5 provinces in emergency. The vulnerable population in need of immediate assistance would be 3,453,510 people, or 16% of the total population. These people would be concentrated in the Sahel (20%), Centre-North (17%), East (17%) and North (16%) regions.
DR-2022-000215-AFG
Drought
Afghanistan
2022-05-23
DR
AFG
2022-000215
GDACS - Medium impact for agricultural drought in 625576 km2 over Syria, Iraq, Iran, Turkey, Pakistan-2021
VW-2022-000214-SOM
Violent Wind
Somalia
2022-05-10
VW
SOM
2022-000214
This is following sudden heavy rains and windstorms which struck North Gaalkacyo on 10th May, impacting the city of Gaalkacyo and its surrounding areas, which host the camps. The most affected people by this disaster are approximately 4,416 internally displaced people (736 HH) people living in 18 of the 20 recorded camps.
FL-2022-000213-IND
Flood
India
2022-05-13
FL
IND
2022-000213
Heavy rainfall has been affecting northeastern India (in particular Arunchal Pradesh, Assam, and Meghalaya States) since 13 May 2022, triggering landslides and causing river overflow and floods that have resulted in casualties and damage. Media report, as of 16 May, seven fatalities, of which one in Itanagar City (southern Arunchal Pradesh), due to a landslide, three in the Dima Hasao District (central Assam) also in this case due to a landslide, and three more across central Meghalaya (two in East Khasi Hills District and one Ri-Bhoi District) due to floods.
LS-2022-000212-COD
Land Slide
Congo Democratic Republic
2022-04-25
LS
COD
2022-000212
A landslide occurred in Kadutu commune of Bukavu, capital of South Kivu Province, in eastern provinces of the Democratic Republic of the Congo (DR Congo) on 25 April. At least 8 people died and 9 were seriously injured. Since then the rain has continued, affecting Maniema, North Kivu and South Kivu provinces.
EP-2022-000211-TZA
Epidemic
Tanzania
2022-04-23
EP
TZA
2022-000211
On 23rd April 2022, the government reported cholera cases in Uvinza DC (Kigoma region) and Tanganyika DC (Katavi region). These cases were reported in Kalya ward in Uvinza DC which has a population of 22,486 (Kigoma region) and Karema and Ikola wards with a population of 15,982) Tanganyika district, Katavi region).

Statistics

Average successful run time: 1 minute

Total run time: about 9 hours

Total cpu time used: about 2 hours

Total disk space used: 21.2 MB

History

  • Auto ran revision 2cd2292d and completed successfully .
    7537 records added, 7533 records removed in the database
  • Auto ran revision 2cd2292d and completed successfully .
    7533 records added, 7533 records removed in the database
  • Auto ran revision 2cd2292d and completed successfully .
    7533 records added, 7533 records removed in the database
  • Auto ran revision 2cd2292d and completed successfully .
    7533 records added, 7533 records removed in the database
  • Auto ran revision 2cd2292d and completed successfully .
    7533 records added, 7533 records removed in the database
  • ...
  • Created on morph.io

Show complete history