Insights

Working with the CHIRPS Dataset

January 31, 2019
Colorful lights being projected onto a woman. Photo credit: ThisIsEngineering.Colorful lights being projected onto a woman. Photo credit: ThisIsEngineering.

High-quality precipitation data is integral to creating accurate, localized, and effective index insurance contracts. In the Arbol white paper, we detail the use of Global Historical Climatology Network (GHCN) station data, which has daily climate summaries from over 100,000 land surface stations in 180 countries. While this data set is certainly helpful to our mission at Arbol, we continue to look for other sources of high quality and publicly available data as no one data set offers sufficient coverage and lack of measurement biases. Specifically, we have been working with Climate Hazards Group Infrared Precipitation (CHIRPS), which incorporates satellite data along with station data to create gridded rainfall time series.

CHIRPS

CHIRPS is a quasi-global rainfall dataset with over 30 years of data collected. This data set is considered quasi-global since it spans from 50 degrees South to 50 degrees North, which is roughly from the bottom tip of South America to the middle of Canada. It covers all latitudes so for agricultural analysis we believe all major production regions are encompassed in this data set. CHIRPS was created in order to deliver reliable, complete, and up to date data that can be used to understand trends in rainfall variations over time. Specifically, dry seasons are best understood in the context of historical data in order to understand the severity of a potential drought and the scale of impact. Satellite data can be used to understand these trends as well but the areal averages provided by satellite data are not always as up to date and suffer from biases due to complex terrain. This causes satellite data to underestimate the intensity of extreme precipitation events. However, CHIRPS data has the same limitations as all station data: in rural areas, the measurements are impacted due to fewer rain gauge stations.

We have found that CHIRPS data has provided a flexible and easy to work with complementary data set. In areas with less density of weather stations such as developing countries, interpolating station and satellite data adds value. We can reuse most of our same internal analytic tools that were built for GHCN to work with CHIRPS. CHIRPS has 0.05 degree resolution as well as a less granular 0.25 degree resolution. The following is an adaption of our GHCN FTP code which will allow you to download and experiment with CHIRPS data:

#use this to download data from ftpfrom ftplib import FTPimport os, sys, os.pathddir='/Users/Osho/CHIRPS/'ftp = FTP("ftp.chg.ucsb.edu")ftp.login()#get 0.25 resolution filesdirectory = '/pub/org/chg/products/CHIRPS-2.0/global_daily/netcdf/p25/'print ('Changing to ' + directory)ftp.cwd(directory)filenames = ftp.nlst()#first element in list is a monthly directory linkdel filenames[0]for filename in filenames:     local_filename = os.path.join(ddir, filename)     print(local_filename)     file = open(local_filename, 'wb')     ftp.retrbinary('RETR '+ filename, file.write)     file.close()ftp.quit() # This is the “polite” way to close a connectionSources:

http://chg.geog.ucsb.edu/data/chirps/

https://www.ncdc.noaa.gov/ghcn-daily-description

Disclaimer

Continue reading