{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Make your own forcing data from ERA5\n",
    "\n",
    "Introduction: There are lots of forcing data can be used to drive the model, such as ERA5, NCEP (https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html), etc. Here, we recommend ERA5 to make your own single point forcing as the NCEP and others are too big to download. The ERA5 data can be obtained in a small scale, which usually light weight to download and store."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the required variables from the cdsapi\n",
    "\n",
    "First, we will use the `cdsapi` package to download the data. If you don't have the package installed, you can install it using the following command:\n",
    "\n",
    "```\n",
    "pip install cdsapi\n",
    "\n",
    "cat <<EOF > ~/.cdsapirc\n",
    "url: {api-url}\n",
    "key: {uid}:{api-key}\n",
    "EOF \n",
    "```\n",
    "\n",
    "[How to get your CDS API?](https://cds.climate.copernicus.eu/how-to-api)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**note: it will take a long time to run this script, so you can run it in the background and check the output file later.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also can use the interface to download the data. The code below is an example of how to download the data using the interface.\n",
    "\n",
    "ref: https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=form"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-06-27 12:00:16,424 INFO Welcome to the CDS\n",
      "2024-06-27 12:00:16,424 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "download: ./era5_data/era5_single_2012_01_51.5_0.12.nc\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-06-27 12:00:16,539 INFO Request is completed\n",
      "2024-06-27 12:00:16,541 INFO Downloading https://download-0019.copernicus-climate.eu/cache-compute-0019/cache/data4/adaptor.mars.internal-1719485113.5101933-9971-11-4b2c9802-6dcb-4b24-9e4a-2705adec9046.nc to ./era5_data/era5_single_2012_01_51.5_0.12.nc (123.8K)\n",
      "2024-06-27 12:00:16,865 INFO Download rate 382.4K/s\n",
      "2024-06-27 12:00:17,862 INFO Welcome to the CDS\n",
      "2024-06-27 12:00:17,862 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels\n",
      "2024-06-27 12:00:17,958 INFO Request is completed\n",
      "2024-06-27 12:00:17,958 INFO Downloading https://download-0018.copernicus-climate.eu/cache-compute-0018/cache/data8/adaptor.mars.internal-1719485668.0006976-4897-15-6013a414-d43b-4ed3-820a-8cdfd184bfe1.nc to ./era5_data/era5_single_2012_02_51.5_0.12.nc (116K)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "download: ./era5_data/era5_single_2012_02_51.5_0.12.nc\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-06-27 12:00:18,261 INFO Download rate 382.6K/s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'/Users/user/Documents/GitHub/pyclmuapp/era5_data/era5_forcing_51.5_0.12_30_2012_1_2012_2.nc'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pyclmuapp import get_forcing\n",
    "lat=51.5\n",
    "lon=0.12\n",
    "zbot=30\n",
    "start_year=2012\n",
    "end_year=2012\n",
    "start_month=1\n",
    "end_month=2\n",
    "get_forcing(\n",
    "    lat=lat, lon=lon, zbot=zbot, \n",
    "    start_year=start_year, end_year=end_year, \n",
    "    start_month=start_month, end_month=end_month,\n",
    "    source='cds')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### command line\n",
    "this is same as above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Namespace(init=False, pwd='/Users/user/Documents/GitHub/pyclmuapp', container_type='docker', input_path=None, output_path=None, log_path=None, scripts_path=None, pyclmuapp_mode='get_forcing', has_container=True, usr_domain=None, usr_forcing=None, usr_surfdata=None, output_prefix='_clm.nc', case_name='usp_case', run_startdate=None, stop_option='ndays', stop_n='1', run_type='coldstart', run_refcase='None', run_refdate='None', iflog=True, logfile='pyclmuapp.log', hist_type='GRID', hist_nhtfrq=1, hist_mfilt=1000000000, clean=False, surf_var=None, surf_action=0, forcing_var=None, forcing_action=0, script=None, urbsurf=None, soildata=None, pct_urban=[0, 0, 100.0], lat=51.5, lon=0.12, outputname='surfdata.nc', zbot=30, start_year=2012, end_year=2012, start_month=1, end_month=2)\n",
      "download: ./era5_data/era5_single_2012_01_51.5_0.12.nc\n",
      "2024-06-27 12:00:31,297 INFO Welcome to the CDS\n",
      "2024-06-27 12:00:31,297 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels\n",
      "2024-06-27 12:00:31,373 INFO Request is completed\n",
      "2024-06-27 12:00:31,373 INFO Downloading https://download-0019.copernicus-climate.eu/cache-compute-0019/cache/data4/adaptor.mars.internal-1719485113.5101933-9971-11-4b2c9802-6dcb-4b24-9e4a-2705adec9046.nc to ./era5_data/era5_single_2012_01_51.5_0.12.nc (123.8K)\n",
      "2024-06-27 12:00:31,674 INFO Download rate 411.9K/s                             \n",
      "download: ./era5_data/era5_single_2012_02_51.5_0.12.nc\n",
      "2024-06-27 12:00:32,532 INFO Welcome to the CDS\n",
      "2024-06-27 12:00:32,532 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels\n",
      "2024-06-27 12:00:32,637 INFO Request is completed\n",
      "2024-06-27 12:00:32,637 INFO Downloading https://download-0018.copernicus-climate.eu/cache-compute-0018/cache/data8/adaptor.mars.internal-1719485668.0006976-4897-15-6013a414-d43b-4ed3-820a-8cdfd184bfe1.nc to ./era5_data/era5_single_2012_02_51.5_0.12.nc (116K)\n",
      "2024-06-27 12:00:32,908 INFO Download rate 427.7K/s                             \n"
     ]
    }
   ],
   "source": [
    "! pyclmuapp --pyclmuapp_mode get_forcing \\\n",
    "    --lat 51.5 --lon 0.12 --zbot 30 \\\n",
    "    --start_year 2012 --end_year 2012 \\\n",
    "    --start_month 1 --end_month 2\n",
    "# will download and save in the default folder `./era5_forcing/`\n",
    "# the output file will be `./era5_forcing/era5_forcing_51.5_0.12_30_2012_01_2012_2.nc`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**test the forcing data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Namespace(init=False, pwd='/Users/user/Documents/GitHub/pyclmuapp', container_type='docker', input_path=None, output_path=None, log_path=None, scripts_path=None, pyclmuapp_mode='usp', has_container=True, usr_domain=None, usr_forcing='era5_data/era5_forcing_51.5_0.12_30_2012_1_2012_2.nc', usr_surfdata='inputfolder/surfdata.nc', output_prefix='_clm.nc', case_name='pyclmuapp', run_startdate='2012-01-01', stop_option='ndays', stop_n='15', run_type='coldstart', run_refcase='None', run_refdate='None', iflog=True, logfile='pyclmuapp.log', hist_type='GRID', hist_nhtfrq=1, hist_mfilt=1000000000, clean='True', surf_var=None, surf_action=0, forcing_var=None, forcing_action=0, script=None, urbsurf=None, soildata=None, pct_urban=[0, 0, 100.0], lat=None, lon=None, outputname='surfdata.nc', zbot=30, start_year=2012, end_year=2012, start_month=1, end_month=12)\n",
      "Folder 'inputfolder' already exists.\n",
      "Folder 'outputfolder' already exists.\n",
      "Folder 'logfolder' already exists.\n",
      "Folder 'scriptsfolder' already exists.\n",
      "Folder '/Users/user/Documents/GitHub/pyclmuapp/inputfolder/usp' created successfully!\n",
      "Copying the file era5_forcing_51.5_0.12_30_2012_1_2012_2.nc to the /Users/user/Documents/GitHub/pyclmuapp/inputfolder/usp\n",
      "The domain file is not provided\n",
      "The case is:  pyclmuapp\n",
      "The log file is:  pyclmuapp.log\n",
      "The output file is:  {'original': ['/Users/user/Documents/GitHub/pyclmuapp/outputfolder/lnd/hist/pyclmuapp_clm0_2024-06-27_12-00-50_clm.nc']}\n"
     ]
    }
   ],
   "source": [
    "! pyclmuapp \\\n",
    "    --container_type docker \\\n",
    "    --iflog True \\\n",
    "    --logfile \"pyclmuapp.log\" \\\n",
    "    --usr_forcing \"era5_data/era5_forcing_51.5_0.12_30_2012_1_2012_2.nc\" \\\n",
    "    --usr_surfdata \"inputfolder/surfdata.nc\" \\\n",
    "    --RUN_STARTDATE \"2012-01-01\" --STOP_OPTION \"ndays\" --STOP_N \"15\" \\\n",
    "    --RUN_TYPE \"coldstart\" \\\n",
    "    --hist_type \"GRID\" --hist_nhtfrq \"1\" \\\n",
    "    --hist_mfilt \"1000000000\" \\\n",
    "    --output_prefix \"_clm.nc\" \\\n",
    "    --CASE_NAME \"pyclmuapp\" \\\n",
    "    --run_type \"coldstart\" --run_refcase \"None\" --run_refdate \"None\" --clean True"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "u_climate_yjj",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}