{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tutorial 2: Aggregation to Precipitation Sums\n", "This tutorial will show you, how to calculate precipitation sums from the data stored in HDF5 and how to export the results to ArcGIS.\n", "\n", "To import your RADOLAN data into the necessary HDF5 file, please follow the tutorial on raw data processing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example 1: Annual Precipitation Sums\n", "In this example, the annual precipitation sums for the time period from 2001 to 2016 are calculated and exported to ArcGIS. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Import radproc" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import radproc as rp" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. Load Data from HDF5 and Aggregate to Annual Precipitation Sums \n", "\n", "The following function loads precipitation data of the specified time period (2001-2016) from an HDF5 file and generates a DataFrame with annual precipitation sums for every raster cell." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Rasterzellen-ID427005427006427903427904427905427906428803428804428805428806...661855661856661857662752662753662754662755662756663652663655
Datum (UTC)
2001-12-31 00:00:00+00:00783.400024809.500000763.299988773.500000779.700012788.500000769.500000780.400024778.700012783.000000...1066.5999761098.6999511125.400024994.000000998.7000121020.5999761056.0999761075.200073989.2000121039.500000
2002-12-31 00:00:00+00:00967.700012980.799988951.000000935.500000947.200012950.700012956.000000960.700012938.700012955.400024...1491.5999761529.5000001539.1999511383.0999761387.5000001407.4000241478.0999761488.8000491380.4000241467.099976
2003-12-31 00:00:00+00:00597.200012614.200012578.000000583.000000592.200012592.299988573.400024587.700012587.299988589.600037...697.600037696.099976683.700012670.400024652.599976638.400024663.599976668.599976663.099976648.000000
2004-12-31 00:00:00+00:00795.000000801.000000772.000000768.099976776.600037792.799988746.500000767.099976779.000000792.900024...851.599976872.200012869.799988803.000000802.000000804.099976825.099976841.599976806.000000813.700012
2005-12-31 00:00:00+00:00728.099976710.799988678.500000676.799988730.200012681.399963671.400024664.099976658.799988666.299988...914.799988922.900024911.099976847.799988861.500000868.099976893.200012903.299988850.799988872.500000
\n", "

5 rows × 23320 columns

\n", "
" ], "text/plain": [ "Rasterzellen-ID 427005 427006 427903 427904 \\\n", "Datum (UTC) \n", "2001-12-31 00:00:00+00:00 783.400024 809.500000 763.299988 773.500000 \n", "2002-12-31 00:00:00+00:00 967.700012 980.799988 951.000000 935.500000 \n", "2003-12-31 00:00:00+00:00 597.200012 614.200012 578.000000 583.000000 \n", "2004-12-31 00:00:00+00:00 795.000000 801.000000 772.000000 768.099976 \n", "2005-12-31 00:00:00+00:00 728.099976 710.799988 678.500000 676.799988 \n", "\n", "Rasterzellen-ID 427905 427906 428803 428804 \\\n", "Datum (UTC) \n", "2001-12-31 00:00:00+00:00 779.700012 788.500000 769.500000 780.400024 \n", "2002-12-31 00:00:00+00:00 947.200012 950.700012 956.000000 960.700012 \n", "2003-12-31 00:00:00+00:00 592.200012 592.299988 573.400024 587.700012 \n", "2004-12-31 00:00:00+00:00 776.600037 792.799988 746.500000 767.099976 \n", "2005-12-31 00:00:00+00:00 730.200012 681.399963 671.400024 664.099976 \n", "\n", "Rasterzellen-ID 428805 428806 ... 661855 \\\n", "Datum (UTC) ... \n", "2001-12-31 00:00:00+00:00 778.700012 783.000000 ... 1066.599976 \n", "2002-12-31 00:00:00+00:00 938.700012 955.400024 ... 1491.599976 \n", "2003-12-31 00:00:00+00:00 587.299988 589.600037 ... 697.600037 \n", "2004-12-31 00:00:00+00:00 779.000000 792.900024 ... 851.599976 \n", "2005-12-31 00:00:00+00:00 658.799988 666.299988 ... 914.799988 \n", "\n", "Rasterzellen-ID 661856 661857 662752 662753 \\\n", "Datum (UTC) \n", "2001-12-31 00:00:00+00:00 1098.699951 1125.400024 994.000000 998.700012 \n", "2002-12-31 00:00:00+00:00 1529.500000 1539.199951 1383.099976 1387.500000 \n", "2003-12-31 00:00:00+00:00 696.099976 683.700012 670.400024 652.599976 \n", "2004-12-31 00:00:00+00:00 872.200012 869.799988 803.000000 802.000000 \n", "2005-12-31 00:00:00+00:00 922.900024 911.099976 847.799988 861.500000 \n", "\n", "Rasterzellen-ID 662754 662755 662756 663652 \\\n", "Datum (UTC) \n", "2001-12-31 00:00:00+00:00 1020.599976 1056.099976 1075.200073 989.200012 \n", "2002-12-31 00:00:00+00:00 1407.400024 1478.099976 1488.800049 1380.400024 \n", "2003-12-31 00:00:00+00:00 638.400024 663.599976 668.599976 663.099976 \n", "2004-12-31 00:00:00+00:00 804.099976 825.099976 841.599976 806.000000 \n", "2005-12-31 00:00:00+00:00 868.099976 893.200012 903.299988 850.799988 \n", "\n", "Rasterzellen-ID 663655 \n", "Datum (UTC) \n", "2001-12-31 00:00:00+00:00 1039.500000 \n", "2002-12-31 00:00:00+00:00 1467.099976 \n", "2003-12-31 00:00:00+00:00 648.000000 \n", "2004-12-31 00:00:00+00:00 813.700012 \n", "2005-12-31 00:00:00+00:00 872.500000 \n", "\n", "[5 rows x 23320 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "HDF = r\"O:\\Data\\RW_2001_2016.h5\"\n", "\n", "annualSum = rp.hdf5_to_years(HDFFile=HDF, year_start=2001, year_end=2016)\n", "# Display the first five rows of the new DataFrame\n", "annualSum.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note:** All of radproc's aggregation functions are intended for analysis of longer time periods and currently only work **for entire years starting in January**!\n", "To resample smaller time periods, you can e.g. import and resample months with\n", "\n", " May2016 = rp.load_months_from_hdf5(HDFFile=HDF, year=2016, months=[5])\n", " freq = 'M' # 'M' for monthly sums, 'D' for daily sums, 'H' for hourly sums\n", " monthSum = May2016.resample(frequency=freq, closed = 'right', label = 'right').sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. Export Results into ArcGIS Geodatabase\n", "\n", "The following function exports all rows from the DataFrame calculated above into raster datasets in an ArcGIS File Geodatabase. Optionally, different statistics rasters can be created, e.g. the mean or the maximum value of each cell." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "idRaster = r\"O:\\Data\\idras\"\n", "outGDBPath = r\"O:\\Data\"\n", "GDBName = \"Years_01_16.gdb\"\n", "statistics = [\"mean\", \"max\"]\n", "\n", "rp.export_dfrows_to_gdb(dataDF=annualSum, idRaster=idRaster, outGDBPath=outGDBPath, GDBName=GDBName, statistics=statistics)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The resulting geodatabase will look like this in ArcGIS:\n", "\n", "![YearGDB](img/YearGDB.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example 2: Monthly Precipitation Sums\n", "In this example, the monthly precipitation sums for the year 2016 are calculated and exported to ArcGIS." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Import radproc" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "import radproc as rp" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. Load Data from HDF5 and Aggregate to Monthly Precipitation Sums\n", "\n", "The following function loads precipitation data of the year 2016 from your HDF5 file and generates a DataFrame with monthly precipitation sums for every raster cell." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Rasterzellen-ID427005427006427903427904427905427906428803428804428805428806...661855661856661857662752662753662754662755662756663652663655
Datum (UTC)
2016-01-31 00:00:00+00:0071.40000271.90000268.00000069.09999870.09999871.69999765.50000067.30000366.19999769.800003...128.199997129.500000129.600006120.199997121.300003123.500000125.699997122.900002118.099998120.400002
2016-02-29 00:00:00+00:0068.69999771.19999761.50000064.30000367.30000368.40000258.20000161.70000165.00000064.300003...105.900002106.199997110.09999896.80000396.699997100.099998102.599998101.40000292.80000397.400002
2016-03-31 00:00:00+00:0041.79999941.29999940.29999939.90000240.09999839.29999937.70000138.20000138.79999937.500000...65.50000064.40000265.19999770.00000067.50000064.40000265.09999865.09999869.80000365.500000
2016-04-30 00:00:00+00:0055.70000156.60000251.79999952.40000254.00000055.29999950.79999951.20000152.79999954.400002...108.500000112.199997115.400002106.700005106.500000107.699997112.599998114.300003105.199997108.099998
2016-05-31 00:00:00+00:0038.20000140.60000247.29999944.79999939.40000234.29999946.79999945.40000243.20000138.500000...121.299995119.300003115.099998128.100006138.199997135.500000124.300003128.500000130.199997134.399994
\n", "

5 rows × 23320 columns

\n", "
" ], "text/plain": [ "Rasterzellen-ID 427005 427006 427903 427904 \\\n", "Datum (UTC) \n", "2016-01-31 00:00:00+00:00 71.400002 71.900002 68.000000 69.099998 \n", "2016-02-29 00:00:00+00:00 68.699997 71.199997 61.500000 64.300003 \n", "2016-03-31 00:00:00+00:00 41.799999 41.299999 40.299999 39.900002 \n", "2016-04-30 00:00:00+00:00 55.700001 56.600002 51.799999 52.400002 \n", "2016-05-31 00:00:00+00:00 38.200001 40.600002 47.299999 44.799999 \n", "\n", "Rasterzellen-ID 427905 427906 428803 428804 \\\n", "Datum (UTC) \n", "2016-01-31 00:00:00+00:00 70.099998 71.699997 65.500000 67.300003 \n", "2016-02-29 00:00:00+00:00 67.300003 68.400002 58.200001 61.700001 \n", "2016-03-31 00:00:00+00:00 40.099998 39.299999 37.700001 38.200001 \n", "2016-04-30 00:00:00+00:00 54.000000 55.299999 50.799999 51.200001 \n", "2016-05-31 00:00:00+00:00 39.400002 34.299999 46.799999 45.400002 \n", "\n", "Rasterzellen-ID 428805 428806 ... 661855 \\\n", "Datum (UTC) ... \n", "2016-01-31 00:00:00+00:00 66.199997 69.800003 ... 128.199997 \n", "2016-02-29 00:00:00+00:00 65.000000 64.300003 ... 105.900002 \n", "2016-03-31 00:00:00+00:00 38.799999 37.500000 ... 65.500000 \n", "2016-04-30 00:00:00+00:00 52.799999 54.400002 ... 108.500000 \n", "2016-05-31 00:00:00+00:00 43.200001 38.500000 ... 121.299995 \n", "\n", "Rasterzellen-ID 661856 661857 662752 662753 \\\n", "Datum (UTC) \n", "2016-01-31 00:00:00+00:00 129.500000 129.600006 120.199997 121.300003 \n", "2016-02-29 00:00:00+00:00 106.199997 110.099998 96.800003 96.699997 \n", "2016-03-31 00:00:00+00:00 64.400002 65.199997 70.000000 67.500000 \n", "2016-04-30 00:00:00+00:00 112.199997 115.400002 106.700005 106.500000 \n", "2016-05-31 00:00:00+00:00 119.300003 115.099998 128.100006 138.199997 \n", "\n", "Rasterzellen-ID 662754 662755 662756 663652 \\\n", "Datum (UTC) \n", "2016-01-31 00:00:00+00:00 123.500000 125.699997 122.900002 118.099998 \n", "2016-02-29 00:00:00+00:00 100.099998 102.599998 101.400002 92.800003 \n", "2016-03-31 00:00:00+00:00 64.400002 65.099998 65.099998 69.800003 \n", "2016-04-30 00:00:00+00:00 107.699997 112.599998 114.300003 105.199997 \n", "2016-05-31 00:00:00+00:00 135.500000 124.300003 128.500000 130.199997 \n", "\n", "Rasterzellen-ID 663655 \n", "Datum (UTC) \n", "2016-01-31 00:00:00+00:00 120.400002 \n", "2016-02-29 00:00:00+00:00 97.400002 \n", "2016-03-31 00:00:00+00:00 65.500000 \n", "2016-04-30 00:00:00+00:00 108.099998 \n", "2016-05-31 00:00:00+00:00 134.399994 \n", "\n", "[5 rows x 23320 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "HDF = r\"O:\\Data\\RW_2001_2016.h5\"\n", "\n", "monthlySum = rp.hdf5_to_months(HDFFile=HDF, year_start=2016, year_end=2016)\n", "# Display the first five rows of the new DataFrame\n", "monthlySum.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. Export Results into ArcGIS Geodatabase\n", "\n", "The following function exports all rows from the DataFrame calculated above into raster datasets in an ArcGIS File Geodatabase. Optionally, different statistics rasters can be created, in this case the mean, maximum and minimum value of each cell as well as the range per cell." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "idRaster = r\"O:\\Data\\idras\"\n", "outGDBPath = r\"O:\\Data\"\n", "GDBName = \"Months_16.gdb\"\n", "statistics = [\"mean\", \"max\", \"min\", \"range\"]\n", "\n", "rp.export_dfrows_to_gdb(dataDF=monthlySum, idRaster=idRaster, outGDBPath=outGDBPath, GDBName=GDBName, statistics=statistics)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The resulting geodatabase will look like this in ArcGIS:\n", "\n", "![MonthGDB](img/MonthGDB.png)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.14" } }, "nbformat": 4, "nbformat_minor": 2 }