{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tutorial 1: Raw Data Processing with Radproc\n", "This tutorial will show you how to get started with RADOLAN processing and import your raw hourly RW data into HDF5.\n", "\n", "Note: For this approach ArcMap is required!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Import radproc" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import radproc as rp" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. Unzip Raw Data Archives\n", "\n", "The RADOLAN RW product is usually provided as monthly tar.gz archives by the German Weather Service (DWD).\n", "These have to be unzipped in order to import the contained hourly binary files.\n", "The radproc function\n", "\n", " unzip_RW_binaries(zipFolder, outFolder)\n", "\n", "can be used to unzip all archives in one directory into the directory tree format needed by the following radproc functions.\n", "Moreover, as the binary files themselves might not be zipped, all binary files are automatically compressed to .gz files to save disk space." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "RW_original = r\"O:\\Data\\RW_archives\"\n", "RW_unzipped = r\"O:\\Data\\RW_unzipped\"\n", "\n", "rp.unzip_RW_binaries(zipFolder=RW_original, outFolder=RW_unzipped)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Side Note:** To unzip the YW or RY products, which might be provided as monthly archives which contain daily archives with the actual binary files, you can use the function\n", "\n", " unzip_YW_binaries(zipFolder, outFolder)\n", " \n", "The further processing workflow is the same for all products except that you need more memory space and patience (or a smaller study area) to process the products with higher temporal resolution.\n", "\n", "\n", "**Side Note:** If you already have unpacked binary files (e.g. after download of recent RADOLAN-Online files from Climate Data Center) you can skip this step, but you need to make sure that the files are saved in monthly directories (if you have data for more than one month) to make all functions of radproc work correctly." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. Import Unzipped Data into HDF5\n", "\n", "To import all RW data you have just unzipped into an HDF5 file - optionally clipping the data to a study area - you can apply\n", "\n", " create_idraster_and_process_radolan_data(inFolder, HDFFile, clipFeature=None, complevel=9)\n", "\n", "Behind the scenes, this function will...\n", "\n", " - create an ID-raster for Germany in ArcGIS, called *idras_ger*,\n", " - if you specified a Shapefile or Feature-Class as clipFeature: Clip the german ID-raster to the extent of the clipFeature and create a second ID-raster called *idras*,\n", " - import all RADOLAN binary files in a directory tree,\n", " - select the data for your study area based on the generated ID-raster,\n", " - convert the selected data into monthly pandas DataFrames and\n", " - store all DataFrames in the specified HDF5 file.\n", " \n", "The result of this function is a HDF5 file with all RADOLAN data of your study area ready for further analysis.\n", "\n", "Note: This function works with RADOLAN-Online data as well as with the reanalyzed RADOLAN climatology data, which differ in data size and location. All necessary information are extracted from the RADOLAN metadata or are inherently contained within radproc.\n", "\n", "More detailed information on the four function parameters are available in the library reference of the function.\n", "\n", "Note: This function needs ArcMap to be installed and connected to your radproc environment!" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "O:\\Data\\RW_unzipped\\2016\\5 processed\n" ] } ], "source": [ "RW_unzipped = r\"O:\\Data\\RW_unzipped\"\n", "outHDF = r\"O:\\Data\\RW.h5\"\n", "studyArea = r\"O:\\Data\\StudyArea.shp\"\n", "\n", "rp.create_idraster_and_process_radolan_data(inFolder=RW_unzipped, HDFFile=outHDF, clipFeature=studyArea, complevel=9)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Alternative: Import Unzipped Data into HDF5 without ArcGIS\n", "\n", "In case you want to use radproc without having ArcGIS installed, you can apply\n", "\n", " process_radolan_data(inFolder, HDFFile, idArr=None, complevel=9)\n", " \n", "If you don't specify an ID Array (which is the default), all RADOLAN data will be processed without clipping. Hence, you get an HDF5 file for Germany, which you can use for analysis within Python, but keep in mind, that all export functions won't work without the ID raster and ArcMap.\n", "\n", "Note: This function works with RADOLAN-Online data as well as with the reanalyzed RADOLAN climatology data, which differ in data size and location. All necessary information are extracted from the RADOLAN metadata or are inherently contained within radproc." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "O:\\Data\\RW_unzipped\\2016\\5 processed\n" ] } ], "source": [ "RW_unzipped = r\"O:\\Data\\RW_unzipped\"\n", "outHDF = r\"O:\\Data\\RW.h5\"\n", "\n", "rp.process_radolan_data(inFolder=RW_unzipped, HDFFile=outHDF, idArr=None, complevel=9)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Now you are ready to start analyzing your data!**" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.14" } }, "nbformat": 4, "nbformat_minor": 2 }