Commit 34644e5d authored by Uwe Schulzweida's avatar Uwe Schulzweida

git pushMerge branch 'develop'

parents 6152b65b e70c36b9
......@@ -17,6 +17,7 @@ build
cdo.pdf
cdo_eca.pdf
cdo_magics.pdf
cdo_cmor.pdf
cdo_refcard.pdf
autom4te.cache
Makefile
Schulzweida, U. (2017), Climate Data Operators (CDO) User Guide, Version 1.8.0,
https://code.zmaw.de/projects/cdo/embedded/index.html
2017-01-24 Uwe Schulzweida
2017-02-14 Uwe Schulzweida
* using CDI library version 1.8.0
* Using CDI library version 1.8.0
* Version 1.8.0 release
2017-02-03 Uwe Schulzweida
* using CDI library version 1.8.0rc7
* Version 1.8.0rc7 release
2017-02-03 Uwe Schulzweida
* Reverse: adjust date/time by -1 second (introduced in last revision)
2017-01-28 Uwe Schulzweida
* Info: check floating-point exceptions
2017-01-25 Uwe Schulzweida
* New operator timrange: time range (timmax-timmin)
* New operator fldrange: field range (fldmax-fldmin)
* remap: optimzation for constant links per value
2017-01-24 Uwe Schulzweida
* write_remap_scrip: write NeCDF4 for data larger than 8GB
2017-01-19 Uwe Schulzweida
* using CDI library version 1.8.0rc6
......
......@@ -7,15 +7,23 @@ Version 1.8.0 (26 October 2016):
* NetCDF: Improved support for horizontal and vertical grids
* Changed default of option -f nc to netCDF2
* masklonlatbox: added support for unstructured grids
* setpartabn: added support for user defined attributes
* Reverse: adjust date/time by -1 second (introduced in last revision)
New operators:
* timcumsum: Cumulative sum over time.
* cmorlite: Apply variable_entry of cmor tables
* shiftx/shifty: Shift fields on rectilinear/curvilinear grids in x/y direction
Changed operators:
* Tim<STAT> (e.g. monmean):
Fixed bugs:
* Cond: bug fix for ntsteps1 == 1 && ntsteps2 != 1
* ml2pl: interpolation failed for data on hybrid half levels [Bug #7225]
Version 1.7.2 (28 June 2016):
New features:
* Adjust date/time by -1 second if the varification time is 00:00:00 and
the verification date is equal to upper time bound
New operators:
* smooth: Smooth grid points
* ap2hl: Air pressure to height level interpolation
......
......@@ -157,6 +157,8 @@ Operator catalog:
Setgatt setgatts Set global attributes
Invert invertlat Invert latitudes
Invertlev invertlev Invert levels
Shiftxy shiftx Shift x
Shiftxy shifty Shift y
Maskregion maskregion Mask regions
Maskbox masklonlatbox Mask a longitude/latitude box
Maskbox maskindexbox Mask an index box
......@@ -549,7 +551,6 @@ Operator catalog:
Strgal strgal Strong gale days index per time period
Hurr hurr Hurricane days index per time period
CMORlite cmorlite CMOR lite
CMOR cmor CMOR
-------------------------------------------------------------
Magics
-------------------------------------------------------------
......
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.68 for cdo 1.8.0rc6.
# Generated by GNU Autoconf 2.68 for cdo 1.8.0rc7.
#
# Report bugs to <http://mpimet.mpg.de/cdo>.
#
......@@ -570,8 +570,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='cdo'
PACKAGE_TARNAME='cdo'
PACKAGE_VERSION='1.8.0rc6'
PACKAGE_STRING='cdo 1.8.0rc6'
PACKAGE_VERSION='1.8.0rc7'
PACKAGE_STRING='cdo 1.8.0rc7'
PACKAGE_BUGREPORT='http://mpimet.mpg.de/cdo'
PACKAGE_URL=''
......@@ -1395,7 +1395,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures cdo 1.8.0rc6 to adapt to many kinds of systems.
\`configure' configures cdo 1.8.0rc7 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
......@@ -1465,7 +1465,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of cdo 1.8.0rc6:";;
short | recursive ) echo "Configuration of cdo 1.8.0rc7:";;
esac
cat <<\_ACEOF
......@@ -1613,7 +1613,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
cdo configure 1.8.0rc6
cdo configure 1.8.0rc7
generated by GNU Autoconf 2.68
Copyright (C) 2010 Free Software Foundation, Inc.
......@@ -2206,7 +2206,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by cdo $as_me 1.8.0rc6, which was
It was created by cdo $as_me 1.8.0rc7, which was
generated by GNU Autoconf 2.68. Invocation command line was
$ $0 $@
......@@ -3155,7 +3155,7 @@ fi
# Define the identity of the package.
PACKAGE='cdo'
VERSION='1.8.0rc6'
VERSION='1.8.0rc7'
cat >>confdefs.h <<_ACEOF
......@@ -21662,7 +21662,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by cdo $as_me 1.8.0rc6, which was
This file was extended by cdo $as_me 1.8.0rc7, which was
generated by GNU Autoconf 2.68. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
......@@ -21728,7 +21728,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
cdo config.status 1.8.0rc6
cdo config.status 1.8.0rc7
configured by $0, generated by GNU Autoconf 2.68,
with options \\"\$ac_cs_config\\"
......
......@@ -4,7 +4,7 @@
# autoconf 2.68
# libtool 2.4.2
AC_INIT([cdo], [1.8.0rc6], [http://mpimet.mpg.de/cdo])
AC_INIT([cdo], [1.8.0rc7], [http://mpimet.mpg.de/cdo])
AC_DEFINE_UNQUOTED(CDO, ["$PACKAGE_VERSION"], [CDO version])
......
......@@ -33,6 +33,7 @@ Setzaxis Modification
Setgatt Modification
Invert Modification
Invertlev Modification
Shiftxy Modification
Maskregion Modification
Maskbox Modification
Setbox Modification
......@@ -141,7 +142,6 @@ Strbre Miscellaneous
Strgal Miscellaneous
Hurr Miscellaneous
CMORlite Miscellaneous
CMOR Miscellaneous
Magplot Magics
Magvector Magics
Maggraph Magics
......
......@@ -18,7 +18,7 @@
\bibitem[CMOR]{CMOR} \ \\
\href{git://github.com/PCMDI/cmor.git}
\href{http://cmor.llnl.gov}
{Climate Model Output Rewriter},
from the
\href{https://www-pcmdi.llnl.gov}
......
......@@ -198,7 +198,7 @@ keepaspectratio]{cdo_libdep.pdf}}%
\end{picture}
\begin{flushright}
\large \textbf{Climate Data Operators \\ Version 1.8.0 \\ January 2017}
\large \textbf{Climate Data Operators \\ Version 1.8.0 \\ February 2017}
\end{flushright}
\vfill
......
......@@ -177,7 +177,7 @@
\put(0,0.1){\line(1,0){16.3}}
\end{picture}
\begin{flushright}
\large\textbf{CMORizing of climate model data \\ January 2017}
\large\textbf{CMORizing of climate model data \\ February 2017}
\end{flushright}
\vfill
......@@ -201,11 +201,17 @@
\input{cmor_int}
\input{cmor_ref}
%\input{cmor_ref}
\chapter{\label{refman}CMOR operator reference manual}
\hspace{3mm}
%\input{ref_list_cmor}
\input{ref_man_cmor_mod}
\input{bibcmor}
%\input{appendix}
\input{cmor_example}
\clearpage
\ifpdfx
......
......@@ -4,7 +4,7 @@ Some of the {\CDO} operators are parallelized with OpenMP.
To use {\CDO} with multiple OpenMP threads, you have to set the number of threads with the option '-P'.
Here is an example to distribute the bilinear interpolation on 8 OpenMP threads:
\begin{lstlisting}[frame=single, backgroundcolor=\color{pcolor2}, basicstyle=\ttfamily, columns=flexible]
cdo -P 8 remapbil,targetgrid ifile ofile
cdo -P 8 remapbil,targetgrid infile outfile
\end{lstlisting}
The following {\CDO} operators are parallelized with OpenMP:
......
......@@ -2,19 +2,19 @@
%\chapter{\label{refman}Reference manual for all operators}
This section gives a description of all operators. Related operators are grouped to modules.
For easier description all single input files are named \texttt{ifile} or \texttt{ifile1}, \texttt{ifile2}, etc.,
and an arbitrary number of input files are named \texttt{ifiles}.
All output files are named \texttt{ofile} or \texttt{ofile1}, \texttt{ofile2}, etc.
For easier description all single input files are named \texttt{infile} or \texttt{infile1}, \texttt{infile2}, etc.,
and an arbitrary number of input files are named \texttt{infiles}.
All output files are named \texttt{outfile} or \texttt{outfile1}, \texttt{outfile2}, etc.
Further the following notion is introduced:
\begin{defalist}{{\em o(t,x)}}
\item[\(i(t)\)\hfill]
Timestep \(t\) of \texttt{ifile}
Timestep \(t\) of \texttt{infile}
\item[\(i(t,x)\)\hfill]
Element number \(x\) of the field at timestep \(t\) of \texttt{ifile}
Element number \(x\) of the field at timestep \(t\) of \texttt{infile}
\item[\(o(t)\)\hfill]
Timestep \(t\) of \texttt{ofile}
Timestep \(t\) of \texttt{outfile}
\item[\(o(t,x)\)\hfill]
Element number \(x\) of the field at timestep \(t\) of \texttt{ofile}
Element number \(x\) of the field at timestep \(t\) of \texttt{outfile}
\end{defalist}
%A field is a horizontal slice of a variable on a spezific level.
......
......@@ -14,7 +14,7 @@
\put(0,0.0){\line(1,0){3.95}}
\end{picture}
\begin{flushright}
{\small{Climate Data Operators \\ Version 1.8.0 \\ January 2017}}
{\small{Climate Data Operators \\ Version 1.8.0 \\ February 2017}}
\end{flushright}
\vspace*{0mm}
......
\appendix
\chapter{Example}
In this simple example a prepared ECHAM NetCDF file with the two 3D
variables \textbf{ta} and \textbf{ua} are cmorized with the {\CDO} operator \textbf{cmor}.
Here is the NetCDF header of the input file named example.nc:
\begin{lstlisting}[frame=single, backgroundcolor=\color{pcolor1}, basicstyle=\footnotesize]
dimensions:
lon = 192 ;
bnds = 2 ;
lat = 96 ;
plev = 17 ;
time = UNLIMITED ; // (120 currently)
variables:
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:bounds = "lon_bnds" ;
float lon_bnds(lon, bnds) ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:bounds = "lat_bnds" ;
float lat_bnds(lat, bnds) ;
double plev(plev) ;
plev:standard_name = "air_pressure" ;
plev:long_name = "pressure" ;
plev:units = "Pa" ;
plev:positive = "down" ;
double time(time) ;
time:standard_name = "time" ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-1-1 00:00:00" ;
time:calendar = "proleptic_gregorian" ;
double time_bnds(time, bnds) ;
float ta(time, plev, lat, lon) ;
ta:units = "K" ;
float ua(time, plev, lat, lon) ;
ua:units = "m s-1" ;
\end{lstlisting}
The example file contains no global attributes and only the units
attribute for the variables \textbf{ta} and \textbf{ua}.
The {\CDO} \textbf{cmor} command is:
\begin{lstlisting}[frame=single, backgroundcolor=\color{pcolor1}, basicstyle=\footnotesize]
cdo cmor,CMIP6_Amon.json,__info=cmor.rc example.nc
\end{lstlisting}
The follwing two output files will be produced:
\begin{lstlisting}[frame=single, backgroundcolor=\color{pcolor1}, basicstyle=\footnotesize]
CMIP6/ta_Amon_piControl_MPI-ESM_r1i1p1f1_185001-185912.nc
CMIP6/ua_Amon_piControl_MPI-ESM_r1i1p1f1_185001-185912.nc
\end{lstlisting}
CMIP6\_Amon.json is the CMIP6 table file for monthly mean atmosphere
data from PCMDI. The option \texttt{\_\_info=cmor.rc} will read settting from
the config file cmor.rc.
Here is the contents of this config file:
\begin{lstlisting}[frame=single, backgroundcolor=\color{pcolor1}, basicstyle=\footnotesize]
__inpath = "/<XXX>/CMOR/cmip6-cmor-tables/Tables"
__grid_table = "CMIP6_grids.json"
_control_vocabulary_file = "CMIP6_CV.json"
_cmip6_option = "CMIP6"
activity_id = "CMIP"
outpath = "CMIP6"
experiment_id = "piControl"
realization_index = 1
initialization_index = 1
physics_index = 1
forcing_index = 1
source_type = "AOGCM"
sub_experiment = "none"
sub_experiment_id = "none"
mip_era = "CMIP6"
contact = "cmip5-mpi-esm@dkrz.de"
institution_id = "MPI-M"
model_id = "MPI-ESM-MR"
references = "ECHAM6: n/a; JSBACH: n/a; MPIOM: n/a; HAMOCC: n/a;"
branch_time = 0
branch_method = "standard"
grid = "native atmosphere T63 gaussian grid (64x128 latxlon)"
grid_label = "gn"
grid_resolution = "5 km"
institution = "Max Planck Institute for Meteorology"
run_variant = "MPI-ESM-MR model output prepared for CMIP5 pre-industrial control"
source_id = "MPI-ESM"
source = "MPI-ESM:"
output_file_template = "<variable_id><table><experiment_id><source_id><variant_label>"
license = "N/A"
# Rename axis
__rename_plev = plev17
\end{lstlisting}
NetCDF header of the output file ta\_Amon\_piControl\_MPI-ESM\_r1i1p1f1\_185001-185912.nc:
\begin{lstlisting}[frame=single, backgroundcolor=\color{pcolor1}, basicstyle=\footnotesize]
dimensions:
time = UNLIMITED ; // (120 currently)
plev = 17 ;
lat = 96 ;
lon = 192 ;
bnds = 2 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-1-1 00:00:00" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double time_bnds(time, bnds) ;
double plev(plev) ;
plev:units = "Pa" ;
plev:axis = "Z" ;
plev:positive = "down" ;
plev:long_name = "pressure" ;
plev:standard_name = "air_pressure" ;
double lat(lat) ;
lat:bounds = "lat_bnds" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:bounds = "lon_bnds" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
double lon_bnds(lon, bnds) ;
float ta(time, plev, lat, lon) ;
ta:standard_name = "air_temperature" ;
ta:long_name = "Air Temperature" ;
ta:comment = "Air Temperature" ;
ta:units = "K" ;
ta:history = "2017-01-31T11:23:09Z altered by CMOR: replaced missing value flag (-9e+33) with standard missing value (1e+20)." ;
ta:missing_values = 1.e+20f ;
ta:_FillValue = 1.e+20f ;
// global attributes:
:Conventions = "CF-1.6 CMIP-6.0" ;
:activity_id = "CMIP" ;
:additional_source_type = "CHEM AER" ;
:branch_method = "standard" ;
:branch_time = 0. ;
:contact = "cmip5-mpi-esm@dkrz.de" ;
:creation_date = "2017-01-31T11:23:09Z" ;
:data_specs_version = "01.beta.36" ;
:experiment = "pre-Industrial control" ;
:experiment_id = "piControl" ;
:forcing_index = 1 ;
:frequency = "mon" ;
:further_info_url = "http://furtherinfo.es-doc.org/CMIP6/MPI-M.MPI-ESM.piControl.none.r1i1p1f1" ;
:grid = "native atmosphere T63 gaussian grid (64x128 latxlon)" ;
:grid_label = "gn" ;
:grid_resolution = "5 km" ;
:history = "2017-01-31T11:23:09Z CMOR rewrote data to be consistent with CF standards and CMIP6 requirements." ;
:initialization_index = 1 ;
:institution = "Max Planck Institute for Meteorology" ;
:institution_id = "MPI-M" ;
:mip_era = "CMIP6" ;
:model_id = "MPI-ESM-MR" ;
:physics_index = 1 ;
:product = "output" ;
:realization_index = 1 ;
:realm = "atmos atmosChem" ;
:references = "ECHAM6: n/a; JSBACH: n/a; MPIOM: n/a; HAMOCC: n/a;" ;
:run_variant = "MPI-ESM-MR model output prepared for CMIP5 pre-industrial control" ;
:source = "MPI-ESM:" ;
:source_id = "MPI-ESM" ;
:source_type = "AOGCM" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "Amon" ;
:table_info = "Creation Date:(11 October 2016) MD5:77b461da88bd079bd1932066c0c7ad4a" ;
:title = "MPI-ESM model output prepared for CMIP6" ;
:variable_id = "ta" ;
:variant_label = "r1i1p1f1" ;
:license = "N/A" ;
:cmor_version = "3.1.0" ;
:tracking_id = "57138bf0-d93b-495e-82d0-cf287a0849c1" ;
\end{lstlisting}
......@@ -3,11 +3,11 @@
The Climate Data Operators ({\CDO}) software is a collection of operators
for standard processing of climate and forecast model data.
This document describes the additional {\CDO} operator cmor. This
This document describes the additional {\CDO} operator \textbf{cmor}. This
operator is an interface to the CMOR library version 3 from PCMDI.
The CMOR support is available with {\CDO} release 1.8.0 or later.
The CMOR (Climate Model Output Rewriter) library comprises a set of
The \cite{CMOR} (Climate Model Output Rewriter) library comprises a set of
functions, that can be used to produce CF-compliant NetCDF files that
fulfill the requirements of many of the climate community's standard
model experiments. These experiments are collectively referred to as
......@@ -21,21 +21,22 @@ site. CMOR relies on these tables to provide much of the metadata
that is needed in the MIP context, thereby reducing the programming
effort required of the individual MIP contributors.
The CDO operator cmor reads the data with the internal IO library CDI
and writes the NetCDF result directly with the CMOR library.
The {\CDO} operator \textbf{cmor} was developed at the DKRZ and MPI for
Meteorology to provide an easy interface to the \cite{CMOR} library for a
standardized preparation of CMIP6 data.
\chapter{Building CDO with CMOR}
This section describes how to build and install {\CDO} with CMOR
support on a UNIX system.
\section{CMOR}
\section{CMOR library}
CMOR version 3 needs to be installed before building {\CDO}.
CMOR depends on the following external libraries:
netCDF-4.0 , HDF5, UDUNITS-2, zlib and uuid.
The CMOR library depends on the following external libraries:
netCDF4, HDF5, UDUNITS2, zlib and uuid.
Make sure that you use exactly the same libraries for the {\CDO}
installation, otherwise the operator cmor will possibly not working correctly.
installation, otherwise the operator \textbf{cmor} will possibly not working correctly.
\section{Compilation}
......