mpim-sw issueshttps://gitlab.dkrz.de/groups/mpim-sw/-/issues2023-10-12T09:28:22Zhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/15Missing support for PDTs 67 and 682023-10-12T09:28:22ZDaniel ReinertMissing support for PDTs 67 and 68Dear Uwe @m214003 , Dear Sergey @m300488,
The product definition templates 67 and 68 for statistically processed chemical constituents are currently not supported by `libCDI`.
This is fixed by the following commit: https://gitlab.dkrz...Dear Uwe @m214003 , Dear Sergey @m300488,
The product definition templates 67 and 68 for statistically processed chemical constituents are currently not supported by `libCDI`.
This is fixed by the following commit: https://gitlab.dkrz.de/mpim-sw/libcdi/-/commit/706f34f075e0a7bda79220335ff4a7088157af92
For our operational code at DWD, I have created the hotfix branch
https://gitlab.dkrz.de/mpim-sw/libcdi/-/tree/cdi-2.2.4-release-hotfix?ref_type=heads
which is based on the release `cdi-2.2.4-release`.
Can you please include this fix into the CDI-master and next release?
Thank you very much,
Daniel
@m300778https://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/14CDI: unexpected reference and forecast date for time-constant fields (steptyp...2023-09-12T16:34:41ZDaniel ReinertCDI: unexpected reference and forecast date for time-constant fields (steptype=TSTEP_CONSTANT)Dear Uwe @m214003 , Dear Sergey @m300488,
we are currently preparing a new ICON binary for our operational weather forecasts at DWD.
During our tests, we stumbled across an unexpected CDI behavior which, unfortunately, prevents us fro...Dear Uwe @m214003 , Dear Sergey @m300488,
we are currently preparing a new ICON binary for our operational weather forecasts at DWD.
During our tests, we stumbled across an unexpected CDI behavior which, unfortunately, prevents us from using the latest ICON version (RC).
The problem we observed is as follows:\
When writing an output file which exclusively contains time-constant fields (i.e. `steptype=TSTEP_CONSTANT`), the reference and forecast date is unconditionally set to the dummy date `10101`. Below is the output of `cdo sinfov` for a forecast run starting at 2017-04-15T00:00:00Z:
```
File format : GRIB2
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter name
1 : DWD unknown v instant 91 1 327680 1 P16 : HHL
2 : DWD unknown v instant 1 2 327680 1 P16 : HSURF
3 : DWD unknown v instant 1 3 327680 1 P16 : FR_LAND
4 : DWD unknown v instant 1 3 327680 1 P16 : CLON
5 : DWD unknown v instant 1 3 327680 1 P16 : CLAT
6 : DWD unknown v instant 1 3 491520 2 P16 : ELON
7 : DWD unknown v instant 1 3 491520 2 P16 : ELAT
Grid coordinates :
1 : unstructured : points=327680
grid : number=24 position=1
uuid : 9b0b03ca-18c4-11e4-9318-776f158edd08
2 : unstructured : points=491520
grid : number=24 position=3
uuid : 9b0b03ca-18c4-11e4-9318-776f158edd08
Vertical coordinates :
1 : generalized_height : levels=91
height : 1 to 91 by 1
bounds : 1-0 to 91-0 by 1
uuid : b3d63ca1-092c-5f0e-b9f4-4c9b2a052be0
typeOfSecondFixedSurface : 101
2 : surface : levels=1
typeOfSecondFixedSurface : 101
3 : surface : levels=1
Time coordinate : 1 step
RefTime = 0001-01-01 00:00:00 Units = minutes Calendar = proleptic_gregorian
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
0001-01-01 00:00:00
```
If I add a single time-dependent variable (e.g. the sea ice fraction), then the reference and forecast date are set 'properly' in the output file. i.e. in accordance with our explicitly chosen reference date which we specify in `mo_name_list_output_init:setup_output_vlist` of the ICON code.
```
idate = cdiEncodeDate(INT(time_config%tc_exp_startdate%date%year), &
& INT(time_config%tc_exp_startdate%date%month), &
& INT(time_config%tc_exp_startdate%date%day))
itime = cdiEncodeTime(time_config%tc_exp_startdate%time%hour, time_config%tc_exp_startdate%time%minute, &
INT(time_config%tc_exp_startdate%time%second))
CALL taxisDefRdate (taxisID, idate)
CALL taxisDefRtime (taxisID, itime)
```
**This behavior is different from the behaviour of CDI 1.8.x:**
With the previous CDI version we were able to switch between from the 'normal' date to the dummy date, by setting
```
vlistDefVarTypeOfGeneratingProcess(vlistID, varID, 196)
```
see also the CDI code
```
gribapiDefTime(int editionNumber, int productDefinitionTemplate, int typeOfGeneratingProcess, grib_handle *gh,
CdiDateTime vDateTime, int tsteptype, int numavg, int taxisID, int gcinit)
{
UNUSED(numavg);
int taxistype = (taxisID == -1) ? -1 : taxisInqType(taxisID);
if (typeOfGeneratingProcess == 196)
{
vDateTime = cdiDateTime_set(10101, 0);
taxistype = TAXIS_ABSOLUTE;
}
```
Or in other words, it was possible to write files which have the 'normal' reference and forecast date, even though the files contain only time-constant fields.
This feature is rather important for use, as we use the reference and forecast date for writing/retrieving these time-constant fields to/from our database.
I was able to enforce the old behavior by changing
```
varID = vlistDefVar(vlistID, gridID, zaxisID, MERGE(TIME_CONSTANT, TIME_VARYING, info%isteptype == TSTEP_CONSTANT))
CALL vlistDefVarTsteptype(vlistID, varID, info%isteptype)
```
to
```
varID = vlistDefVar(vlistID, gridID, zaxisID, TIME_VARYING)
CALL vlistDefVarTsteptype(vlistID, varID, info%isteptype)
```
in `mo_util_cdi:create_cdi_variable`, but I am not sure if this is a proper solution.
Would it be possible to revert the responsible code changes or do you have any suggestion how we can enforce the desired behavior?
Best wishes,
Daniel
@b381078 @b381074 @m222072 @b380996https://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/12CDI GRIB2 warnings in gribapiEncode2023-09-12T16:34:41ZDaniel ReinertCDI GRIB2 warnings in gribapiEncodeDear Uwe @m214003 , Dear Sergey @m300488
with the upgrade to CDI 2.2.x some additional GRIB2 checks have been implemented in `cdilib.c/gribapiEncode`, which check if the GRIB2 shortName matches the (ICON) internal variable name. If not...Dear Uwe @m214003 , Dear Sergey @m300488
with the upgrade to CDI 2.2.x some additional GRIB2 checks have been implemented in `cdilib.c/gribapiEncode`, which check if the GRIB2 shortName matches the (ICON) internal variable name. If not, a warning is printed at every output date of the form
```
cdi warning (gribapiEncode): *** GRIB2 shortName does not correspond to chosen variable name: "p" ("pres").
cdi warning (gribapiEncode): *** GRIB2 shortName does not correspond to chosen variable name: "asob_s_cs" ("asobclr_s").
cdi warning (gribapiEncode): *** GRIB2 shortName does not correspond to chosen variable name: "aswdifd_s" ("asodifd_s").
...
```
From our perspective it is good to have such a check, but we would prefer to have it optional. It increases the size of the ICON LOG file quite significantly and makes it look even more messy than before ;-)
Would it be possible to put this check into an `if (CDI_Debug) ...` condition?
The fact that the GRIB2 shortName differs from the internal variable name is not unusual in my opinion, as GRIB2 shortNames are site specific. Even if we would modify all the internal variable names to match e.g. the DWD-specific GRIB2 shortNames, the warning would pop up again, as soon as any partner institution runs the model using their own set of shortNames.
@m222072 @m300196
Than you,
Danielhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/11CDI GRIB2 support: Additional ProductDefinitionTemplates for distribution fun...2023-07-03T11:37:11ZDaniel ReinertCDI GRIB2 support: Additional ProductDefinitionTemplates for distribution functionsDear Uwe @m214003 , Dear Sergey @m300488
Jochen Foerstner @m300778 kindly asks for a small extension of the CDI routine `getAvailabilityOfRelativeTimes` (`src/gribapi_utilities.c`), in order to support some additional GRIB2 ProductDefi...Dear Uwe @m214003 , Dear Sergey @m300488
Jochen Foerstner @m300778 kindly asks for a small extension of the CDI routine `getAvailabilityOfRelativeTimes` (`src/gribapi_utilities.c`), in order to support some additional GRIB2 ProductDefinitionTemplates for ICON-ART. I have added the corresponding patch based on cdi-1.8.4 which is currently used by ICON [cdi_pdt57.patch](/uploads/b5891d7fb7102fbc7a8d6fcd67efd447/cdi_pdt57.patch)
What would be the best way to go? Do you think it's possible to create an updated CDI version based on cdi-1.8.4 and push it to the ICON-RC?
Thanks a lot.
Best,
Danielhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/10How to install libcdi (on a Mac)?2023-04-04T08:26:17ZMarco GiorgettaHow to install libcdi (on a Mac)?Today I wanted to install `libcdi` on my Mac. The reason is that the ParaView developers no longer include `libcdi` in the "dmg" package file for ParaView 5.11.0 on Macs. Instead they expect that this is installed as an external:
```
htt...Today I wanted to install `libcdi` on my Mac. The reason is that the ParaView developers no longer include `libcdi` in the "dmg" package file for ParaView 5.11.0 on Macs. Instead they expect that this is installed as an external:
```
https://www.kitware.com/paraview-5-11-0-release-notes/ "Plugin updates":
CDI Reader plugin updates
The CDI reader plugin now uses CDI version 2.0. Also, CDI is now an external dependency so building this plugin requires obtaining libcdi (see https://gitlab.dkrz.de/mpim-sw/libcdi).
```
So I cloned libcdi (`git clone --recursive https://gitlab.dkrz.de/mpim-sw/libcdi`) and read the README, and tried to follow the recipe. But:
* the `configure` script is missing, see **(1)**
* the documentation files (./doc/cdi_cman.pdf, ./doc/cdi_fman.pdf) are missing as well
So I wonder, how can I install `libcdi`?
Best regards,
Marco
P.S. The most convenient way would be to get it through MacPorts, but unfortunately `libcdi`is not available in MacPorts.
----
**(1)** I noticed that the `case "${HOSTNAME}"`list in `config/default` needs to be modified. After having done this I tried to run `./config/default`, which then tried to execute `configure`, but did not find it:
```
marcogiorgetta@d146-149 libcdi % pwd
/Users/marcogiorgetta/gitlab.dkrz.de/libcdi
marcogiorgetta@d146-149 libcdi % ./config/default
++ hostname
+ HOSTNAME=d146-149.mpimet.mpg.de
+ test 0 '!=' 0
+ case "${HOSTNAME}" in
+ ./configure --prefix=/Users/marcogiorgetta/local/cdi --enable-maintainer-mode --enable-iso-c-interface --enable-swig --with-eccodes=/opt/local --with-netcdf=/opt/local --with-szlib=/opt/local/lib/libaec CC=gcc 'CFLAGS=-g -pipe -D_REENTRANT -Wall -Wwrite-strings -W -Wfloat-equal -pedantic -O3'
./config/default: line 45: ./configure: No such file or directory
```https://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/8Switch off unused bitmap2022-03-10T13:08:32ZSebastian BorchertSwitch off unused bitmap
Hello Uwe,
I would like to ask you if you see the possibility to implement the following functionality for GRIB files into CDI?
**A check for an unnecessary bitmap:** I.e. a bitmap has been switched on for a GRIB record, but it is in...
Hello Uwe,
I would like to ask you if you see the possibility to implement the following functionality for GRIB files into CDI?
**A check for an unnecessary bitmap:** I.e. a bitmap has been switched on for a GRIB record, but it is in fact
not required, since the number of coded values turned out to be equal to the number of grid points.
**Possible solution:** right before writing the record, something of the sort (in ecCodes meta language):
```
if (bitmapPresent) {
if (numberOfCodedValues == numberOfDataPoints) { # OR: if (numberOfMissing == 0) {
set bitmapPresent = 0 ;
}
}
```
is applied to the metadata.
**Example of use:** In our operational use of ICON, there is GRIB output on a native triangular limited-area grid.
The boundary zone of the grid is masked by a bitmap.
In addition, there is output interpolated on a regular lat-lon grid. It inherits metadata from the native grid,
including the bitmap being switched on. In fact, no grid point of the lat-lon grid lies in (or beyond) the boundary zone,
so no bitmap would be necessary in most cases.
It would be very helpful if these unnecessary bitmaps could be switched off.
First, because of postprocessing becoming more expensive if a(n unnecessary) bitmap has to be evaluated.
Second, because deleting an unnecessary bitmap reduces the file size by about 100%/(bitsPerValue + 1).
There are probably many more use cases that could profit from deleting unused bitmaps.Uwe SchulzweidaUwe Schulzweidahttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/6[CI]make builders more flexible from the developer perspective2022-09-29T06:32:44ZRalf Mueller[CI]make builders more flexible from the developer perspectivedevelopers should have the opportunity to change the testing procedure for each builder from https://bb-cdi.dkrz.de
currently it's all set in the builder wrapper below `.ci/bb`developers should have the opportunity to change the testing procedure for each builder from https://bb-cdi.dkrz.de
currently it's all set in the builder wrapper below `.ci/bb`Ralf MuellerRalf Muellerhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/5[CI] provide buildbot client at CSCS2022-09-27T16:20:29ZRalf Mueller[CI] provide buildbot client at CSCSRalf MuellerRalf Muellerhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/4CDI consolidation2022-12-02T16:15:19ZSergey KosukhinCDI consolidation**Goal**
Consolidate all CDI developments into the `develop` branch of this repository so that there is a common codebase of the library that has all features that are currently implemented and maintained in different repositories and b...**Goal**
Consolidate all CDI developments into the `develop` branch of this repository so that there is a common codebase of the library that has all features that are currently implemented and maintained in different repositories and branches.
Apart from the obvious maintenance benefits (e.g. no need to back merge certain features to different versions) in the long run, the immediate (and quite urgent) goal is to have a version of the library that provide the following features:
1. FDB5 support needed for the ACROSS project (available in [`develop@libcdi`](https://gitlab.dkrz.de/mpim-sw/libcdi/-/tree/develop) starting version `2.0.0`).
2. Certain GRIB2 compression features requested by DWD (available in [`develop@libcdi`](https://gitlab.dkrz.de/mpim-sw/libcdi/-/tree/develop) starting version `1.9.2`).
3. Parallel I/O (available in [`master@cdi-pio`](https://gitlab.dkrz.de/dkrz-sw/cdi-pio/-/tree/master), which is based on version `1.8.2`).
4. Compatibility with the ICON build system (fully available only in [`cdi-1.8.x@libcdi`](https://gitlab.dkrz.de/mpim-sw/libcdi/-/commits/cdi-1.8.x), which is based on version `1.8.2`).
**Tasks**
1. Merge `master@cdi-pio` into `develop@libcdi` (not necessarily directly, see the steps below). Some time ago, Thomas (@jahns) reported on problems that hinder the merge (see [2021-06-11_CDI-PIO_merge_activities.pdf](/uploads/ba3a589b1087eebc24fee6dda0030166/2021-06-11_CDI-PIO_merge_activities.pdf)). However, according to Uwe (@m214003), most of them (if not all) are resolved, starting version `2.0.0`. We need confirmation from Thomas on this.
2. Make sure that all commits in `cdi-1.8.x@libcdi` starting https://gitlab.dkrz.de/mpim-sw/libcdi/-/commit/7ce36fce909d5825bb0586dc64985e1e01ab1aaf either have already found their way into `develop@libcdi` or are irrelevant.
3. Update the build system of the library to make it compatible with the ICON build system. For example, the configure script should allow for better protection against invalid values of certain environment variables and do not assume that, for example, YAXT is already built and installed because that is not the case when configuring and building the bundled libraries of ICON.
4. Set up a test infrastructure for `develop@libcdi`, make sure that all features and configurations of interest are covered with respective tests and that the `HEAD` of `develop@libcdi` passes them at all times. Ralf (@k202125) has already made the first steps in that direction (see https://bb-cdi.dkrz.de and https://gitlab.dkrz.de/mpim-sw/libcdi/-/pipelines) but they are not applicable to `develop@libcdi` yet.
**Steps**
We basically need to consolidate three branches — [`develop@libcdi`](https://gitlab.dkrz.de/mpim-sw/libcdi/-/tree/develop), [`master@cdi-pio`](https://gitlab.dkrz.de/dkrz-sw/cdi-pio/-/tree/master) and [`cdi-1.8.x@libcdi`](https://gitlab.dkrz.de/mpim-sw/libcdi/-/commits/cdi-1.8.x) — plus introduce certain changes to the build system. The tentative plan, which we have come up with in the meeting between MPI-M and DKRZ on the 25th of October, was to merge `master@cdi-pio` into `develop@libcdi` as the first step. However, in the follow-up meeting between me and Thomas on the 27th of October, we agreed to base our work on https://gitlab.dkrz.de/mpim-sw/libcdi/-/merge_requests/8, which consolidates `cdi-1.8.x@libcdi` and `master@cdi-pio` plus almost fully covers tasks 3 and 4 (see above). Therefore, the plan is:
- [x] 1. Thomas reviews https://gitlab.dkrz.de/mpim-sw/libcdi/-/merge_requests/8.
- [x] 2. I introduce changes to the merge request that are required to get Thomas's approval and do the merge into `cdi-1.8.x@libcdi`.
- [x] 2.1 Thomas review/tests !11 - approval required in case of positive result
- [x] 3. Ralf makes sure that the test pipelines of CDI work correctly for `cdi-1.8.x@libcdi`.
- [x] 4. I finalize the integration of `cdi-1.8.x@libcdi` into ICON (see https://gitlab.dkrz.de/icon/icon-cimd/-/merge_requests/32).
- [ ] 5. review/test needed for !13 ~~DKRZ merges `cdi-1.8.x@libcdi` into `develop@libcdi`. This step is currently the heaviest and the most uncertain one and will probably be split into several substeps.~~
- [x] 6. Ralf makes sure that the test pipelines of CDI work correctly for `develop@libcdi` (and they must never go "red" ever after).
- [ ] 7. I integrate `develop@libcdi` into ICON from the build system perspective (to account for the new configure options and macros that currently exist only in `develop@libcdi`).
- [ ] ~~8. Luis (@m214089) integrates `develop@libcdi` into ICON from the API perspective.~~
- [x] 9. Ralf updates and extends the BB of ICON to cover the parallel I/O functionality better (results-comparison with reference data)
**Timeline**
MPI-M are very interested in having this done by the end of this year and are ready to minimize any delays on their side. DKRZ are kindly asked to give this work higher priority and share their plans on when they can get item 1 on the list above done and start working on item 5.
Other people who might be interested in this but have not been mentioned yet: @k202061 @m218027 @m300910 @m300913 @m300196 @m300173 @b380563https://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/3Erroneous behaviour setting GRIB2 bitsPerValue2021-10-05T09:16:09ZFlorian PrillErroneous behaviour setting GRIB2 bitsPerValueHi,
Recently, I've encountered a problem regarding the creation of GRIB2 files and, in particular, the setting of the `bitsPerValue` information. One might argue if the following problem is caused by an erroneous behaviour of the CDI or...Hi,
Recently, I've encountered a problem regarding the creation of GRIB2 files and, in particular, the setting of the `bitsPerValue` information. One might argue if the following problem is caused by an erroneous behaviour of the CDI or the underlying ECCodes library:
To reproduce the problem, I've created a branch `ex_bitsPerValue` where I modified the example program `examples/cdi_write.c` accordingly (commit 2f5f90b9). Here, the CDI library creates a multi-level variable "QC" and sets
```
vlistDefVarDatatype(vlistID, varID2, 16);
```
We can see that this information is forwarded to the ECCodes library:
<details>
```
CDI_DEBUG=1 CDI_GRIBAPI_DEBUG=1 ./cdi_write
cdiGetenvInt : set CDI_GRIBAPI_DEBUG to 1
vlistCreate : create vlistID = 26
vlistDefVarTiles : gridID = 24 zaxisID = 25 timetype = 1
taxisCreate : taxistype: 1
taxisCreate : taxisID: 27
streamOpenID : Open GRIB2 mode w file example.grb
vlistDuplicate : call to vlistDuplicate
vlistCreate : create vlistID = 29
vlistCopy : call to vlistCopy, vlistIDs 26 -> 29
stream_new_var : gridID = 24 zaxisID = 25
stream_new_var : varID 0: create 1 tiles with 5 level(s), zaxisID=25
stream_new_var : streamptr->vars[varID].recordTable[isub].recordID[0]=-1
grib_set_long( grib_handle* h, "grib2LocalSectionPresent", 0)
grib_set_long( grib_handle* h, "numberOfValues", 0)
cdiStreamDefTimestep_: streamID = 28 tsID = 0
cdiStreamWriteVar_: streamID = 28 varID = 0
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "typeOfGeneratingProcess", 0)
grib_set_long( grib_handle* h, "backgroundProcess", 0)
grib_set_long( grib_handle* h, "productDefinitionTemplateNumber", 0)
grib_set_string( grib_handle* h, "stepType", "instant")
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850101)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_string( grib_handle* h, "shortName", "QC")
grib_set_long( grib_handle* h, "shapeOfTheEarth", 0)
grib_set_long( grib_handle* h, "bitsPerValue", 16)
grib_set_string( grib_handle* h, "packingType", "grid_simple")
grib_set_long( grib_handle* h, "numberOfValues", 72)
grib_set_string( grib_handle* h, "gridType", "regular_ll")
grib_set_long( grib_handle* h, "Ni", 12)
grib_set_double( grib_handle* h, "longitudeOfFirstGridPointInDegrees", 0.000000)
grib_set_double( grib_handle* h, "longitudeOfLastGridPointInDegrees", 330.000000)
grib_set_double( grib_handle* h, "iDirectionIncrementInDegrees", 30.000000)
grib_set_long( grib_handle* h, "Nj", 6)
grib_set_double( grib_handle* h, "latitudeOfFirstGridPointInDegrees", -75.000000)
grib_set_double( grib_handle* h, "latitudeOfLastGridPointInDegrees", 75.000000)
grib_set_long( grib_handle* h, "iScansNegatively", 0)
grib_set_long( grib_handle* h, "jScansPositively", 1)
grib_set_double( grib_handle* h, "jDirectionIncrementInDegrees", 30.000000)
grib_set_long( grib_handle* h, "typeOfFirstFixedSurface", 100)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 101300)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850101)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 92500)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850101)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 85000)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850101)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 50000)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850101)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 20000)
cdiStreamDefTimestep_: streamID = 28 tsID = 1
cdiStreamWriteVar_: streamID = 28 varID = 0
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850102)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 101300)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850102)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 92500)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850102)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 85000)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850102)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 50000)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850102)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 20000)
cdiStreamDefTimestep_: streamID = 28 tsID = 2
cdiStreamWriteVar_: streamID = 28 varID = 0
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850103)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 101300)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850103)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 92500)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850103)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 85000)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850103)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 50000)
grb_write_var_slice: gridID = 24 zaxisID = 25
grib_set_long( grib_handle* h, "significanceOfReferenceTime", 0)
grib_set_long( grib_handle* h, "stepRange", 0)
grib_set_long( grib_handle* h, "dataDate", 19850103)
grib_set_long( grib_handle* h, "hour", 12)
grib_set_long( grib_handle* h, "minute", 0)
grib_set_long( grib_handle* h, "second", 0)
grib_set_long( grib_handle* h, "scaleFactorOfFirstFixedSurface", 0)
grib_set_long( grib_handle* h, "scaledValueOfFirstFixedSurface", 20000)
streamClose : streamID = 28 filename = example.grb
vlist_delete : call to vlist_delete, vlistID = 29
stream_delete_entry: Removed idx 28 from stream list
vlist_delete : call to vlist_delete, vlistID = 26
```
</details>
However, what's special about my setup is the fact that the first levels of my example are _constant_.
Therefore we see in the example output `bitsPerValue=0` - I interpret this as "all values are equal, therefore GRIB2 encodes this in a trivial way".
But when we write out a mixture of constant and non-constant records, we get a problem: ECCodes obviously recognizes the first levels as constant (`bitsPerValue=0`) and does not apply the original CDI setting `bitsPerValue=16`. But this information is not restored when the non-constant QC levels are written: We get `bitsPerValue=24` for these levels instead of 24 bits.
More precisely: If we enable the code section
```
// Init var2
for ( size_t i = 0; i < nlon*nlat*2; i++ ) var2[i] = 0.;
for ( size_t i = nlon*nlat*2; i < nlon*nlat*nlev; i++ ) var2[i] = sin((double)i);
```
i.e. a mixture of constant and non-constant levels, we get
```
$ /sw/rhel6-x64/eccodes/eccodes-2.5.0-intel14/bin/grib_ls -P bitsPerValue example.grb
example.grb
bitsPerValue edition centre date dataType gridType stepRange typeOfLevel level shortName packingType
0 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 1013 qc grid_simple
0 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 925 qc grid_simple
24 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 850 qc grid_simple
24 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 500 qc grid_simple
24 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 200 qc grid_simple
0 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 1013 qc grid_simple
0 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 925 qc grid_simple
24 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 850 qc grid_simple
24 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 500 qc grid_simple
24 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 200 qc grid_simple
0 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 1013 qc grid_simple
0 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 925 qc grid_simple
24 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 850 qc grid_simple
24 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 500 qc grid_simple
24 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 200 qc grid_simple
```
On the other hand, if we have all levels non-constant:
```
// Init var2
for ( size_t i = 0; i < nlon*nlat*nlev; i++ ) var2[i] = sin((double)i);
```
we get
```
$ /sw/rhel6-x64/eccodes/eccodes-2.5.0-intel14/bin/grib_ls -P bitsPerValue example.grb
example.grb
bitsPerValue edition centre date dataType gridType stepRange typeOfLevel level shortName packingType
16 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 1013 qc grid_simple
16 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 925 qc grid_simple
16 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 850 qc grid_simple
16 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 500 qc grid_simple
16 2 ecmf 19850101 af regular_ll 0 isobaricInhPa 200 qc grid_simple
16 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 1013 qc grid_simple
16 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 925 qc grid_simple
16 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 850 qc grid_simple
16 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 500 qc grid_simple
16 2 ecmf 19850102 af regular_ll 0 isobaricInhPa 200 qc grid_simple
16 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 1013 qc grid_simple
16 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 925 qc grid_simple
16 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 850 qc grid_simple
16 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 500 qc grid_simple
16 2 ecmf 19850103 af regular_ll 0 isobaricInhPa 200 qc grid_simple
```https://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/2cdi-1.9.8 does not create mo_cdi.mod file?2020-10-22T12:49:46ZFlorian Prillcdi-1.9.8 does not create mo_cdi.mod file?Hello Sergey, hello Uwe,
While Sergey's branch "cdi-1.8.x" creates the src/mo_cdi.mod file the more recent tags don't? Which fix is required for this?
Best regards,
Florian
@m214003
@m300488Hello Sergey, hello Uwe,
While Sergey's branch "cdi-1.8.x" creates the src/mo_cdi.mod file the more recent tags don't? Which fix is required for this?
Best regards,
Florian
@m214003
@m300488Sergey KosukhinSergey Kosukhinhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/1RUBY problems ...2020-03-09T11:43:23ZFlorian PrillRUBY problems ...@m214003 , @m300488
Hello Uwe and Sergey,
In ICON's new (and nice) build system I found that $(RUBY) is still contained in a Makefile target although we have configured with "ENABLE_RUBY='false".
See the line
https://gitlab.dkrz.de/m...@m214003 , @m300488
Hello Uwe and Sergey,
In ICON's new (and nice) build system I found that $(RUBY) is still contained in a Makefile target although we have configured with "ENABLE_RUBY='false".
See the line
https://gitlab.dkrz.de/mpim-sw/libcdi/blob/cdi-1.8.x/src/Makefile.am#L270
Shouldn't we guard this rule with
```
if ENABLE_RUBY
...
endif
```
What do you think?
Best regards,
FlorianSergey KosukhinSergey Kosukhin