mpim-sw issueshttps://gitlab.dkrz.de/groups/mpim-sw/-/issues2022-01-10T19:42:39Zhttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/7Fehler im Append-Mode?2022-01-10T19:42:39ZFlorian PrillFehler im Append-Mode?Hallo Uwe (nebenbei: Ein Frohes Neues)!
Auf unserem System sind wir (@b381035) in einem ICON-Testfall auf ein Problem gestoßen, das mit dem Aufruf von `streamClose` und einem anschließendem, explizitem Aufräumen von `vlistDestroy` [4] z...Hallo Uwe (nebenbei: Ein Frohes Neues)!
Auf unserem System sind wir (@b381035) in einem ICON-Testfall auf ein Problem gestoßen, das mit dem Aufruf von `streamClose` und einem anschließendem, explizitem Aufräumen von `vlistDestroy` [4] zusammenzuhängen scheint.
Normalerweise dürfte die Reihenfolge, bestehend aus `CALL streamClose(streamID)` [5], und erst danach `CALL vlistDestroy(vlistID)` unproblematisch sein, siehe das Beispielprogramm [1].
In unserem Fall bricht aber `taxisDestroy` (das in `vlistDestroy` enthalten ist) mit einer Fehlermeldung in `reshGetElem` ab, da es das Element in der "Resouce Handle"-Liste nicht mehr gibt.
Eine Besonderheit in unserem Testfall ist jedoch
```fortran
streamptr->filemode != 'a'
```
d.h. die Datei wurde im "Append-Mode" geöffnet.
Dann könnte diese Stelle hier die Ursache unseres Absturzes sein [2]:
```fortran
if (streamptr->filemode != 'w' && vlistInqTaxis(vlistID) != -1) taxisDestroy(vlistInqTaxis(vlistID));
```
Uns erscheint es sinnvoll, dies für den Append-Mode zu erweitern:
```fortran
if (streamptr->filemode != 'w' && streamptr->filemode != 'a' && vlistInqTaxis(vlistID) != -1)
taxisDestroy(vlistInqTaxis(vlistID));
```
Allerdings scheint das noch nicht die vollständige Lösung zu sein, da auch `cdiVlistDestroy_` [3] übersprungen werden muss, damit nicht der anschließende, manuelle Aufruf von`vlistDestroy` mit einer ähnlichen Fehlermeldung wie das Taxis-Objekt abbricht. Meine Vermutung ist die, dass auch hier der Append-Mode nicht berücksichtigt wurde?
Viele Grüße
Florian
[1] https://gitlab.dkrz.de/mpim-sw/libcdi/-/blob/master/examples/cdi_write_f.f#L93
[2] https://gitlab.dkrz.de/mpim-sw/libcdi/-/blob/master/src/stream.c#L1181
[3] https://gitlab.dkrz.de/mpim-sw/libcdi/-/blob/master/src/stream.c#L1182
[4] https://gitlab.dkrz.de/icon/icon-nwp/-/blob/master/src/io/shared/mo_name_list_output.f90#L512
[5] https://gitlab.dkrz.de/icon/icon-nwp/-/blob/master/src/io/shared/mo_name_list_output.f90#L490Uwe SchulzweidaUwe Schulzweidahttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/9ecCodes: make key startDate/Time per variable2022-03-10T13:33:03ZUwe SchulzweidaecCodes: make key startDate/Time per variableIn the current implementation the key startDate/Time is defined per timestep.
But it could be differ for each variable.
Example: variable TMAX_2M in data/cdt/dwd/testdata/Testdaten/tst_icon:
```
grib_ls tst_icon
tst_icon
edition ce...In the current implementation the key startDate/Time is defined per timestep.
But it could be differ for each variable.
Example: variable TMAX_2M in data/cdt/dwd/testdata/Testdaten/tst_icon:
```
grib_ls tst_icon
tst_icon
edition centre date dataType gridType typeOfLevel level stepRange shortName packingType
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 0 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 0 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 0-360 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-360 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-360 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 360 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 360 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 360-720 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-720 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-720 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 720 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 720 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 720-1080 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-1080 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-1080 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 1080 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 1080 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 1080-1440 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-1440 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-1440 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 1440 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 1440 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 1440-1800 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-1800 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-1800 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 1800 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 1800 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 1800-2160 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-2160 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-2160 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 2160 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 2160 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 2160-2520 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-2520 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-2520 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 2520 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 2520 sp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 2 2520-2880 TMAX_2M grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-2880 ASHFL_S grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 0-2880 tp grid_simple
2 edzw 20210224 fc unstructured_grid heightAboveGround 10 2880 10u grid_simple
2 edzw 20210224 fc unstructured_grid surface 0 2880 sp grid_simple
45 of 45 messages in tst_icon
```https://gitlab.dkrz.de/mpim-sw/yaco/-/issues/2Improve assignment of parallel tasks2023-09-21T12:19:44ZSven Willnersven.willner@mpimet.mpg.deImprove assignment of parallel tasksCurrently, the global MPI rank of the process needs to be specified in the configuration to assign the current process to the correct configuration subset.
- Use MPI_Comm_split to define a communicator only for YACO processes with (sub)r...Currently, the global MPI rank of the process needs to be specified in the configuration to assign the current process to the correct configuration subset.
- Use MPI_Comm_split to define a communicator only for YACO processes with (sub)ranks starting from 0. (Ok if only a subset of all MPI processes call MPI_Comm_split?)
- Adjust the configuration schema to have a list of the sub configuration under `ranks:` instead of a maphttps://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/16Uninitialized data in struct leads to segmentation fault of cdo.2023-12-05T13:28:07ZThomas JahnsUninitialized data in struct leads to segmentation fault of cdo.Use of uninitialized struct entries lbounds and ubounds in [vlist_generate_zaxis](src/vlist.c#L469) can lead to problems if any of the two is not NULL accidentally.
This was introduced in b1c602b52242a7 and affects releases from 2.0.5 o...Use of uninitialized struct entries lbounds and ubounds in [vlist_generate_zaxis](src/vlist.c#L469) can lead to problems if any of the two is not NULL accidentally.
This was introduced in b1c602b52242a7 and affects releases from 2.0.5 onward.https://gitlab.dkrz.de/mpim-sw/libcdi/-/issues/13use of uninitialized values in mo_cdi::vlistInqVarName2023-12-06T09:25:17ZRoland Wirthuse of uninitialized values in mo_cdi::vlistInqVarNameIn a valgrind run, I get
```
==3339454== Conditional jump or move depends on uninitialised value(s)
==3339454== at 0xDBD7AC9: string_len_trim (string_intrinsics_inc.c:240)
==3339454== by 0xDBD7AC9: _gfortran_string_len_trim (string...In a valgrind run, I get
```
==3339454== Conditional jump or move depends on uninitialised value(s)
==3339454== at 0xDBD7AC9: string_len_trim (string_intrinsics_inc.c:240)
==3339454== by 0xDBD7AC9: _gfortran_string_len_trim (string_intrinsics_inc.c:186)
==3339454== by 0x24918DE: __mo_cdi_MOD_vlistinqvarname (mo_cdi.f90:4179)
==3339454== by 0x84CE59: __mo_util_cdi_MOD_makeinputparameters (mo_util_cdi.f90:191)
==3339454== by 0x6B8663: __mo_ext_data_init_MOD_read_ext_data_atm.constprop.0 (mo_ext_data_init.f90:1203)
==3339454== by 0x6BD2B5: __mo_ext_data_init_MOD_init_ext_data (mo_ext_data_init.f90:350)
==3339454== by 0x442A8A: __mo_atmo_model_MOD_construct_atmo_model (mo_atmo_model.f90:579)
==3339454== by 0x44451D: __mo_atmo_model_MOD_atmo_model (mo_atmo_model.f90:186)
==3339454== by 0x40E4B6: MAIN__ (icon.f90:227)
==3339454== by 0x40D78C: main (icon.f90:30)
==3339454==
```
The line referenced, `if(name_dummy(name_i:name_i) /= ' ') exit`, doesn't call `LEN_TRIM`. But it might be that gfortran is very smart optimizing the enclosing DO to the equivalent `name_temp(len_trim(name_dummy)+1:) = c_null_char`. Anyway, in this context `vlistInqVarName` is called with an uninitialized `name_dummy`. This is correct, I believe, because the name should be an output of the function.
Digging deeper, it seems like the value of `name_temp` isn't read at all. The copy-in is unnecessary. There is another problem, though: the C function `vlistInqVarName` claims that the maximum length of the buffer it passes into `cdiInqKeyString` is `CDI_MAX_NAME` (=256). On the Fortran side the length of the buffer is `len(name_dummy) + 1`, which might be shorter and cause memory corruption. In this context `name_dummy` has a length of 1024, so it isn't an issue here.
My suggestion is to remove the copy-in part, declare `name_temp` with a fixed length `CDI_MAX_NAME`, and copy-out the first `len(name_dummy)` characters (this is already implemented correctly).https://gitlab.dkrz.de/mpim-sw/yaco/-/issues/3Make use of YAC's metadata channel2024-02-23T07:14:33ZSven Willnersven.willner@mpimet.mpg.deMake use of YAC's metadata channelYAC 3 now has a way to add metadata to each fields at definition. For reguar output coupling some of these metadata (the NetCDF part) is already provided in ICON (cf. `src/coupling/mo_output_coupling.f90`). This needs to be extended by t...YAC 3 now has a way to add metadata to each fields at definition. For reguar output coupling some of these metadata (the NetCDF part) is already provided in ICON (cf. `src/coupling/mo_output_coupling.f90`). This needs to be extended by the relevant GRIB metadata (for the parameter, horizontal grid, and vertical levels). Also, for coupling output_nml results, the metadata side from ICON needs to be added.