Skip to content

Consolidation with CDI-PIO (1.8.x)

Sergey Kosukhin requested to merge cdi-1.8.x-pio-merge into cdi-1.8.x

This is a consolidated version of the cdi-1.8.x branch that is used in ICON and the master branch of CDI-PIO.

First, all automatically generated Autoconf files were removed from the master branch of CDI-PIO to make conflict resolution easier. The only modification to the files that I noticed not to be exported as a patch file was this one. It's now a patch. @jahns, please, feel free to remove it if it's not needed anymore and add other ones that I missed and that are relevant.

The next step was to rebase the master branch of CDI-PIO onto the cdi-1.8.x branch and resolve conflicts. There might be things that I did wrong here but I would like to ask @jahns and @m214003 to review a couple of things:

  1. The conflict between A@libcdi and B@cdi-pio was resolved in C. Does anybody know whether this code is relevant for ICON? @m214089?
  2. The conflict between D@libcdi and E@cdi-pio + F@cdi-pio was resolved in G + H.
  3. The modification I@cdi-pio was addresses in K.

At this point, the code passed all the tests from the test-suite, at least with gcc and gfortran.

The rest of the modifications should be trackable from the history but here is a short summary:

  1. Configuration:
    • more checks are run only if needed for a particular configuration;
    • apply almost all patches (with the already existing ./autogen.sh script) for Libtool that YAXT does;
    • real checks (header and library presence) for YAXT and PPM;
    • protection from a potentially "dirty" build environment (e.g. when nc-config in the PATH does not belong to the NetCDF library that is provided in the LIBS, which is the case when CDI is configured as part of the ICON building process);
  2. Testing:
    • all tests can be run in parallel;
    • MPI tests are run one after another unless configured with --with-concurrent-mpi-checks;
    • examples and tests are built during make all unless configured with --without-example-programs and --with-on-demand-check-programs, respectively;
    • if a feature is disabled, at least one of the tests is skipped (easier to see that some of the features were disabled when running make check);
    • do not check for less when configured with --enable-mpi, e.g. always run pio_write_serial and additionally run pio_write_parallel (in the previous implementation, only one of the two was run, depending on whether MPI support was enabled);
    • prototypes for CI for different platforms/compilers, including tests that check the way CDI is going to be integrated into ICON.

@k202125, could you please "protect" the cdi-1.8.x and cdi-1.8.x-pio-merge branches as follows:

  • Allowed to merge: Developers + Maintainers
  • Allowed to push: Developers + Maintainers
  • Allow force push: disabled.

Everyone is welcome to introduce the required changes directly into cdi-1.8.x-pio-merge. The plan is to merge the branch into cdi-1.8.x once we get a "good to go" from DKRZ.

In the meantime, the Buildbot of ICON is green with cdi-1.8.x-pio-merge (including MISTRAL_intel18_pio, which was modified not to use an external installation of CDI-PIO, YAXT and PPM): https://gitlab.dkrz.de/icon/icon-cimd/-/merge_requests/32

Other people who might be interested in this: @k202082, @k202061, @m218027, @m300433.

Edited by Sergey Kosukhin

Merge request reports