Skip to content

Rework of file handling to object-oriented approach

Andreas Baer requested to merge dev/input-rework into main

The whole (input) file handling mechanism was reworked to use an object-oriented approach. Following this, most of the scripts (prepare_member, remapping, samoa and validation) are rewritten in Python to make use of it. A few other internals were changed:

  • Python utils are now stored in a separate directory (in LOGDIR).
  • There is a workflow directory (in LOGDIR) storing information on used domains and files (as a list of Python objects).
  • Added a job to remap ERA5 data to the ICON grid (Levante only).
  • Removed bash_utils.sh (obsolete).
  • Renaming: process_grid_info -> icon_file_handler.
  • Separate job added to collect all input files in the beginning.
  • Added job to clean up remote before execution (for testcases).
  • Added switches for turning pipelines on and off for Horeka and Levante.
  • Added an experiment config python module.
  • Did a major cleanup.

Fixes #113 (closed). Closes #51 (closed), #116 (closed), #126 (closed).

Edited by Andreas Baer

Merge request reports

Loading