Orog file

Deals with one of the point mentioned here: #4 (closed)

Changes:

• Stepwise filtering of available orog files based on given facets
• Selection of first entry within foun dlist of available orog files

pet-wrapper-api.py

@@ -139,9 +139,34 @@ class PET(plugin.PluginAbstract):
             name="debug",
             default=False,
             help="Turn on to get better detailed debug information in case you face any issue."
-        )
+        ),
     )
 
+    def find_orog(self, config_dict):
+        '''
+        Retrieves a list of orographic files based on given facets and returns first entry as suggestion
+
+            Parameters:
+                config_dict (dict): Dictionary containing facet values to search database
+            Returns:
+                orog_files[0] (string): First entry of list of found files
+        '''
+        facets = ["product", "institute", "model", "experiment", "ensemble"]
+        #search_args = {"project": config_dict["project"], "variable": "orog"} for later if project facte is given
+        search_args = {"project": "nukleus", "variable": "orog"}
+        orog_files = list(freva.databrowser(**search_args))
+
+        for facet in facets:
+            search_args[facet] = config_dict[facet]
+            file_number = freva.count_values(**search_args)
+            
+            # if there are results, keep them and the facet
+            if (file_number > 0):
+                orog_files = list(freva.databrowser(**search_args))
+            else:
+                del search_args[facet]
+
+        return orog_files[0]
 
     def find_files(self, config_dict, variables: List[str], time_frequency: str, time: Optional[str] = None):
         '''
@@ -157,7 +182,7 @@ class PET(plugin.PluginAbstract):
         '''
         
         search_keys = (
-            # "project",
+            #"project",
             "product",
             "institute",
             "model",
@@ -173,8 +198,6 @@ class PET(plugin.PluginAbstract):
         file_dict = {}
         for variable in variables:
             search_args["variable"] = variable
-            if variable == "orog":
-                del search_args["ensemble"] ## orog files are part of another ensemble than the other variables
             files = list(freva.databrowser(**search_args))
             file_dict[variable] = files
         
@@ -317,7 +340,7 @@ class PET(plugin.PluginAbstract):
     def run_tool(self, config_dict=None):
 
         config_dict["files"] = self.find_files(config_dict, ["tasmax", "tasmin", "sfcWind", "pvap", "rsds"], "day", config_dict["start_date"] + " to " + config_dict["end_date"])
-        config_dict["alti_file"] = self.find_files(config_dict, ["orog"], "fx")["orog"]
+        config_dict["alti_file"] = self.find_orog(config_dict)
         if "alti_file" not in config_dict or not config_dict["alti_file"]:
             logger.error("Can't find altitude file")
         

[Mostafa Hadizadeh]

I think this config does the job. What do u think? do u see any concern on this in long run? https://www-regiklim.dkrz.de/plugins/pet/setup/

-        ),
+        ),
+        SolrField(
+            name="alti_file",
+            facet="uri",
+            mandatory=True,
+            help="Altitude file.",
+            predefined_facets={
+                "project": ["nukleus"],
+                "variable": ["rsds"],
+                "time_frequency": ["1day"],
+            },
+        ),

[Mostafa Hadizadeh]

here u go, Solr is down now with this approach

I don't think it's an efficient solution based on our resources

ERROR StatusLogger Reconfiguration failed: No configuration found for '1c4af82c' at 'null' in 'null'
Aborting due to java.lang.OutOfMemoryError: Java heap space
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (debug.cpp:362), pid=16, tid=93
#  fatal error: OutOfMemory encountered: Java heap space
#
# JRE version: OpenJDK Runtime Environment Temurin-17.0.13+11 (17.0.13+11) (build 17.0.13+11)
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.13+11 (17.0.13+11, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e" (or dumping to /opt/solr-9.6.1/server/core.16)
#
# An error report file with more information is saved as:

[Mostafa Hadizadeh]

in case if it works in web, for CLI we need to figure out how user is going to provide the full path of alti file. So the suggested approach is rejected!

I think we have to give up to the run_tool

[Bianca Wentzel]

So we could provide a field to provide an alti file if the user wants to specify it. Otherwise we choose one based on the submitted approach. What do you think?

[Mostafa Hadizadeh]

I think there is no need to a field. Let’s keep with this approach in run_tool as is.