How to run my example plugin?

Let's find out a solution for this in a meeting on Friday after the data management meeting, then we will write the solution here afterward

@g300050 could you please move the plugin to ch1187/plugins4freva, then I will be able to push a couple of commits there to fix?

https://docs.gitlab.com/ee/user/project/settings/migrate_projects.html#:~:text=To%20transfer%20a%20project%3A

@k202187 To transfer the project is a good idea, but it does not work as described in the gitlab docs for me. I am not able to select a namespace other than my own --> no group ch1187/plugins4freva can be selected. I think we need to tackle this transfer problem on Monday together. I will reach out ...

then if it's possible, could you please add me as maintainer to this repo

ok, that should have worked.

If you have time to work on it today (and explain it to me), please let me know. I am available until 3:30 pm today

@k202187 sorry, I am not a natural with this git-stuff, but I hope you have maintainer access now

Hi hi, yes i have the maintainer access now to this repo, thakns

Cause I'm on a freva-related task from morning, if it's possible please give me a bit more time to have a look and push commits here and also test it on one Freva instance to find out a solution for it. then I will check with you.

If you are in rush, let me know to change my schedule and make it happen for today, otherwise i will let u know about the time of meeting

And about transferring the project to Nukleus group, we can do it when we make meeting together, i mean no worries

I am not in a rush exactly, but I will leave for holidays on Thursday (for 3 weeks), and I would like to finish this task until then. I we could solve "everything" until or on Wednesday, that would be great!

Thanks for making your schedule clear so i can make better plan. no worries, tomorrow I will go for it for sure, and I will keep you posted ...

I will be offline for today. Let's talk tomorrow!

Sorry Bente, whole day I was plugged into another task, and i couldn't answer earlier ...

This evening i dive here

Hmmm, I had a look at the scientific_code. As there is no core there to run, it seems you want to see how it works.

I highly recommend you to have a look at the following plugin and see how it defines parameters and how it calls the scientific core and so on. Then make it step by step based on this to see how it works.

https://gitlab.dkrz.de/freva/plugins4freva/plugin_workshop/example_plugin

Just a points:

Config_dic is the parameters that user enters when it runs the plugin.

For example config_dict is the keys here:

Freva plugin <name of plugin> <keys>

And after 11 I’m all free to make meeting, how about u?

Good morning @k202187! Let's meet at 11:45 here: https://hereon-de.zoom.us/j/63511166751?pwd=5vGUTVNgmazJzK6lK9lU8UOGStmEMK.1 Meeting ID: 635 1116 6751 Password: 168904

I have already looked at the example plugin from the workshop, but I will do it again right now to focus my questions to you. Overall, I find it confusing that all the plugins I have looked at (old and new) look different. I mean the basic commands are the same (mostly), but they are all structured a little differently...

Overall, I find it confusing that all the plugins I have looked at (old and new) look different. I mean the basic commands are the same (mostly), but they are all structured a little differently...

exactly this is the reason I made plugin template to have a structure on our plugins and make them readable

Btw good morning and yeppp, see u at 1145

Transferring the project worked after Mo changed something with his admin/owner/whatever-rights in the background, so that the access to the group namespaces was granted and visible for me.

:)) i changed your role from developer to maintaner to be able to transfer and then immidiately after transferring I changed it back to developer

How are the input/facets (that are defined in the wrapper as a configuration dictionary (config_dict)) passed to a scientific python code?

Important lines in the wrapper:

def run_tool(self, config_dict=None):
    import os
    import json
    from pathlib import Path
    os.makedirs(config_dict["cache"], exist_ok=True)
    os.makedirs(config_dict["outputdir"], exist_ok=True)
    errorfile = Path(config_dict["cache"]) / "error.log"
    json_file = Path(config_dict["cache"]) / "out.json" 
    with json_file.open("w") as json_f:
        json.dump(config_dict, json_f, indent=4)
    with errorfile.open("w") as error_f:
        self.call(f"python {Path(__file__).parent / '/src/heat2urbanimpact_test/scientific_code.py'} {json_file}", 
                    stdout=error_f, stderr=error_f)stderr=error_f,)

Assuming that you have written a python function in your scientific code that needs as input what is dumped (and stored in cache) in the json file, these lines are important:

import sys
import json

with open(sys.argv[1]) as f:
    config_dict = json.load(f)

def some_function(outputdir,experiment,....):
    #write function

some_funtion(config_dict["outputdir"],config_dict["experiment"],...)

Thanks. I think in the first code snippet there is one exceeding stderr=error_f,) in the very end, i.e. it should read

def run_tool(self, config_dict=None):
    import os
    import json
    from pathlib import Path
    os.makedirs(config_dict["cache"], exist_ok=True)
    os.makedirs(config_dict["outputdir"], exist_ok=True)
    errorfile = Path(config_dict["cache"]) / "error.log"
    json_file = Path(config_dict["cache"]) / "out.json" 
    with json_file.open("w") as json_f:
        json.dump(config_dict, json_f, indent=4)
    with errorfile.open("w") as error_f:
        self.call(f"python {Path(__file__).parent / '/src/heat2urbanimpact_test/scientific_code.py'} {json_file}", 
                    stdout=error_f, stderr=error_f)

How to plugin the plugin?

cd work/ch1187/clint/freva-dev/share/work/g300050/freva-dev/
git clone -b <branchname> https://gitlab....git
module load clint freva-dev
export EVALUATION_SYSTEM_PLUGINS=/work/ch1187/clint/freva-dev/share/work/g300050/freva-dev/heat2urbanimpact-test,heat2urbanimpact-test-wrapper-api
freva plugin heat2urbanimpact_test

#to list loaded plugins
freva plugin --list
#to see the input variables, help texts etc.
freva plugin heat2urbanimpact_test --doc

Got to https://www.freva.dkrz.de/ to plugin your plugin via button

hey @g300050, is everything alright in this plugin? any help? any progress? it's a long time i haven't heard from you, I wanted to just check in with you :)

Feel free to ping us if you need any help with the plugin. As an option, we’d be happy to make a meeting, whenever you have time, to find out an initial point of development or any support we can offer. As another option, during the meeting, you can walk us through the research you’re conducting with the plugin, and we can brainstorm some ideas together. What do you think?

ping @k202206

Hi @k202187, thanks for checking in and the offer to help. I will take up the work on the plugin next week. So stay tuned ;-)

great to hear, looking forward ...

moved from heat2urbanimpact-test#3 (moved)

How to run my example plugin?

Designs

Child items ...

Activity