[pyicon_dwd] Difference plots for the atmosphere, area weighted horizontal mean of fields, new fields for atmosphere plots, etc.
The most important changes included:
-- diff plots (diff, RMSE, RMSE-diff) for the atmosphere are introduced in order to compare pairs of experiments. This is based on tools/run_qp_compare_atm_dwd.sh and pyicon/quickplots/qp_compare_atm_dwd.py. The latter uses the new function qp_timeseries_comp from pyicon/quickplots/pyicon_quickplots.py.
-- for the do_write_data_range option the simple horizontal mean/stdev calculation is replaced by an area-weighted horizontal mean/stdev calculation. See the new function calc_grid_area in pyicon/pyicon_plotting.py, which is called from hplot_base.
-- ts_gmean* plots limited to the time period defined by --tave_int (both in single exp and compare mode). This consists of a calculation of lend based on t2 (if not defined as argumrent) in qp_timeseries (and qp_timeseries_comp) in pyicon/quickplots/pyicon_quickplots. I did this because some of my colleagues needed a simple way to enable plotting time-series of sub-periods of the full experiment period.
-- new maps of fields added to the atmosphere (both single exp and comparison mode): TOA sw net flux map, TOA lw net flux map, Surface sensible heat flux map, Surface latent heat flux map, SST, seaice-fraction (the latter two might seem silly as we already have that for the ocean, but we wanted to have these also in AMIP type of runs to see how the prescribed ocean looks like beneath our atmo simulation).
-- I revitalized the link "pyicon documentation", which in our case was empty. Therefore, I created doc/doc_dwd/html/index.html, which is copied to the needed link when the QP runs. This is saved with a do_conf_dwd statement. I hope I did not overlook a better way to do this.
-- I created add_info_comp in pyicon/quickplots/pyicon_quickplots.py in order to give sense to he "additional info" link in case of diff plots (to have the namelists there from both experiments) I also added (both in add_info and add_info_comp a search for nml.atmo*, nml.ocean*, so that these are also clickable under the link "aditional info".
Merge request reports
Activity
requested review from @m300602
assigned to @m300602
added 1 commit
- a2d16e26 - [pyicon_dwd] Changes to get back to qp_driver.py from old_qp_driver.py:
added 1 commit
- 37b58ae8 - [pyicon_dwd] Use time_mode_atm=num2date. Update...
Hi Gergely, your changes look reasonable and I think that they will provide important expansions of the atmosphere plots! I am willing to except those as is but I would point you to some smaller changes/features which you might consider changing if you agree:
-- ts_gmean* plots limited to the time period defined by --tave_int: I think that this is not the best way of doing things. One can limit the time series by the amount of data which is supplied to the quickplots, e.g. by setting an appropriate value for tstep_ts. This decides which data is taken into account for doing the time series plots. If you always cut the timeseries to only tave_int then this will limit to see the averaging period in the context of the full simulation. I guess this could be a drawback for some applications.
-- the link for pyicon documentation should point to https://m300602.gitlab-pages.dkrz.de/pyicon/ This is the one which is build automatically by gitlab and which is (hopefully) updated soon.
Hi Nils,
Thanks for taking up these changes and for your comments.
-- I have changed the link to https://m300602.gitlab-pages.dkrz.de/pyicon/ as the main documentation website. This is already pushed to pyicon_dwd
-- regarding the ts_gmean* plots controled by --tave_int: I understand that there is the possibility to limit the set of files to be used via tstep, tstep_ts but it is not very flexible. It allows only the use of full days/months/years/decades/centuries and not to use something in between, e.g. 2001 January - 2001 July. Such short periods can be important when doing tuning. Moreover if using subsets of your data via tstep, tstep_ts, the dates in the headings of the html sites are misleading as they will show the period defined via --tave_int. We at DWD have also found it sometimes misleading that if we wanted to check only a sub-period of all the available data via --tave_int, the time-mean maps and cross-sections were not valid for the same period as the time series ts_gmean*. We would prefer to have these two sorts of diagnostics always valid for the same period to avoid misunderstandings. Another example when the current implementation is not optimal for us, is that when we have already a relatively long (say 20 years) baseline simulation and we want to tune something with many short (5 years) simulations and compare these tuning runs with the long baseline. In this case, with the current implementation, the comparison of the ts_gmean* figures is not handy because the baseline is always shown for 20 years but the tuning runs only for 5 years. One can of course create links of data subsets of the long baseline run in a separate directory for 5 years, but this seems much less comfortable than just controling the lenght of the ts_gmean* periods with a command line flag. On the other hand, I can well understand if this is not what you want. So, what would you say if I would restrict the calculation of lend based on t2 (as I proposed in the current merge-request) to the case when do_conf_dwd is true additionally? I would have to give then do_conf_dwd as an argument to qp_timeseries (and qp_timeseries_comp), which would mean that I would also have to give this variable over when calling these functions all over qp_driver.py (and qp_compare_atm_dwd.py). Would this be OK with you in this way?
Another solution I could imagine without passing do_conf_dwd to qp_timeseries (and qp_timeseries_comp) as argument is that in qp_driver.py we always call qp_timeseries with lend=lend, where lend is either defined as None (if not do_conf_dwd) or as 1 (if do_conf_dwd). Then in qp_timeseries (and qp_timeseries_comp) we redefine lend based on t2 only if lend==1.
Hi Gergely, sorry for this late response. I am not sure whether we understand each other correctly. I thought with tave_int you can do all what you wanted to do. E.g. you have 20 year spinup (assume 1900 - 1919) and then a five year production runs (1920-1925). In this case you would do someting like this: tstep = "192" and tstep_ts = "192". Now, only the year 1920, 1921, 1922, 1923, 1924 are taken for the time series plots.
However, I admit that it could get complicated to find appropriate wildcards if you wanted to select a time period, say 1917-1922.
What about the following suggestion: you define an additional variable, let's say tave_ts which per default contains the whole range of data but which can be set to e.g. tave_ts = 'tave_int' to use the same range as tave_int or which can be specified individually? By doing this, we get more flexibility and the default remains the same. I would try to use the "do_conf_dwd" as little as possible but prefere to implement more flexible settings.
I think you got my point. We would like to have more flexibility, eg. to be able plot time series like 1917-1922 or 01/01/1979 - 01/06/1980, etc.
On the other hand, I agree that the default should remain as it is now in master, i.e. that all the available data are plotted in the timeseries if not defined otherwise. What would you say if I add a boolean argument (say take_tave_int_for_ts) for qp_timeseries, which is by default False but if set to True (in the runscript executing qp_driver.py), the data is sliced (as with lstart, lend) to t1, t2. I suggest this because t1, t2 are already passed to qp_timeseries as argument and I would find it strange to pass yet another time interval (like you suggested tave_ts) containing basically the same information.
added 1 commit
- 31de83a4 - [pyicon_dwd] Update link for ducumentation. Update small stuff in some dwd tools.
added 1 commit
- 7350579f - [pyicon_dwd] small fix path_data --> path_data1 for printout when checking...
added 1 commit
- 1a7d32b7 - [pyicon_dwd] qp_compare_atm_dwd.py: skip SST and sea-ice fraction if the...
added 1 commit
- 8ab40792 - [pyicon_dwd] improve color scheme for sensible/latent heat flux maps
added 1 commit
- c1018ca0 - [pyicon_dwd] Rewrite optional restriction of time-series plots to the...
mentioned in commit e8804376