The source_idMPI-ESM1-2-CR already has been used for Maries data in a different configuration and we cannot overwrite it anymore as soon as her data has been published.
We have three options:
Define all model components in the source_id entry MPI-ESM1-2-CR and refer to the activated ones using the source_type attribute. For example MPI-ESM1-2-CR would have atmosChem: unnamed (accelerated methane chemistry)\n defined, and for the experiments where this component is active, the source_type would be appended by CHEM, eg. AOGCM CHEM. This option is possible until we publish Maries data (where the atmosChem component would have to be added to the source description using NCOs). In a similar case, Marie has already spoken out against this procedure, so you would have to discuss it with her. This would however be the proper solution (also when there have been numerous examples within CMIP6 also for the next two options).
Another option is to use Maries source_id entry but add a comment to the metadata of Thomas' files to denote that this accelerated methane chemistry has been active / used.
The 3rd option is to add new source_id entries for MPI-ESM1-2-CR anytime there is a small change to the model, eg. MPI-ESM1-2-CR-AM (for accelerated methane). In that regard it might be good to know from the modelers how many such entries we can expect, since it can get confusing quite quickly.
One could somehow switch between options 2 and 3, depending on how severe the changes are.
experiment_id
I suggest the following experiment entries (more in accordance with CMIP6 and stripped of illegal characters):
Question: Is this experiment the same as transient-deglaciation-prescribed-glac1d and only one more model component was active? Or did the experiment + forcing really change? In the first case we don't have to add a new experiment entry but can rely on the changes with respect to Maries simulations to be denoted by the source_id or other metadata.
Question: Do the future experiments all branch from the transient-deglaciation run? Then we can add this to the CV as well as parent_experiment_id (as shown here)? How do the future experiments differ from the standard CMIP6 ssp-experiments? Has the same forcing been used or has additional forcing been used? From the excel sheet it looks like the future experiments are a combined "historical experiment + future scenario"?
For source_id, I'd suggest to use Marie's ID and add a comment, as otherwise we will have quite a number of separate IDs, basically one for each modeller.
For your question on the experiment_id -- it's a completely separate experiment, with a few differences between the experiments.
For the future experiments -- these do branch from the deglaciation experiment. They differ from the standard SSP experiments mainly by having an interactive methane cycle and by their time scale: 1000 years instead of the usual 100 years. Forcings were pretty much standard, though simplified (no aerosols, no landuse). And yes, it's combined historical and future.
@m300020
For the transient deglacation methane experiment, you replaced pmt0531_a-d with pmt0585_a-d:
pmt0531_a (24900-18001 BP = 2100-8999)pmt0531_b (18000-11001 BP = 2000-8999)pmt0531_c (11000-4001 BP = 2000-8999)pmt0531_d (4000-1 BP = 2000-5999)
pmt0585_a (24000-18001 BP = 3000-8999)pmt0585_b (18000-11001 BP = 2000-8999)pmt0585_c (11000-4001 BP = 2000-8999)pmt0585_d (4000-1 BP = 2000-5999)
Now I wonder, how to define the parent attributes for the SSP* simulations, because in the CV we have the SSP* experiments branching from the transient deglaciation methane experiment.
The SSP* runs branched at the year 5900 from pmt0531_d and with our plan to publish pmt0531 as transient deglacation methane r1i1p1f1, this would be no problem, but now it was replaced with pmt0585 and thus we need to make sure to correct the parent attributes if necessary.
If I understood properly, pmt0585 is basically the same as pmt0531 (only qualitatively or are the restart files equal as well?)?! Which year of the pmt0585_d would then correspond to pmt0531_d@5900, is it 5900 as well? If it is not possible to make that assumption / statement, we would have the SSP* experiments branch of an unpublished transient deglaciation methane r2i1p1f1 (or whatever member-id you then choose for that unpublished pmt0531 run).
First SSP119 standardized data is available at:
/work/bk1192/k204212/palmod_TK/ssp119-future-methane_r1i1p1f1-CR/archive/cmor_1950-2049
It seems herbivore CH4 is not part of the model output for the SSP runs?!
There is a maximum string length in CMOR, leading to the references lacking the paper about high resolution. I will add the missing reference using the NCO toolkit. Also the branch / parent attributes are not yet final. Are the attributes else ok (for the transient simulation I accidently used Maries attribute configuration, which I will correct soon)?
@m300020 Now I produced all possible variables for 100 years of the SSP119 simulation:
/work/bk1192/k204212/palmod_TK/test_ssp119-future-methane_r1i1p1f1-CR/archive/cmor_1950-2049/
The following variables did not get produced:
jsbach_methane_mm: variables missing in stream
herbivoreCH4.Emon
mpiom_data_2d_mm: variables missing in stream
simass.SImon
sisnthick.SImon
sithick.SImon
/pool/data/MPIOM/input/r0010/GR30/GR30_basin.nc: basin file does only exist for GR15 and TP04
basin.Ofx
mpiom_data_3d_mm: stream missing
volcello.Ofx
volcello.Omon
uo.Omon
vo.Omon
wmo.Omon
umo.Omon
vmo.Omon
thetao.Omon
deptho.Omon
so.Omon
thkcello.Ofx
thkcello.Omon
yassomon, vegmon: streams missing
npp.Lmon
cLitter.Lmon
gpp.Lmon
cVeg.Lmon
rh.Lmon
fFire.Lmon
hd_higres_mon: stream missing
rivo.Lmon
areacellr.fx
mpiom_data_moc: data processing problems
sltbasin.EmonZ
The produced variables you find under above path. Are variables missing entirely you would want to publish, do you need any of the variables that did not get produced successfully or can some produced variables be excluded from the post processing?
that looks great. Metadata checks out, I think, and we don't really need the high resolution ECHAM reference, so we might as well skip that -- less postprocessing.
I am not all that happy about the "yassomon, vegmon: streams missing" error message and the derived output files that couldn't be created. These would be great to have, if possible at all. We can generate those from other input streams, I believe -- should I provide you with recipes?
With regard to the parent attributes for the SSP* simulations, this did indeed become a little tricky.
I'll lay out what I did to initialise those experiments: I branched off from pmt0531_d in model year 5600, which corresponds to 400BP / 1550 CE. I then ran a historical experiment with all forcings, which did not differ significantly from pmt0531_d forcings before 1750 CE / 5800.
The pmt0585_X is identical to pmt0531_X, with segments started off the pmt0531_X restart files, the only difference is in some of the methane code, with no feedback to climate. The climate therefore is identical (though not bit-identical) to the pmt0531_X experiment.
Thus for all practical means and purposes, we branched the experiments off from the pmt0585_d experiment in model year 5800. Climate would be identical, there would only be minor differences in C-cycle state between the pmt0585_d and the pmt0531_d.
yes, I think it is better to use pmt0585_d@5800 as official branch time + parent. I do not think it should matter for any users of the data that it actually branched from pmt0531_d, or do you think there might be any confusion when they look at the results?
For the SSP* runs, there is only one restart / input file for jsbach (jsbach_T31GR30_11tiles_5layers_natural-veg_100k.nc), so I am using this for the entire timeline of the simulations. If that is not correct, please say so.
Regarding the missing yassomon/vegmon variables: I was using the mapping information from CMIP6 because no one updated them so far for PalMod. And in PalMod the streams are called yasso_mm and veg_mm. After making this change, the variables got produced:
/work/bk1192/k204212/palmod_TK/test_ssp119-future-methane_r1i1p1f1-CR/archive/cmor_2050-2149/jsbach/
Still missing are the SImon variables and the Omon variables. In case of the Omon variables I saw that you have an annual mean instead of a monthly mean stream. Do you want to publish the annual mean 3d ocean variables as a replacement for the missing monthly mean 3d variables?
Regarding the additional NCO postprocessing: It is anyways required to run NCO for some other attributes that cannot be added at this point, so it is no problem to add the reference paper as well.
Lastly, there is one more variable (fVegLitter, see below) that I was asked to add to the MIP tables at some point. For this variable I do not have any information yet about, which variable(s) in the jsbach output it corresponds to. If you want to submit this variable as well, I would require this information.
Else I can start with the processing of the full SSP* simulations.
fVegLitter
Total Carbon Mass Flux from Vegetation to Litter
mass_flux_of_carbon_into_litter_from_vegetation
In accordance with common usage in geophysical disciplines, flux implies per unit area, called flux density in physics. Vegetation means any living plants e.g. trees, shrubs, grass. Litter is dead plant material in or above the soil. It is distinct from coarse wood debris. The precise distinction between fine and coarse is model dependent. The sum of the quantities with standard names mass_flux_of_carbon_into_litter_from_vegetation_due_to_mortality and mass_flux_of_carbon_into_litter_from_vegetation_due_to_senescence is mass_flux_of_carbon_into_litter_from_vegetation.
I am glad you were able to find the vegmon variables.
We can get the fVegLitter from the box_litter_flux in the jsbach_veg stream (code 175) -- needs sum over all tiles and multiplication with molar mass of carbon (12.01 x 1.e-3 kg/mol).
For the SImon, I have no idea how to obtain that, and yes, i didn't output the monthly 3D ocean state since I never needed that, and annual should be suficient. So yes, let's publish the annual variables instead (I assume it's better to have annual than to have nothing at all)
Other than that, I think it's best to use the pmt0585 as a base experiment for the SSP ones -- that should not cause any issues / confusion -- and you are quite right, there is only one jsbach.nc for the SSP experiments.
Anything else? yes, the most important thing: Have a good weekend!
The problem with the annual 3D mpiom stream is now, that it does not contain any of the variables that are missing (uo, umo, so, deptho,...), but rather variables that are not (yet?) part of the PalMod2 variable list (tendency of sea water salinity / tendency of potential temperature in different variations, diffusivity, ...). Since they were part of CMIP6, these variables could be processed for PalMod2 quite easily if you wanted that? But sadly the other ocean variables cannot be provided in any frequency.
I found some problem with the transient simulation that you would need to address:
for mpiom output, in the pmt0585_d subfolder, there are some files of pmt0585_c. Similarly for the other sub-simulations, they each contain some data from the previous sub-simulation. Could you move these files please?
@k204212 Sorry for the mixup with the MPIOM files -- I thought I had already solved that issue, but I was obviously mistaken. Should be fixed by now, please let me know if not.
If the MPIOM 3D output is missing important stuff, then let's skip it -- I assume it's far less interesting for the majority of potential users, and if someone really needs it, they can always look at Marie's experiments.
Can you provide these files please?
If it is meant to use the 100k file for the last decades, could you please create softlinks to the 100k file with above names?
@k204212 Sorry about that, and you were right, the boundary conditions are assumed constant after 1850, so the *100k file is used.
I linked it accordingly.
@m300020 I processed the transient run, the results are here:
/work/kd1292/k204212/palmod_TK/transient-deglaciation-prescribed-glac1d-methane_r1i1p1f1-CR/archive/PalMod2/
I will run the usual QC required for ESGF publications, but you can also have a look (spot checks) if you notice any problems.
@k204212 Sounds fabulous. I'll have a look at the data, but as we're going on the MPI retreat next week, I likely won't be able to really look at it until 27/01.
@m300020 The QC for the transient simulation and the ssp* runs was successful. The only problem was the following:
Attribute <emilnox>:units = <kg m-2 s-1> is not CF compatible with standard_name = <tendency_of_atmosphere_moles_of_nox_expressed_as_nitrogen>.
There are 3 options to deal with this:
My preferred option: Regard this as a non-crucial warning and ignore it. This is also backed by our inhouse ESGF QC expert.
We provide this variable in mol s-1.
There is no CF standard_name yet with mass content instead of moles, like tendency_of_atmosphere_mass_content_of_nox_expressed_as_nitrogen, so we could also register it, which however would take weeks to months.
Since I forgot to tag your username in my last gitlab issue update, here it is again: All 5 ssp* runs are processed. The CMORized data of all your 6 simulations is available here:
After your OK the data can be published quite quickly. I will only have to create a tag of this gitlab repo (scripts + config + CMOR-tables) beforehand and add this information to the cmorized files using NCO ncatted, to make everything reproducible.
@k204212 Thanks for the update and greetings from Berlin.
For emilnox: If we are free to ignore that warning, then let's do so. The conversion to mol is not a real issue, though, so we could also do that -- I rely on you for guidance in this, as I have zero experience here.
With regard to spot-checks -- I am still in Berlin, won't be able to do real work before Friday. I'll have a look at the results then, I assume we can then publish it quickly.
@k204212 I've done a quick check -- as far as I can see, everything looks quite all right. So I guess we can publish it now.
However, I am wondering how much trouble it would be to add another reference to the deglaciation data -- we do have a paper using that data in review now, which is already published as a citable discussion paper. If it's a lot of trouble, I'd say let's not do it, but if it's not a lot of trouble, then it would be nice.
Also -- in that paper there's a second experiment referenced, branched off from this one, so it's nearly identical. We should get that no the server at some point, too...
@m300020 It is no problem to add another reference. It would be added on the top of the list or at which position? You can edit the GoogleSpreadSheet (#14) or just paste it here.
If there is another experiment, you can add it to the GoogleSpreadSheet referenced in #14 and let me know here when it's there, then we can process it as well for ESGF. Important for now is the CV-information (eg. parent experiment, name of the experiment and a title). It would be great if you did that before we do the current ESGF publication, so it can already be included in the tag of the data standard and scripts for Maries and your simulations.
@k204212 I am preparing the data for the second experiment right now, should be ready by tonight, and I'll add everything to the google sheet later today -- though it'll likely be after lunch before i get around to it.
@k204212 OK... I think I've added everything to the google sheet.
I am unsure, however, about experiment_id and variant_label -- for now I've used the same experiment_id and a variant_label with "f2" instead of "f1", as it is a perturbation experiment branched off from the original pmt0585. I've put a description of what I did into the "comment" field -- please do let me know if I should better change that.
Data is being copied to the directory right now, should be complete later tonight.
Furthermore, I've put the additional reference into the google doc. As that paper describes the experiment, it should be listed first, I believe.
@m300020 Good morning Thomas, I would have two more questions:
Should the additional reference paper also be added to the metadata of the ssp*-future-methane simulations?
I suggest the following text as variant_info. As it still counts as a transient-deglaciation-glac1d-methane simulation (that officially has no parent experiment), one has to put the branch information into the variant_info attribute (or alternatively define it as its own experiment that branches of transient-deglaciation-glac1d-methane):
Palmod CR setup with full methane cycle. Model physics identical to MPI-ESM1-2-CR simulation transient-deglaciation-prescribed_r1i1p1f1. This simulation branched of transient-declaciation-prescribed-glac1d-methane_r1i1p1f1 at 7001-01-01 00:00 (corresponding to 18ka BP). Compared to the original simulation it includes a meltwater perturbation: Meltwater from Laurentide ice sheet stored from 15.2 ka BP to 12.8 ka BP, then released to ocean for 1200 years. This induces an AMOC collapse, leading to a Younger Dryas event, including recovery, as described in Kleinen et al. (2022).
@m300020 Good morning Thomas, I would have two more questions:
Should the additional reference paper also be added to the metadata of the ssp*-future-methane simulations?
I suggest the following text as variant_info. As it still counts as a transient-deglaciation-glac1d-methane simulation (that officially has no parent experiment), one has to put the branch information into the variant_info attribute (or alternatively define it as its own experiment that branches of transient-deglaciation-glac1d-methane):
Palmod CR setup with full methane cycle. Model physics identical to MPI-ESM1-2-CR simulation transient-deglaciation-prescribed_r1i1p1f1. This simulation branched of transient-declaciation-prescribed-glac1d-methane_r1i1p1f1 at 7001-01-01 00:00 (corresponding to 18ka BP). Compared to the original simulation it includes a meltwater perturbation: Meltwater from Laurentide ice sheet stored from 15.2 ka BP to 12.8 ka BP, then released to ocean for 1200 years. This induces an AMOC collapse, leading to a Younger Dryas event, including recovery, as described in Kleinen et al. (2022).
@m300020 I cmorized the run at:
/work/kd1292/k204212/palmod_TK/transient-deglaciation-prescribed-glac1d-methane_r1i1p1f2-CR/archive/PalMod2/
Please have a look if any metadata needs to be changed.
I added the additional reference to transient*_r1i1p1f1 as well.
@m300020 Then everything should be "good to go" from my perspective. The QC did not have anything more to warn about than it had to with the f1-run. I wait for another final "OK" of you before advancing the runs to the publication.
Only one comment: for the f2-run there were four *_mpiom_fx_* files, for the simulation years 2000, 4000, 6000 and 8000. I processed the first one. Since the mpiom_data_3d_mm stream is missing, only the initial values of thkcello (ocean layer thickness), volcello (ocean grid cell volume) and deptho (ocean layer depth) can be captured this way (for all your simulations). For MK's simulations these variables are additionally provided at decadal frequency.
@k204212 Excellent! Then everything is good to go from my perspective, too. I have been in contact with Beate about some verbal description of the experiments, I guess that would be needed at some point, too?
About the *_mpiom_fx_* files -- frankly I have no idea what they are good for, I only know they are produced by the *.run_start script. So I have always ignored them in the past...
@m300020 Okay, I will create the publication request then and add you in cc. And I will enjoy closing the first PalMod2 gitlab issue
The experiment descriptions etc. are required for the long term archival (LTA in the WDCC archive). The LTA will start after ESGF publication, but they collect some metadata for that already so it can happen swiftly and in time for the associated PalMod2 deliverable deadline.