Could not allocate memory

ed8454d8 · Fabian Wachsmann · 2359c916 · ed8454d8
Commit ed8454d8 authored 3 years ago by Fabian Wachsmann
--- a/notebooks/demo/use-case_plot-unstructured_psyplot_cmip6.ipynb
+++ b/notebooks/demo/use-case_plot-unstructured_psyplot_cmip6.ipynb
@@ -205,7 +205,6 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "!rm test.nc\n",
    "dset2.to_netcdf(\"test.nc\")"
   ]
  },

 %% Cell type:markdown id: tags:

 # Plot ESM data on *unstructured* grids with `psyplot`

 This notebook introduces you to the `mapplot` function of the package `psyplot` and its plugin `psy_maps`.
 It is suitable to plot maps from data on unstructured grids like the ones from ICON and FESOM.

 We therefore search for the corresponding data in the CMIP6 data pool with intake-esm.
 Afterwards, we open a file with `xarray` and configure the opened xarray dataset as well as psyplot for a map plot.

 This Jupyter notebook is meant to run in the [Jupyterhub](https://jupyterhub.dkrz.de/hub/login?next=%2Fhub%2Fhome) server of the German Climate Computing Center [DKRZ](https://www.dkrz.de/). The DKRZ hosts the CMIP data pool including 4 petabytes of CMIP6 data. Please, choose the Python 3 unstable kernel on the Kernel tab above, it contains all the common geoscience packages. See more information on how to run Jupyter notebooks at DKRZ [here](https://www.dkrz.de/up/systems/mistral/programming/jupyter-notebook).

 Running this Jupyter notebook in your premise, which is also known as [client-side](https://en.wikipedia.org/wiki/Client-side) computing, will require that you install the necessary packages
 `intake`, `xarray`, `maplotlib`, `psyplot`, `psy_maps`

 and either download the data or use the `opendap_url` column of the intake catalog if available.

 %% Cell type:markdown id: tags:

 ### Learning Objectives

 - How to access data on an *unstructured* grid from the DKRZ CMIP data pool with `intake-esm`
 - How to subset data with `xarray`
 - How to visualize the results with `matplotlib`, `psyplot` and `psy_maps`.

 %% Cell type:code id: tags:

 ``` python
 import psyplot.project as psy
 import matplotlib as mpl
 import xarray as xr
 import intake
 ```

 %% Cell type:markdown id: tags:

 We open a swift catalog from dkrz cloud which is accessible without authentication.

 %% Cell type:code id: tags:

 ``` python
 col_url = "https://swift.dkrz.de/v1/dkrz_a44962e3ba914c309a7421573a6949a6/intake-esm/dkrz_data-pool_cloudcatalog.yaml"
 parent_col=intake.open_catalog(col_url)
 list(parent_col)
 ```

 %% Cell type:code id: tags:

 ``` python
 col=parent_col["dkrz_cmip6_disk_netcdf_fromcloud"]
 ```

 %% Cell type:markdown id: tags:

 In this example, we aim at plotting the **Sea Surface Temperature** `tos` of the upper boundary of the liquid ocean, including temperatures below sea-ice and floating ice shelves from the earth system model **AWI-CM-1-1-MR**.
 We therefore search for `tos` in the catalog for monthly frequency. We only use one realization `r1i1p1f1` of one experiment only.

 %% Cell type:code id: tags:

 ``` python
 tos=col.search(source_id="AWI-CM-1-1-MR",
               experiment_id="ssp370",
               variable_id="tos",
               table_id="Omon",
              member_id="r1i1p1f1")
 ```

 %% Cell type:code id: tags:

 ``` python
 tos.df["path"].to_list()[0]
 ```

 %% Cell type:markdown id: tags:

 We now open the file on the mistral lustre file system. Note that if you work remotely, you could try to use `opendap_url` instead of `path`.

 %% Cell type:code id: tags:

 ``` python
 dset = xr.open_dataset(tos.df["path"].to_list()[0],
                      decode_cf=True,
                      chunks={"time":1})
 dset
 ```

 %% Cell type:markdown id: tags:

 In order to make `tos` plottable, we set the following configuration.
 - The `CDI_grid_type` is a keyword for `psyplot`. It must match the *grid type* of the source model.
 - Coordinates are not fully recognized by `xarray` so that we have to add some manually (version from Dec 2020).

 %% Cell type:code id: tags:

 ``` python
 dset["tos"]["CDI_grid_type"]="unstructured"
 coordlist=["vertices_latitude", "vertices_longitude", "lat_bnds", "lon_bnds"]
 dset=dset.set_coords([coord for coord in dset.data_vars if coord in coordlist])
 ```

 %% Cell type:markdown id: tags:

 The following is based on the [psyplot example](https://psyplot.readthedocs.io/projects/psy-maps/en/latest/examples/example_ugrid.html#gallery-examples-example-ugrid-ipynb). We set a resoltion for the land sea mask `lsm` and a color map via `cmap`.

 %% Cell type:code id: tags:

 ``` python
 psy.rcParams['plotter.maps.xgrid'] = False
 psy.rcParams['plotter.maps.ygrid'] = False
 mpl.rcParams['figure.figsize'] = [10, 8.]
 ```

 %% Cell type:code id: tags:

 ``` python
 def plot_unstructured():
    iconplot11=psy.plot.mapplot(
        dset, name="tos", cmap='rainbow',
        clabel=dset["tos"].description,
        stock_img=True, lsm='50m')
 ```

 %% Cell type:markdown id: tags:

 We now do the same with a smaller subset to highlight the fine resolution and the structure of the AWI ocean model FESOM.
 We first *subset* the data because otherwise plotting takes too long. We choose indices of dimensions with the `xarray` function `isel`. We select a slice of two time steps and focus on a region Ireland. We have to save the data to an intermediate file `test.nc` because otherwise we receive an error.

 %% Cell type:code id: tags:

 ``` python
 dset2 = dset.isel(time=slice(1,2)).where( (dset.lon > -10. ) &
                    (dset.lon < 50. ) &
                    (dset.lat > 40. ) &
                    (dset.lat < 70. ), drop=True).drop("time_bnds")
 ```

 %% Cell type:code id: tags:

 ``` python
 dset2
 ```

 %% Cell type:code id: tags:

 ``` python
-!rm test.nc
 dset2.to_netcdf("test.nc")
 ```

 %% Cell type:code id: tags:

 ``` python
 del dset2
 ```

 %% Cell type:code id: tags:

 ``` python
 dset=xr.open_dataset("test.nc",
                      decode_cf=True)
 ```

 %% Cell type:code id: tags:

 ``` python
 dset["tos"]["CDI_grid_type"]="unstructured"
 coordlist=["vertices_latitude", "vertices_longitude", "lat_bnds", "lon_bnds"]
 dset=dset.set_coords([coord for coord in dset.data_vars if coord in coordlist])
 ```

 %% Cell type:code id: tags:

 ``` python
 psy.plot.mapplot(
    dset, name="tos", cmap='rainbow',
    lonlatbox='Ireland',
    clabel=dset["tos"].description,
    stock_img=True,
    lsm='50m',
    datagrid=dict(c='b', lw=0.2)).show()
 ```

 %% Cell type:markdown id: tags:

 ### Used data

 - [Semmler et al., 2019: AWI AWI-CM1.1MR model output prepared for CMIP6](https://doi.org/10.22033/ESGF/CMIP6.2803)

 %% Cell type:markdown id: tags:

 We acknowledge the CMIP community for providing the climate model data, retained and globally distributed in the framework of the ESGF. The CMIP data of this study were replicated and made available for this study by the DKRZ.”