## Using Jupyter Notebooks for Model Data Analysis
...
...
@@ -26,21 +26,13 @@ To run the notebooks, you only need a browser (like Firefox, Chrome, Safari,...)
1. Open the [DKRZ Jupyterhub](https://jupyterhub.dkrz.de) in your browser.
2. Login with your DKRZ account (if you do not have one account yet, see the links above).
3. Pick a profile (``Preset -> Start from Preset Profile``). You need a **prepost** node (they have internet access, more info [here](https://www.dkrz.de/up/systems/mistral/running-jobs/partitions-and-limits)). Choose profile ``5GB memory, prepost``.
> NOTE: Everytime you run the notebook you will use some of that RAM, we recomend to click on ``Kernel -> Shutdown kernel`` often so the memory is released. If you want to run several notebooks at the same time or one notebook several times and you cannot shoutdown the kernel each time, please, choose a job profile with a larger memory.
4. Press "start" and your Jupyter server will start (which it is also known as spawning).
5. Open a terminal in Jupyter (``New -> Terminal``, on the right side)
6. A terminal window opens on the node where your Jupyter is running.
8. Go back to your Jupyter and open a notebook from the notebooks folder:
```
tutorials-and-use-cases/notebooks/
```
9. Make sure you use the Jupyter ``Python 3 unstable`` kernel (``Kernel -> Change Kernel``).
3. Select a *preset* spawner Option.
4. Choose *job profile* which matches your processing requirements. We recommend to use at least 10GB memory. Find info about the partitons [here](https://docs.dkrz.de/doc/levante/running-jobs/partitions-and-limits.html) or note the *mouse hoover*. Specify an account (the luv account which your user belongs to, e.g. bk1088).
5. Press "start" and your Jupyter server will start (which it is also known as spawning). The server will run for the specified time in which you can always come back to the server (i.e. reopen the web-url) and continue to work.
6. In the upper bar, click on ``Git -> Clone a Repository``
7. In the alert window, type in ``https://gitlab.dkrz.de/data-infrastructure-services/tutorials-and-use-cases.git``. When it is successfull, a new folder appears in the data browser which is the cloned repo.
8. In the data browser, change the directory to ``tutorials-and-use-cases/notebooks`` and browse and open a notebook from this folder.
9. Make sure you use a recent ``Python 3`` kernel (``Kernel -> Change Kernel``).
### Advanced
...
...
@@ -61,10 +53,6 @@ $ bash make_kernel.sh
* notebooks/demo/tutorial_*
1. We prepared a tutorial on how to use [Intake](https://intake.readthedocs.io/en/latest/) in the DKRZ data pool. [](https://nbviewer.jupyter.org/urls/gitlab.dkrz.de/data-infrastructure-services/tutorials-and-use-cases/-/raw/master/notebooks/demo/tutorial_intake.ipynb)
2. ESMVal-Tool
### Use-cases
* notebooks/demo/use-case_*
...
...
@@ -72,14 +60,10 @@ $ bash make_kernel.sh
## Further Infos
* Find more in the DKRZ Jupyterhub [documentation](https://jupyterhub.gitlab-pages.dkrz.de/jupyterhub-docs/index.html).
**prepost* nodes at DKRZ have internet access [info](https://www.dkrz.de/up/systems/mistral/running-jobs/partitions-and-limits).
*``Python 3 unstable`` kernel: This kernel already contains all the common geoscience packages that we need for our notebooks.
* Find more in the DKRZ Jupyterhub [documentation](https://docs.dkrz.de/doc/software%26services/jupyterhub/index.html).
* See in this [video](https://youtu.be/f0wZX9i0uWQ) the main features of the DKRZ Jupterhub and how to use it.
* Advanced users developing their own notebooks can find there how to create their own environments that are visible as kernels by the Jupyterhub.
Besides the information on the Jupyterhub, in these DKRZ [docs](https://www.dkrz.de/up/systems/mistral/programming/jupyter-notebook) you can find how to run Jupyter notebooks directly in the DKRZ server, that is, out of the Jupyterhub (it entails that you install the geoscience packages you need).
## Exercises
In this hands-on we will find, analyze, and visualize data from our DKRZ data pool. The goal is to create two maps, one showing the number of tropical nights for 2014 (the most recent year of the historical dataset) and another one showing a chosen year in the past. The hands-on will be split into two exercises:
@@ -16,7 +16,7 @@ Welcome to Tutorials and Use Cases's!
This `Gitlab repository <https://gitlab.dkrz.de/data-infrastructure-services/tutorials-and-use-cases/>`_ collects and prepares Jupyter notebooks with coding examples on how to use state-of-the-art *processing tools* on *big data* collections. The Jupyter notebooks highlight the optimal usage of *High-Performance Computing resources* and adress data analysists and researchers which begin to work with resources of German Climate Computing Center `DKRZ <https://www.dkrz.de/>`_.
The Jupyter notebooks are meant to run in the `Jupyterhub portal <https://jupyterhub.dkrz.de/>`_. See in this `video <https://youtu.be/f0wZX9i0uWQ>`_ the main features of the DKRZ Jupterhub/lab and how to use it. Clone this repositroy into your home directory at the DKRZ supercomputers Levante and Mistral. The contents will be visible from the Jupyterhub portal. When you open a notebook in the Jupyterhub, make sure you choose the Python 3 unstable kernel on the Kernel tab (upper tool bar in the Jupyterhub). This kernel contains most of the common geoscience packages in current versions.
The Jupyter notebooks are meant to run in the `Jupyterhub portal <https://jupyterhub.dkrz.de/>`_. See in this `video <https://youtu.be/f0wZX9i0uWQ>`_ the main features of the DKRZ Jupterhub/lab and how to use it. Clone this repositroy into your home directory at the DKRZ supercomputers Levante and Mistral. The contents will be visible from the Jupyterhub portal. When you open a notebook in the Jupyterhub, make sure you choose a *recent* Python 3 kernel on the Kernel tab (upper tool bar in the Jupyterhub). Such a kernel contains most of the common geoscience packages in current versions.
Direct and fast access to DKRZ's data pools is a main benefit of the `server-side <https://en.wikipedia.org/wiki/Server-side>`_ data-near computing demonstrated here. Note that running the notebooks on your local computer will generally require much memory and processing resources.
