Skip to content
Snippets Groups Projects

Update "command line" lecture

Merged Lukas Kluft requested to merge command-line into main
All threads resolved!
1 file
+ 25
40
Compare changes
  • Side-by-side
  • Inline
---
title: "The command line"
subtitle: "Controlling local and remote machines"
author: "Lukas Kluft and Florian Ziemen"
---
# Shell
* A shell exposes an operating system to a human user or other programs
* On most Linux systems the default shell is _Bash_ (_PowerShell_ on Windows; _Zsh_ on MacOs)
* On most Linux systems, the default shell is _Bash_ (_PowerShell_ on Windows; _Zsh_ on MacOs)
* Different shells come with different syntax and characteristics
## Built-in commands
@@ -32,7 +31,7 @@ author: "Lukas Kluft and Florian Ziemen"
```{.sh}
vim test.txt
```
* Basic usage: hit {{< kbd i >}} to activate _insert_ mode. Type something and hit {{< kbd Esc >}} to fall back into _normal_ mode. Save and close the file using {{< kbd :wq >}}.
* Basic `vim` usage: hit {{< kbd i >}} to activate _insert_ mode. Type something and hit {{< kbd Esc >}} to fall back into _normal_ mode. Save and close the file using {{< kbd :wq >}}.
## Scripts
@@ -53,13 +52,13 @@ author: "Lukas Kluft and Florian Ziemen"
## Take home messages
* The command line is a text interface to interact with a computer
* The command line is a text interface for interacting with a computer
* Built-in commands and core utilities are available on (almost) all machines
* Sequences of commands can be stored and executed as shell scripts
# Hands-on session {.handson}
1. Open the command line (_Terminal_)
1. Open the command line (_Terminal_^[More difficult for Windows users])
1. Create a shell script using an editor of your choice
1. Run the script and check the output
@@ -68,16 +67,7 @@ author: "Lukas Kluft and Florian Ziemen"
## Configuration
::: {.r-stack}
::: {.fragment .fade-out fragment-index=1}
Supercomputers consist of various components
:::
::: {.fragment .fade-in fragment-index=1}
High-performance computers consist of various components
:::
:::
. . .
:::: {.columns}
@@ -99,7 +89,8 @@ High-performance computers consist of various components
Different ways to work on a cluster depending on the task:
* Housekeeping/compiling (login nodes)
* Interactive sessions/scripts on a compute node (`salloc`/`sbatch`)
* Interactive sessions (compute nodes, `salloc`)
* Batch scripts (compute nodes, `sbatch`)
* External services (e.g. _Jupyterhub_)
## SSH
@@ -108,8 +99,7 @@ Different ways to work on a cluster depending on the task:
```{raw}
ssh <YOUR_USERNAME>@levante.dkrz.de
```
(use your user id instead of a123456)
* Now you have access to the command line on levante
* Now you have access to the command line **on levante**
## Public keys
@@ -117,12 +107,12 @@ Different ways to work on a cluster depending on the task:
## Generating SSH keys
* Generate a private / public ssh key pair for authentication on your local system
* Generate a private / public SSH key pair on your local system
```bash
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_levante
```
* Upload the public (`.pub`) key to your [DKRZ profile](https://luv.dkrz.de) ([instructions](https://docs.dkrz.de/doc/levante/access-and-environment.html)), so levante can check your connection attempts
* Use your private ssh key instead of your password when connecting to levante
* Upload the public (`.pub`) key to your DKRZ profile ([how to](https://docs.dkrz.de/doc/levante/access-and-environment.html#public-key-authentication))
* Use your private SSH key to authenticate when connecting to levante
```bash
ssh -i ~/.ssh/id_ed25519_levante <YOUR_USERNAME>@levante.dkrz.de
```
@@ -132,8 +122,8 @@ Different ways to work on a cluster depending on the task:
* Create a config file for convenience
```{.sshconfig filename=~/.ssh/config}
Host levante # the name you use in the shell
Hostname levante.dkrz.de # the official name of the system
User <YOUR_USERNAME> # add your user id
Hostname levante.dkrz.de # real host name of the system
User <YOUR_USERNAME> # your user id
IdentityFile ~/.ssh/id_ed25519_levante # your private key
```
(call `man ssh_config` for more info)
@@ -147,23 +137,16 @@ Different ways to work on a cluster depending on the task:
## Remote file transfer {auto-animate=true}
* There are different tools to transfer between your local machine and a remote server
* For single files, a simple `scp` is sufficient
```{raw}
scp <YOUR_USERNAME>@levante.dkrz.de:file_in_home .
```
* `rsync` is a more powerful alternative
* There are various tools to transfer data between your local machine and a remote server (e.g. [`scp`](https://linux.die.net/man/1/scp), [`rsync`](https://linux.die.net/man/1/rsync), [`uftp`](https://uftp-multicast.sourceforge.net))
* `rsync` is a powerful option available on most machines
```{raw}
rsync <YOUR_USERNAME>@levante.dkrz.de:file_in_home .
```
## Remote file transfer {auto-animate=true visibility="uncounted"}
* There are different tools to transfer between your local machine and a remote server
* For single files, a simple `scp` is sufficient
```{raw}
scp levante:file_in_home .
```
* `rsync` is a more powerful alternative with more options
* There are various tools to transfer data between your local machine and a remote server (e.g. [`scp`](https://linux.die.net/man/1/scp), [`rsync`](https://linux.die.net/man/1/rsync), [`uftp`](https://uftp-multicast.sourceforge.net))
* `rsync` is a powerful option available on most machines
```{raw}
rsync levante:file_in_home .
```
@@ -175,9 +158,11 @@ Different ways to work on a cluster depending on the task:
|keep scripts | store output | temporary stuff |
| small | big | big |
| SSD | HDD | HDD |
|backuped | no backup | deleted after 2 weeks |
|backup | no backup | deleted after 2 weeks |
We all lost data to `/scratch`, many killed something on `/work`
::: {.fragment .fade-up}
> Anything not saved will be lost --- _Nintendo_
:::
## Compute nodes
@@ -191,7 +176,7 @@ We all lost data to `/scratch`, many killed something on `/work`
## Take home messages
* High-performance computers like Levante are technically a cluster of numerous _normal_ computers that share the same hard disks
* Using a secure shell (`ssh`) one can login to Levante and use it as any other machine
* Using a secure shell (`ssh`) you can log into Levante and (mostly) use it like any other machine
* Computationally demanding or long-lasting jobs have to be submitted to the job queue
# Hands-on session {.handson}
@@ -215,7 +200,7 @@ We all lost data to `/scratch`, many killed something on `/work`
## tmux
`tmux` enables to create and (re)attach to terminal sessions
`tmux` enables to create and attach to terminal sessions
* Crate a new session
```{raw}
@@ -229,10 +214,10 @@ We all lost data to `/scratch`, many killed something on `/work`
## X-forwarding
You can use X-forwarding to forward graphical user interfaces (GUIs) from the server
forward graphical user interfaces (GUIs) from the server
* Install an X-server on your local machine (e.g. XQuartz)
* Pass the `-X` option to your `ssh` command^[or check the SSH config option]
* Pass the `-X`^[Mac users might want to use `-Y`, or MacOS will quit the forwarding after about 20 min.] option to your `ssh` command^[or check the SSH config option]
```{raw}
ssh -X levante
```
Loading