Tooling & CI

lectures/tooling-ci/slides.qmd

@@ -3,3 +3,252 @@ title: "Tooling and Continuous Integration"
 subtitle: ""
 author: "Lukas Kluft, Tobias Kölling, (Flo)"
 ---
+
+# *Software Development* Tooling
+
+## Software to create software 😵
+* Validate code
+* Test functionality
+* Speedup development
+* Make things easier
+
+## Making things easier 
+:::: {.columns}
+
+::: {.column width="50%"}
+Python
+```python
+def say_hi():
+    print("Hello World")
+
+
+```
+ 
+ 
+:::{.justify}
+Python **interpreter** is implemented in C, which is **compiled** into Machine Code.
+
+This abstraction makes the process of developing a program, **easier** and **faster**.
+:::
+:::
+::: {.column width="50%"}
+Assembly
+```asm
+  0           RESUME                   0
+
+  1           LOAD_CONST               0 (<code object say_hi at 0x5f19d84931b0, file "example.py", line 1>)
+              MAKE_FUNCTION
+              STORE_NAME               0 (say_hi)
+              RETURN_CONST             1 (None)
+
+Disassembly of <code object say_hi at 0x5f19d84931b0, file "example.py", line 1>:
+  1           RESUME                   0
+
+  2           LOAD_GLOBAL              1 (print + NULL)
+              LOAD_CONST               1 ('Hello world')
+              CALL                     1
+              POP_TOP
+              RETURN_CONST             0 (None)
+```
+:::
+:::
+
+
+## Are programming languages tools?🤷
+
+* Compiler/Interpreter is a tool
+	- `/usr/bin/python3` follows the [Python Language Reference](https://docs.python.org/3/reference/index.html)
+* Libraries / modules (often) come with tools
+	- `pdb` - python debugger
+	- `pip` - famous package installer
+
+## What about the Operating System?
+
+It's a software that manages running programs. So yes!
+
+It often supplies users with software development tools:
+
+* [🪟](https://developer.microsoft.com/en-us/windows/dev-tools/) `dev-tools`
+* [🍎](https://developer.apple.com/xcode/) `xcode`
+* [🐧](https://itsfoss.com/build-essential-ubuntu/) `build-essential`
+
+
+## Tools in the development environment
+ 
+- Package managers (`conda`, `pyenv`, `uv`)
+- Version control systems (`git`)
+- Editors, IDEs - Integrate Development Environments
+- Linters, formatters, type-checkers (`flake8`, `black`, `ruff`)
+- Compilers and/or Interpreters (e.g. `gcc`, `python`)
+- Testing tools (`pytest`,[`unittest`](https://docs.python.org/3/library/unittest.html))
+	- AI helpers!? (`copilot`)
+
+## Why!?
+
+* Short cuts & Automation! 🤖
+* Increased efficiency
+* Quality assurance ✅->✅->❌
+* Improve collaboration
+
+
+## Static checks
+
+* All languages follow a syntax.
+* Easy for us to make mistakes/typos
+* Hard to understand where the problem is
+
+::: {.columns}
+
+::: {.column}
+```python
+def helloworld():
+    print('Hello',end='')
+     print('world')
+
+helloworld()
+ ```
+:::
+
+:::{.column}
+```
+$ python3 helloworld.py 
+  File "/home/reis/dir/helloworld.py", line 3
+    print('world')
+                  ^
+IndentationError: unindent does not match any outer indentation level
+```
+:::
+
+:::
+
+## Linters & formatters
+
+::: {.columns}
+
+::: {.column}
+Popular linters
+```
+$ flake8 helloworld.py
+helloworld.py:3:5: E999 IndentationError: unexpected indent
+```
+```
+$ pylint helloworld.py
+************* Module helloworld
+helloworld.py:3:4: E0001: Parsing failed: 'unexpected indent (helloworld, line 3)' (syntax-error)
+```
+
+:::
+
+::: {.column}
+Formatters
+```
+$ black --check  helloworld.py
+error: cannot format helloworld.py: Cannot parse: 3:0:     print('world')
+
+Oh no! 💥 💔 💥
+1 file would fail to reformat
+```
+```
+$ autopep8 helloworld.py
+def helloworld():
+    print('Hello', end='')
+    print('world')
+
+helloworld()
+```
+
+:::
+
+:::
+
+```
+$ ruff check --output-format=concise helloworld.py
+helloworld.py:3:1: SyntaxError: Unexpected indentation
+helloworld.py:5:1: SyntaxError: Expected a statement
+Found 2 errors.
+```
+
+## Hands-on Session {.handson}
+
+`git precommit hook` exercise?
+- It could be linting .gitlab.yaml
+- 
+- Apply provided faulty patch, check if pushing code  
+
+
+[//]:... The idea is that when introducing a change to a code-base, it should be checked **automatically**. 
+# Continuous Integration (CI)
+ * Automated builds and tests with each code change
+ * Rapid feedback on build and test status
+ * Consistent, reproducible testing environment
+ * Frequent merging to reduce integration conflicts
+ * Improved collaboration across development teams
+
+## Where?
+
+* Git repositoris are the **perfect place for CI**!
+* The CI assesses if a commit is up to the expectations ("Passes/Fails the CI").
+* Provided on popular git hosting systems
+	- [GitLab CI](https://docs.gitlab.com/ee/ci/)
+	- [GitHub Actions](https://docs.github.com/en/actions/about-github-actions/understanding-github-actions).
+
+
+## What to expect?
+
+ * Syntax errors should be immediately spotted.
+ * Building/compiling the latest changes/commit.
+   - [`cpython`](https://github.com/python/cpython/actions) latest linting check: ![](https://github.com/python/cpython/actions/workflows/lint.yml/badge.svg?branch=main&event=push)
+ * Tests must run and assess functionality
+   - [`cpython`](https://github.com/python/cpython/actions) latest tests ![](https://github.com/python/cpython/actions/workflows/build.yml/badge.svg?branch=main&event=push)
+
+
+## How!?
+
+* Define stages/steps in order of dependency
+* For each prepare the environment and tools to be used
+	- One can have many variants of the same stage
+* Tests to be done in each of them (to be continued...)
+
+#
+[//]::(https://gitlab.dkrz.de/generic-software-skills/lecture-materials/-/pipelines/96666)
+[//]::(https://github.com/faster-cpython/cpython/actions/runs/13454935981)
+
+## Example
+
+:::: {.columns}
+
+::: {.column width="50%"}
+```yml
+stages:
+ - lint
+ - build
+ - test
+
+linter:
+  stage: lint
+  image: python # container with basic python tools
+  script:
+    - pip install flake8
+    - flake8 . --count --exit-zero --statistics
+```
+:::
+::: {.column width="50%"}
+```yml
+type-check:
+	stage: build
+	image: python
+	script:
+		- pip install mypy
+		- mypy .
+```
+:::
+::::
+
+## Hands-on Session {.handson}
+
+* Create a gitlab CI pipeline for [repo](https://gitlab.dkrz.de/generic-software-skills/ci-example) that:
+1. Install tools to lints your changes
+2. Makes sure environment has required dedependecies 
+3. Runs the program (no testing yet)
+4. Bonus: creates an executable?
+

scripts/md2ical.py

@@ -23,8 +23,8 @@ def extract_lecture_dates(markdown_file, output_file):
     cal.add("REFRESH-INTERVAL;VALUE=DURATION", "P1D")
 
     soup = BeautifulSoup(html_content, "html.parser")
-    for table_entry in soup.findAll("tr")[1:]:
-        date, title, lecturers = [e.contents[0] for e in table_entry.findAll("td")]
+    for table_entry in soup.find_all("tr")[1:]:
+        date, title, lecturers = [e.contents[0] for e in table_entry.find_all("td")]
 
         if lecturers == "-" or table_entry.find(class_="inactive"):
             # Table entries without lecturer are considered to be breaks/holidays.