diff --git a/lectures/tooling-ci/slides.qmd b/lectures/tooling-ci/slides.qmd index 7f32d23773dfbc327bc062e8ceab83a42fb787b1..e334be6ddb8620b7a2801488750e02045ff5dc9e 100644 --- a/lectures/tooling-ci/slides.qmd +++ b/lectures/tooling-ci/slides.qmd @@ -3,3 +3,252 @@ title: "Tooling and Continuous Integration" subtitle: "" author: "Lukas Kluft, Tobias Kölling, (Flo)" --- + +# *Software Development* Tooling + +## Software to create software 😵 +* Validate code +* Test functionality +* Speedup development +* Make things easier + +## Making things easier +:::: {.columns} + +::: {.column width="50%"} +Python +```python +def say_hi(): + print("Hello World") + + +``` + + +:::{.justify} +Python **interpreter** is implemented in C, which is **compiled** into Machine Code. + +This abstraction makes the process of developing a program, **easier** and **faster**. +::: +::: +::: {.column width="50%"} +Assembly +```asm + 0 RESUME 0 + + 1 LOAD_CONST 0 (<code object say_hi at 0x5f19d84931b0, file "example.py", line 1>) + MAKE_FUNCTION + STORE_NAME 0 (say_hi) + RETURN_CONST 1 (None) + +Disassembly of <code object say_hi at 0x5f19d84931b0, file "example.py", line 1>: + 1 RESUME 0 + + 2 LOAD_GLOBAL 1 (print + NULL) + LOAD_CONST 1 ('Hello world') + CALL 1 + POP_TOP + RETURN_CONST 0 (None) +``` +::: +::: + + +## Are programming languages tools?🤷 + +* Compiler/Interpreter is a tool + - `/usr/bin/python3` follows the [Python Language Reference](https://docs.python.org/3/reference/index.html) +* Libraries / modules (often) come with tools + - `pdb` - python debugger + - `pip` - famous package installer + +## What about the Operating System? + +It's a software that manages running programs. So yes! + +It often supplies users with software development tools: + +* [🪟](https://developer.microsoft.com/en-us/windows/dev-tools/) `dev-tools` +* [ðŸŽ](https://developer.apple.com/xcode/) `xcode` +* [ðŸ§](https://itsfoss.com/build-essential-ubuntu/) `build-essential` + + +## Tools in the development environment + +- Package managers (`conda`, `pyenv`, `uv`) +- Version control systems (`git`) +- Editors, IDEs - Integrate Development Environments +- Linters, formatters, type-checkers (`flake8`, `black`, `ruff`) +- Compilers and/or Interpreters (e.g. `gcc`, `python`) +- Testing tools (`pytest`,[`unittest`](https://docs.python.org/3/library/unittest.html)) + - AI helpers!? (`copilot`) + +## Why!? + +* Short cuts & Automation! 🤖 +* Increased efficiency +* Quality assurance ✅->✅->⌠+* Improve collaboration + + +## Static checks + +* All languages follow a syntax. +* Easy for us to make mistakes/typos +* Hard to understand where the problem is + +::: {.columns} + +::: {.column} +```python +def helloworld(): + print('Hello',end='') + print('world') + +helloworld() + ``` +::: + +:::{.column} +``` +$ python3 helloworld.py + File "/home/reis/dir/helloworld.py", line 3 + print('world') + ^ +IndentationError: unindent does not match any outer indentation level +``` +::: + +::: + +## Linters & formatters + +::: {.columns} + +::: {.column} +Popular linters +``` +$ flake8 helloworld.py +helloworld.py:3:5: E999 IndentationError: unexpected indent +``` +``` +$ pylint helloworld.py +************* Module helloworld +helloworld.py:3:4: E0001: Parsing failed: 'unexpected indent (helloworld, line 3)' (syntax-error) +``` + +::: + +::: {.column} +Formatters +``` +$ black --check helloworld.py +error: cannot format helloworld.py: Cannot parse: 3:0: print('world') + +Oh no! 💥 💔 💥 +1 file would fail to reformat +``` +``` +$ autopep8 helloworld.py +def helloworld(): + print('Hello', end='') + print('world') + +helloworld() +``` + +::: + +::: + +``` +$ ruff check --output-format=concise helloworld.py +helloworld.py:3:1: SyntaxError: Unexpected indentation +helloworld.py:5:1: SyntaxError: Expected a statement +Found 2 errors. +``` + +## Hands-on Session {.handson} + +`git precommit hook` exercise? +- It could be linting .gitlab.yaml +- +- Apply provided faulty patch, check if pushing code + + +[//]:... The idea is that when introducing a change to a code-base, it should be checked **automatically**. +# Continuous Integration (CI) + * Automated builds and tests with each code change + * Rapid feedback on build and test status + * Consistent, reproducible testing environment + * Frequent merging to reduce integration conflicts + * Improved collaboration across development teams + +## Where? + +* Git repositoris are the **perfect place for CI**! +* The CI assesses if a commit is up to the expectations ("Passes/Fails the CI"). +* Provided on popular git hosting systems + - [GitLab CI](https://docs.gitlab.com/ee/ci/) + - [GitHub Actions](https://docs.github.com/en/actions/about-github-actions/understanding-github-actions). + + +## What to expect? + + * Syntax errors should be immediately spotted. + * Building/compiling the latest changes/commit. + - [`cpython`](https://github.com/python/cpython/actions) latest linting check:  + * Tests must run and assess functionality + - [`cpython`](https://github.com/python/cpython/actions) latest tests  + + +## How!? + +* Define stages/steps in order of dependency +* For each prepare the environment and tools to be used + - One can have many variants of the same stage +* Tests to be done in each of them (to be continued...) + +# +[//]::(https://gitlab.dkrz.de/generic-software-skills/lecture-materials/-/pipelines/96666) +[//]::(https://github.com/faster-cpython/cpython/actions/runs/13454935981) + +## Example + +:::: {.columns} + +::: {.column width="50%"} +```yml +stages: + - lint + - build + - test + +linter: + stage: lint + image: python # container with basic python tools + script: + - pip install flake8 + - flake8 . --count --exit-zero --statistics +``` +::: +::: {.column width="50%"} +```yml +type-check: + stage: build + image: python + script: + - pip install mypy + - mypy . +``` +::: +:::: + +## Hands-on Session {.handson} + +* Create a gitlab CI pipeline for [repo](https://gitlab.dkrz.de/generic-software-skills/ci-example) that: +1. Install tools to lints your changes +2. Makes sure environment has required dedependecies +3. Runs the program (no testing yet) +4. Bonus: creates an executable? + diff --git a/scripts/md2ical.py b/scripts/md2ical.py index 41fa5c84e242777a7f489a7aa0e41306bf894bed..cebfaac4c44c2aad545d01d6f78d89a1a0749c6e 100755 --- a/scripts/md2ical.py +++ b/scripts/md2ical.py @@ -23,8 +23,8 @@ def extract_lecture_dates(markdown_file, output_file): cal.add("REFRESH-INTERVAL;VALUE=DURATION", "P1D") soup = BeautifulSoup(html_content, "html.parser") - for table_entry in soup.findAll("tr")[1:]: - date, title, lecturers = [e.contents[0] for e in table_entry.findAll("td")] + for table_entry in soup.find_all("tr")[1:]: + date, title, lecturers = [e.contents[0] for e in table_entry.find_all("td")] if lecturers == "-" or table_entry.find(class_="inactive"): # Table entries without lecturer are considered to be breaks/holidays.