Development setup#

Create a virtual environment, activate it and install required packages:

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,test,docs]"

# enable basic style checks once
pre-commit install

Important

Make sure to activate the virtual environment in every new shell session:

source .venv/bin/activate

If you want to automate this install direnv and allow it once via direnv allow (see .envrc configuration file).

Windows-specifics

On Windows the syntax for virtual environment activation is a bit different:

# The following may need to be run once. Please check the docs for its consequences:
# https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policiess
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser

# Activate via:
.venv\Scripts\Activate.ps1

CLI scripts can now be simply run:

msh2vtu --help

Using make for shortcuts!

Development-related tasks can also be done with make (requires a Bash shell with make). The above development setup can also be generated with:

make setup

To get all available make-targets run make help:

help                                                               Show this help
setup                                                              Setup a virtual environment and install all development dependencies
setup_headless                                                     Install vtk-osmesa and gmsh without X11 dependencies
setup_devcontainer                                                 Internal usage
test                                                               Runs the unit tests
coverage                                                           Runs the unit tests generating code coverage reports
check                                                              Runs various checks with pre-commit
clean                                                              Cleans up temporary files
docs                                                               Builds the documentation
cleandocs                                                          Cleans up temporary documentation files
preview                                                            Runs an auto-updating web server for the documentation
	conda init zsh                                                   
	conda create --prefix /tmp/ogstools-test-env-py312  python=3.12  
	conda activate /tmp/ogstools-test-env-py312

Testing with pytest#

Tests are executed via pytest (shortcut: make test):

pytest [--capture=tee-sys]

Test coverage#

The following commands run the tests and create a test coverage report (shortcut: make coverage):

coverage run -m pytest
coverage combine
coverage report --no-skip-covered
coverage html
...
TOTAL                                                                  1698    292    83%
coverage html
Wrote HTML report to htmlcov/index.html

You can view a test coverage report by opening htmlcov/index.html in a browser.

Build documentation#

For generating the documentation we use Sphinx (shortcut: make docs):

cd docs
make html

This will create the documentation files in docs/_build/html.

You can use an auto-generating and -reloading web server (Linux / macOS only) for developing the documentation (shortcut: make preview):

make html -C docs # Generate docs once
python docs/server.py
# Open http://127.0.0.1:5500 in a web browser
# ...
# You can stop the server in the terminal with CTRL-D

Galleries#

Python files in docs/examples will be added to the Examples-gallery based on sphinx-gallery. Please note that text blocks are written reStructuredText-format. The examples can be downloaded from the final website as Jupyter notebook files.

You can interactively run and debug these files in Visual Studio Code, see the Python Interactive window documentation.

If you want to link to a gallery page from another page use the following syntax (prefix with sphx_glr_, replace directory separator with _):

{ref}`meshlib example <sphx_glr_auto_examples_howto_meshlib_plot_meshlib_pyvista_input.py>`

Guidelines for examples#

In order to maintain consistency in style and structure between different examples, please follow those recommendations when creating a new one:

  • All imports and global settings must be done in first python block.

  • No heading for import and settings cell.

  • Examples that cover multiple steps and/or datasets must contain section.

  • Sections have to be give a title, indicated by highest level heading.

  • Section titles cannot start with ordinal number or letter (no 1./I./A. Example section title)

Further information#

For syntax references and extension usage see the following links:

Run checks#

We use pre-commit to run various checks (shortcut make check):

pre-commit run --all-files

Create a package#

pyproject-build

Packages can then be found in dist/.

Development in a container with VSCode#

A full-featured (including e.g. FEFLOW-functionality), ready-to-use development environment can be used via VSCode’s Dev Container feature. You can do all development-related tasks with it, e.g. testing, previewing documentation or even debugging.

Requirements#

How-to#

  • Open the ogstools-project in VSCode

  • Click the blue button in the left-bottom corner

  • Click on Reopen in Container

Now you are inside the container. For example, you can open a new terminal (Terminal / New Terminal) and then run some tests with pytest or use the Testing-sidebar to select specific tests to debug.

Container specification#

.devcontainer/devcontainer.json (see also available settings):

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/python
{
  "name": "ogstools with FEFLOW",
  "build": {
    "dockerfile": "Dockerfile"
  },
  // Features to add to the dev container. More info: https://containers.dev/features.
  "features": {
    "ghcr.io/devcontainers/features/git-lfs:1": {}
  },
  // Use 'forwardPorts' to make a list of ports inside the container available locally.
  // "forwardPorts": [],
  "postCreateCommand": "make setup_devcontainer",
  "customizations": {
    "vscode": {
      "extensions": [
        "ms-python.python",
        "ms-python.black-formatter",
        "ms-toolsai.jupyter"
      ],
      "settings": {
        "python.defaultInterpreterPath": ".venv-devcontainer/bin/python",
        "python.terminal.activateEnvInCurrentTerminal": true,
        "python.formatting.provider": "none",
        "[python]": {
          "editor.defaultFormatter": "ms-python.black-formatter",
          "editor.formatOnSave": true
        },
        "python.testing.pytestArgs": [
          "."
        ],
        "python.testing.unittestEnabled": false,
        "python.testing.pytestEnabled": true
      }
    }
  },
}

.devcontainer/Dockerfile:

FROM mcr.microsoft.com/devcontainers/python:3.10-bookworm

# FEFLOW, instructions from https://download.feflow.com/download/FEFLOW/linux/
ENV \
    # Adapt when changing base image:
    DIST_DIR=ub2204x64
RUN apt-get update \
    && apt-get install -yq --no-install-recommends gpg curl ca-certificates
# Error: The following packages have unmet dependencies:
#   feflow-console81 : Depends: aspnetcore-runtime-6.0 but it is not installable
# Add dot net repos for debian 12 (bookworm base image)
RUN curl -fsSL https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -o packages-microsoft-prod.deb \
    && dpkg -i packages-microsoft-prod.deb && rm packages-microsoft-prod.deb
RUN curl -fsSL https://download.feflow.com/download/FEFLOW/linux/dhi-feflow-key.asc | gpg --dearmor | sudo tee /usr/share/keyrings/feflow.gpg > /dev/null \
    && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/feflow.gpg] https://download.feflow.com/download/FEFLOW/linux/apt-repo/$DIST_DIR stable main" | sudo tee /etc/apt/sources.list.d/feflow.stable.list \
    && apt-get update \
    && apt list | grep feflow  \
    && apt-get install -yq --no-install-recommends feflow-ifm-devel feflow-python81 \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

ENV \
    PYTHONPATH=/opt/feflow/8.1/python \
    LD_LIBRARY_PATH=/opt/feflow/8.1/lib64:/opt/feflow/common/qt/lib64:/opt/feflow/common/lib64 \
    FEFLOW81_ROOT=/opt/feflow/8.1

Container usage without VSCode#

Advanced topic

If you are familiar with Docker, you can also start the container manually, e.g. with:

docker run --rm -it -v $PWD:$PWD -w $PWD registry.opengeosys.org/ogs/tools/ogstools/devcontainer-3.10 /bin/bash
# Inside the container:
make setup_devcontainer
pytest

Please also be aware of file permission issues when mounting your working directory.

To prevent these issues we recommend running via Apptainer:

apptainer shell docker://registry.opengeosys.org/ogs/tools/ogstools/devcontainer-3.10
# Inside the container:
make setup_devcontainer
pytest

Release procedure#

  • Make sure there is a complete changelog at docs/releases and added to the corresponding index.md.

  • Create a tag.

  • Wait for the tag pipeline to complete. Copy the output of the pages-job to a new directory in the ogs/tools/ogstools-docs-repo.

On this page
Show Source