Description

DASF Logo

Data Analytics Software Framework

python module wrapper for the data analytics software framework DASF

Abstract

DASF: Messaging Python is part of the Data Analytics Software Framework (DASF, https://codebase.helmholtz.cloud/dasf), developed at the GFZ German Research Centre for Geosciences (https://www.gfz-potsdam.de). It is funded by the Initiative and Networking Fund of the Helmholtz Association through the Digital Earth project (https://www.digitalearth-hgf.de/).

DASF: Messaging Python is a RPC (remote procedure call) wrapper library for the python programming language. As part of the data analytics software framework DASF, it implements the DASF RPC messaging protocol. This message broker based RPC implementation supports the integration of algorithms and methods implemented in python in a distributed environment. It utilizes pydantic (https://pydantic-docs.helpmanual.io/) for data and model validation using python type annotations. DASF distributes messages via a central, django-based message broker: dasf-broker-django. (see also: Message Broker)

Documentation

see: https://dasf.readthedocs.io/en/latest/

Installation

Install this package in a dedicated python environment via

python -m venv venv
source venv/bin/activate
pip install dasf

Optional features come as extras: backend (the generation of code), broker (the development message broker that the runserver command starts), openai (conversations with an LLM) and mcp (MCP servers). Install them in brackets, e.g. pip install "dasf[backend,broker]".

To use this in a development setup, clone the source code from gitlab, start the development server and make your changes::

git clone https://codebase.helmholtz.cloud/dasf/dasf-messaging-python
cd dasf-messaging-python
python -m venv venv
source venv/bin/activate
make dev-install

The quickstart shows how to set up a backend module and the development message broker.

Technical note

This package has been generated from the template https://codebase.helmholtz.cloud/hcdc/software-templates/python-package-template.git.

See the template repository for instructions on how to update the skeleton for this package.

Source Code Examples

The examples directory of the source code has an example backend module (backend_module.py), the client stub that has been generated from it, an MCP server, and a script that calls the module via the client stub (client.py). Its README describes how to run them, and how to generate the client stub.

Recommended Software Citation

Eggert et al., (2022). DASF: A data analytics software framework for distributed environments. Journal of Open Source Software, 7(78), 4052, https://doi.org/10.21105/joss.04052

License information

Code files in this repository are licensed under the Apache-2.0, if not stated otherwise in the file.

Documentation files in this repository are licensed under CC-BY-4.0, if not stated otherwise in the file.

Supplementary and configuration files in this repository are licensed under CC0-1.0, if not stated otherwise in the file.

Please check the header of the individual files for more detailed information.

License management

License management is handled with reuse. If you have any questions on this, please have a look into the contributing guide or contact the maintainers of dasf-messaging-python.