Introduction to Freva: general usage and plugin development

12 Jun 2024, 09:00 → 13 Jun 2024, 16:00 Europe/Berlin

Martin Bergemann, Mostafa Hadizadeh, Christopher Kadow, Etor Lucio Eceiza

Target group(s): anyone in the below mentioned projects (general usage), users with a plugin in mind to develop (plugin development).

Deadline for registration: May 28 23:59 CET

Freva, the free evaluation system framework, is a data search and analysis platform developed at DKRZ-CLINT group.  With help of Freva researchers can:

• quickly and intuitively search for data stored at typical data centers that host many datasets.
• create a common interface for user defined data analysis tools (plugins).
• apply data analysis tools in a reproducible manner.

 

There are currently several projects using different Freva instances running in Levante. These course(s) are (mainly) directed to the following 4 projects:

• Coming Decades (codes)
• GEMS-Warmworld (gems), tied to project 1414.
• RegIKlim-Nukleus (regiklim), tied to project ch1187.
• ClimXtreme (xces), tied to project bm1159.

 

We are offering 2 consecutive courses: 

• general usage: targets a broader audience interested on learning the basics of Freva, e.g. data search and plugin run. 12th of June, Wed 09.00-12.30
• plugin development: targets a smaller group of people interested on developing analysis tools for their respective projects. 13th of June, Thur 09.00-15.30

 

About the courses:

• The assistance to both courses is not compulsory (one can pick the first one, the second one or both).
• Participants are required to have a DKRZ account (link to create one), it is recomended that the users already belong to one of the aforementioned projects (link to request access, once you have a DKRZ account).
• The hands-on exercises will be run with our open freva instance (web) and at DKRZ's jupyterhub (rest).
• The current content of the course is orientative and subjected to (some) updates.

 

Workshop Material:

Repository with all the necessary information of the workshop: exercises, talks, video recordings, plugin.

Wednesday, 12 June

Day 1: Freva: general usage

Conveners: Christopher Kadow, Etor Lucio Eceiza, Martin Bergemann, Mostafa Hadizadeh

1 Introduction to Freva

A comprehensive overview on Freva:
• history and description of the tools
• overview on the main Freva functionalities: data-browser, plugin, history of runs.

2 Hands-on: Freva via web

First block of hands-on exercises via Freva web:
- overview of the web-page
- databrowser: learning to find different data through it
- plugin usage: exercise with a couple of tool
- history-results: checking on the runs we did previously, learning how to re-run, share different runs.
- result-browser: history result of everyone

Q&A

10:40 Coffee Break

3 Set up for 2nd and 3rd hands-on

Time to check and solve out problems with DKRZ account, jupyter hub access etc.

4 Hands-on: Freva via command line (option a, depending on poll results)

Second block of hands-on exercises via Freva command line (cli):
- overview of the cli functionalities
- databrowser: learning to find different data through it
- plugin usage: exercise with a couple of tool
- history-results: checking on the runs we did previously, learning how to re-run, share different runs.
- user-add: functionalities to add or link your own data.

Q&A

12:20 Lunch Break

5 Hands-on: Freva via python client (option b, depending on poll results)

Third block of hands-on exercises via Freva python client:
- overview of the python functionalities
- databrowser: learning to find different data through it
- plugin usage: exercise with a couple of tool
- history-results: checking on the runs we did previously, learning how to re-run, share different runs.
- user-add: functionalities to add or link your own data.

Q&A

6 closing remarks

Thursday, 13 June

Day 2

Conveners: Etor Lucio Eceiza, Martin Bergemann, Mostafa Hadizadeh

7 Introduction to a Freva plugin

• overall structure of a plugin
• dependencies of a plugin
• git & gitlab, where plugins are located
• typical interaction workflow to move from a tool to a freva-plugin
• licensing of a plugin
• how to connect to levante with e.g. vscode

Q&A

8 Hands-on 1: accessing gitlab and cloning a plugin

10:30 Coffee Break

9 Dissecting a Freva Plugin

• what is a Freva wrapper? Main components
• how to connect a tool to Freva
• Freva databrowser restAPI for non-python languages
• how to install all needed libraries of a tool
• the link2fordata utility
• how to plug a plugin to Freva
• extra: calling a plugin within a plugin with git submodules

Q&A

11:45 Lunch break

10 Hands-on 2: playing with a plugin

Interactive example with the previously cloned plugin:
• looking at the diff components
• testing how to connect via cli, web, python
• modifying, debugging

Q&A

11 Closing remarks