Giskard RAGET Evaluator

This node allows to identify potential weaknesses and areas for improvement in a RAG system provided as a workflow by evaluating the correctness of the answers. An LLM is used to compare the workflow's answers to the reference answers of the test set. The test set can be generated with the Giskard RAGET Test Set Generator node.

For more details see the Giskard documentation .

Note : If you use the Credentials Configuration node and do not select the "Save password in configuration (weakly encrypted)" option for passing the API key for the LLM connector node, the Credentials Configuration node will need to be reconfigured upon reopening the workflow, as the credentials flow variable was not saved and will therefore not be available to downstream nodes.

Input ports

1. Type: org.knime.python3.nodes.PythonBinaryBlobFileStorePortObject
   LLM or Chat Model

   A large language model or chat model.

2. Type: Workflow Port Object
   RAG Workflow

   The prediction workflow to analyze with Giskard.

3. Type: Table
   Test Set

   A test set table generated by the Giskard RAGET Test Set Generator.

Output ports

1. Type: Table
   Giskard report data

   The Giskard report as table.