HF TGI LLM Connector

The Text Generation Inference is a Rust, Python, and gRPC server specifically designed for text generation inference. It can be self-hosted to power LLM APIs and inference widgets.

This node can connect to locally or remotely hosted TGI servers which includes Text Generation Inference Endpoints of popular text generation models that are deployed via Hugging Face Hub.

For more details and information about integrating with the Hugging Face TextGen Inference and setting up a local server, refer to the LangChain documentation .

Output ports

1. Type: org.knime.python3.nodes.PythonBinaryBlobFileStorePortObject
   Huggingface TGI Configuration

   Connection to an LLM hosted on a TGI server.

Hugging Face Hub Connection (Dynamic Inport)

An optional Hugging Face hub connection that can be used to access protected Hugging Face inference endpoints.

1. Type: org.knime.python3.nodes.PythonBinaryBlobFileStorePortObject