This node can connect to locally or remotely hosted TGI servers which includes Text Generation Inference Endpoints of popular text generation models that are deployed via Hugging Face Hub.
Protected endpoints require a connection with a HF Hub Authenticator node in order to authenticate with Hugging Face Hub.
The Text Generation Inference is a Rust, Python, and gRPC server specifically designed for text generation inference. It can be self-hosted to power LLM APIs and inference widgets.
For more details and information about integrating with the Hugging Face TextGen Inference and setting up a local server, refer to the LangChain documentation .
Note : If you use the Credentials Configuration node and do not select the "Save password in configuration (weakly encrypted)" option for passing the API key via the HF Hub Authenticator node, the Credentials Configuration node will need to be reconfigured upon reopening the workflow, as the credentials flow variable was not saved and will therefore not be available to downstream nodes.