LLM - Large Language Model

In this app, LLM is used for several purposes:

1. Extracting knowledge from docs;
2. Generating responses to user queries.

Configure LLM

After logging in with an admin account, you can configure the LLM in the admin panel.

1. Click on the Models > LLMs tab;

2. Click on the New LLM button to add a new LLM;

   

3. Input your LLM information and click Create LLM button;

4. Done!

Supported LLM providers

Currently Autoflow supports the following LLM providers:

OpenAI

To learn more about OpenAI, please visit OpenAI.

Google Gemini

To learn more about Google Gemini, please visit Google Gemini.

Anthropic Vertex AI

To learn more about Anthropic Vertex AI, please visit Anthropic Vertex AI

Amazon Bedrock

To use Amazon Bedrock, you’ll need to provide a JSON Object of your AWS Credentials, as described in the AWS CLI config global settings:

{
    "aws_access_key_id": "****",
    "aws_secret_access_key": "****",
    "aws_region_name": "us-west-2"
}

To learn more about Amazon Bedrock, please visit Amazon Bedrock.

Gitee AI

To learn more about Gitee AI, please visit Gitee AI.

OpenAI-Like

Autoflow also support the providers that conform to the OpenAI API specification.

To use OpenAI-Like LLM providers, you need to provide the api_base of the LLM API as the following JSON format in Advanced Settings:

{
    "api_base": "{api_base_url}"
}

OpenRouter

Default config:

{
   "api_base": "https://openrouter.ai/api/v1/"
}

To learn more about OpenRouter, please visit OpenRouter.

ZhipuAI BigModel

Default config:

{
    "api_base": "https://open.bigmodel.cn/api/paas/v4/",
    "is_chat_model": true
}

To learn more about BigModel, please visit BigModel.

Ollama

Default config:

{
    "api_base": "http://localhost:11434"
}

To learn more about Ollama, please visit Ollama.

vLLM

Default config:

{
    "api_base": "http://localhost:8000/v1/"
}

To learn more about vLLM, please visit vLLM.

Xinference

Default config:

{
    "api_base": "http://localhost:9997/v1/"
}

To learn more about Xinference, please visit Xinference.