Configure Reranker Model

In a RAG system, a reranker is a model that re-ranks the retrieved documents based on the user’s query. The reranker model is used to improve the relevance of the retrieved documents.

Configure Reranker Model

After logging in with an admin account, you can configure the Reranker Model in the admin panel.

1. Click on the Models > Reranker Models tab;
2. Click on the New Reranker Model button to add a new Reranker;

3. Input your Reranker information and click Create Reranker button;

Supported Reranker Providers

Currently AutoFlow supports the following reranker providers:

JinaAI

To learn more about JinaAI reranking, please visit Jina AI Reranker.

Cohere

To learn more about Cohere reranking, please visit Cohere Rerank.

vLLM

To use vLLM rerankers, you need to provide the base_url of the reranker API as the following JSON format in Advanced Settings:

{
    "base_url": "{api_base_url}"
}

Default config:

{
    "base_url": "http://localhost:8000"
}

To learn more about vLLM reranking, please visit vLLM Sentence Pair Scoring Models.

Xorbits Inference (Xinference)

To use Xinference rerankers, you need to provide the base_url of the reranker API as the following JSON format in Advanced Settings:

{
    "base_url": "{api_base_url}"
}

Default config:

{
    "base_url": "http://localhost:9997"
}

To learn more about Xinference reranking, please visit Xinference Rerank.

Amazon Bedrock

To use Amazon Bedrock rerankers, you’ll need to provide a JSON Object of your AWS Credentials, as described in the AWS CLI config global settings:

{
    "aws_access_key_id": "****",
    "aws_secret_access_key": "****",
    "aws_region_name": "us-west-2"
}

To find more reranking models supported by Amazon Bedrock, please visit Amazon Bedrock Models Reference.