Checking Whether LLMs Meet FineChatBI Capability Requirements

FineChatBI requires large language models (LLMs) to strictly output results in the specified format. Otherwise, FineChatBI cannot be used properly. This document is intended for users to check whether the local LLM to be connected meets the requirements.

Sending Requests

Ensure that the FineAI service network is accessible to LLM services.

On the server where FineAI is located, use the curl command to send requests to the LLM service. (The following example is based on OpenAI Chat API format.) During actual usage, modify the following parameters according to your specific configuration:

• IP/PORT/URL: Replace them with actual service addresses. The URL must end with /chat/completions. Otherwise, it will not be compliant with the OpenAI API.

• MODEL_NAME: Replace it with the actual model name.

• API_KEY: If no authentication is required, delete the line -H "Authorization: Bearer API_KEY".

• max_tokens: Set it to a value larger than 8192. If the context length of the model is less than 8192, FineChatBI will be incompatible.

curl http://IP:PORT/URL/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer API_KEY" \
-d '{
    "model": "MODEL_NAME",
    "messages": [
        {"role": "system", "content": "You are a synonym generation assistant. I will give you some table names and corresponding field names. You can generate synonyms or abbreviations for the table names or field names based on reasonable associations and inferences of the table names and field names. \n```\nThe format of the field names you generate must meet the following rules:\n1.You can only return JSON code. No markdown code symbol is allowed.\n2.You need to generate five synonyms for each field name or table name. Synonyms can be some abbreviations. For example, the synonyms of \"Store Information by Dimension\" can be Store Info and Store.\n3.The synonyms of field names can be one combined with table names. For example, if the table name is \"Store\" and the field name is \"Name\", the synonyms can be \"Store Name\" and \"Shop Name\".\n4.You cannot generate the same synonym for fields in the same table.\n5.Field names mixed in Chinese and English must remain consistent with the input.\n6.You must return the data in the following JSON format: \n{\n  \"table_aliases_dic\":\n    [\n      {\n        \"table_name\":  Table Name,\n        \"aliases\": Array of table name synonyms,\n        \"fields\": [\n          {\n            \"field_name\": Field Name,\n            \"aliases\": Array of field name synonyms\n          }\n        ]\n      }\n    ]\n}\n7.The returned JSON data must be \"table_aliases_dic\" rather than \"tableAliasesDic\".\n```\nExample:\nThe following tables and fields exist:\nTable Name: Company Table\nField: ID, Name, Abbreviation, Section\nTable Name: Subject Table\nField: Name, Abbreviation\nTable Name: Budget Detail Table\nField: Date, Budget\n\nQ：Please generate synonyms or abbreviations for table names and fields.\nA：\n{ \"table_aliases_dic\":[\n  {\n    \"table_name\": \"Company Table\",\n    \"aliases\": [\"Company\", \"Enterprise\", \"Organization\", \"Institution\", \"Agency\"],\n    \"fields\": [\n      {\"field_name\": \"ID\", \"aliases\": [\"UUID\", \"Identifier\", \"Company ID\", \"Company Code\", \"Company Number\"]},\n      {\"field_name\": \"Name\", \"aliases\": [\"Name\", \"Company Name\", \"Organization Name\", \"Enterprise Name\", \"Title\"]},\n      {\"field_name\": \"Abbreviation\", \"aliases\": [\"Company Abbreviation\", \"Short Company Name\", \"Company Acronym\", \"Acronym\", \"Short Form\"]},\n      {\"field_name\": \"Section\", \"aliases\": [\"Block\", \"Business Section\", \"Partition\", \"Chunk\", \"Segment\"]}\n    ]\n  },\n  {\n    \"table_name\": \"Subject Table\",\n    \"aliases\": [\"Domain\", \"Category\", \"Type\", \"Subject\", \"Branch\"],\n    \"fields\": [\n      {\"field_name\": \"Name\", \"aliases\": [\"Alias\", \"Subject Name\", \"Category Name\", \"Project Name\"，\"Ttitle\"]},\n      {\"field_name\": \"Abbreviation\", \"aliases\": [\"Acronym\", \"Short Form\", \"Field Short Form\", \"Initialism\", \"Short Name\"]}\n    ]\n  },\n  {\n    \"table_name\": \"Budget Detail Table\",\n    \"aliases\": [\"Budget Detail\", \"Budget\", \"Estimation Detail\", \"Detail\", \"Budget Planning Detail\"],\n    \"fields\": [\n      {\"field_name\": \"Date\", \"aliases\": [\"Time\", \"Detailed Date\", \"Year-Month-Day\", \"Budget Time\"，\"Specific Time\"]},\n      {\"field_name\": \"Budget\", \"aliases\": [\"Target\", \"Task\", \"Amount\", \"Detail\", \"Revenue\"]}\n    ]\n  },\n]\n}\nExample End\n```"},
        {"role": "user","content": "The following tables and fields exist:\n{\"tables\":[{\"tableName\":\"Product Detail\",\"tableFields\":[{\"fieldName\":\"Product Code\"}]}]}\nPlease help me generate synonyms."}
    ],
    "max_tokens": 8192,
    "stream": false
}'

Format Requirement of Returned Content

The example format of the returned content is as follows: (This example is based on the OpenAI Chat API. The actual format of the returned content may vary depending on the API.)

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "LLM-generated content"
      },
      "finish_reason": "stop",
    }
  ],
  ……    // Other parameters, for example, ID, can be ignored.
}

(1) Check whether the content meets the format requirements.

The choices.message.content in the response body is output by the LLM. If the request is sent as described in section "Sending Requests", the returned content must meet the following format requirements.

Note:

The content returned by some LLMs (such as DeepSeek-R1 and QwQ) may include additional thinking processes, but still meet the following basic format requirements.

{

  "table_aliases_dic": [
    {
      "table_name": "Omitted",
      "aliases": ["Omitted"],
      "fields": [
        {
          "field_name": "Omitted",
          "aliases": ["Omitted"]
        }
      ]
    }
  ]
}

(2) Execute the curl command (as described in section "Sending Requests") five times consecutively. If the LLM can stably return the content in the correct format each time, the LLM is deemed as meeting the FineChatBI requirements.

(1) Execute the curl command (as described in section "Sending Requests"). The following figure shows the sent command and response body.

The content in the response body is as follows:

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n  \"table_aliases_dic\": [\n    {\n      \"table_name\": \"Product Detail\",\n      \"aliases\": [\"Product Info\", \"Product Feature\", \"Product List\", \"Product Data\", \"Product Record\"],\n      \"fields\": [\n        {\n          \"field_name\": \"Product Code\",\n          \"aliases\": [\"Product ID\", \"Code\", \"Product Number\", \"Product Identifier\", \"Goods Code\"]\n        }\n      ]\n    }\n  ]\n}"
      },
      "finish_reason": "stop",
    }
  ]
,  ……    // Other parameters, for example, ID, can be ignored.}
}

(2) Since the choices.message.content in the response body is output by the LLM, parse the content value to obtain the following content:

Compare the content format here with that described in section "Format Requirement of Returned Content" and check whether the content part fully complies with the format.

{


  "table_aliases_dic": [    { 
     "table_name": "Product Detail",
      "aliases": ["Product Info", "Product Feature", "Product List", "Product Data", "Product Record"]
,      "fields": [
        {
          "field_name": "Product Code",
          "aliases": ["Product ID", "Code", "Product Number", "Product Identifier", "Goods Code"]
        }
      ]
    }
  ]
}

(3) Perform continuous five tests.

Execute the curl command (as described in section "Sending Requests") five times consecutively. If the LLM can stably return the content in the correct format each time, the LLM is deemed as meeting the FineChatBI requirements.