How to Connect to a Custom AI Model

This article explains how the custom AI connection expects inputs, outputs, and authentication. Custom AI connections allow you to connect to AI models that are hosted outside of Azure by following the guidelines in this article.

Configuration options in Templafy

When establishing the Custom connection in Templafy, the following information is expected:

1. Endpoint - The request will be a POST request directly to the endpoint configured in the AI connections endpoint property.
2. Input token limit - This is only used check the input limit when using a prompt template and will not be passed to the API.
3. Output token limit - This is only used check the output limit and will not be passed to the API.
4. Include user email address - Selecting this option means Templafy will include the email address of the user using the prompt template. The email address of the user will be sent in an emailAddress property.
5. Connect with Microsoft Entra ID - It is possible to authenticate with either an API key or Entra ID.
6. Request headers - It's possible to configure extra headers that will be sent as part of the request. These are configured with name and value, and when the request is sent to the API, these will be included as headers in the request.

Authentication

When the option to Connect with Microsoft Entra ID is enabled, Entra ID authentication will be used. When that option is not enabled, and API key will be required to authenticate.

API Key

With API key, there will be a header added to the request with the following values:

"[Authorization header name]": "[Authentication scheme] [API key]"

• Authorization header name - Header name for the authorization header. The default is "Authorization" but it can be set to a custom value.
• Authentication scheme - The scheme is used for authentication. This is prepended to the authorization header value. The default is “Bearer” but it can be changed or left blank if needed.
• API key - The value of the authorization header. This is appended to the authentication scheme, separated by a space.

Entra ID

With Entra ID Authentication, Templafy will get an access token for a configured service principal with the scope: “https://management.azure.com/.default”. The following has to be configured:

• Tenant ID - The Azure Tenant ID for the Azure directory where the service principal is created.
• Client ID - The Client ID for the service principal.
• Secret - The value of a secret created for the service principal.

Input and Output

As input, the API will receive instructions, message, form data, and optionally email address of the user. As output, the API should return a simple JSON object with just an output property. This is illustrated below in JSON:

{
  "instructions": "<instructions configured in the action>",
  "message": "<the message combined with the user input configured in the action>",
  "formData": {
      "<Form field reference name>": "<Value inputted by the user>"
  }
  "emailAddress": "<email address of the user, if set to include>"
}

{
  "output": "{The output that will be shown directly to the user}"
}

Example request

This is an example cURL request which is similar to the request the Custom Connection send to the API.

If the API to be used by the Custom connection can support this cURL request, it should be able to be configured inside of Templafy.

curl -X POST "<Endpoint>"
 -H "<Authorization header name>: <Authentication scheme> <API Key>"
 -H "Content-Type: application/json"
 -H "<Configured header name 1>: <Header value 1>"
 -H "<Configured header name 2>: <Header value 2>"
 -d ‘{
      "instructions": "<Instruction from Action>",
      "message": "<Message combined with user input>",
      "formData": {
                   "<Form field reference name>": "<form field data from user>"
                  }
      "emailAddress": "<email address of the user triggering the prompt"
     }’

Error handling

Templafy will handle the following errors:

• Error code 429: The user will be notified that the service is busy, and they can try again.
• Error code 400 with an error message containing "context_length_exceeded": The user will be notified that the input text is too long.
• Error code 400 with an error message containing "content_filter": The user will be notified that the input text contains inappropriate content.