Articles in this section

Leveraging AI to Write Data Transformation or Explain Existing Transformation

Updated
Follow

About this article

This article explains how to use the AI-powered Data Transformation Builder to help write data transformations or explain what an existing transformation does.

Sections in this article:

Prerequisites

 
  • Admin access in Templafy tenant
  • Data Connector and Data Transformation modules enabled
  • External Data Source configured

 

About Templafy's AI-powered Data Transformation Builder

The AI-powered data transformation builder is an efficient and self-service way to create or understand data transformation configuration in plain English using AI, specifically Microsoft Azure OpenAI Services. It contains two elements of generative AI assistance: 

  1. Creating a JMESPath query expression for the transformation that is described in plain English based on the provided prompt and sample input data.
  2. Explaining an existing JMESPath query expression for data transformation in plain English.

In order to leverage AI in Templafy's data transformation feature, admins must first enable the AI-powered Data Transformation Builder, and by doing so, consent to external data processing outside of Templafy. The AI-powered Data Transformation Builder relies upon exchange of prompts and completions data between Templafy and Microsoft Azure OpenAI Services. For more information about security and data processing, see the Security and Data Privacy Information section.

 

How to Enable the AI-powered Data Transformation Builder

To use the AI widget in your data transformation setup, you must first enable it on your Templafy tenant.

  1. In your Templafy Admin Center, navigate to the Resources tab.
  2. On the top right corner of the page, click on More Options.
  3. Select Settings from the dropdown menu.



  4. Check the box next to Allow AI to be used for data transformation.



  5. Click Save, and close out of the settings pane.
 
  • By checking this box, the admin consents to external data processing outside of Templafy.
  • Consent for external data processing may be revoked at any time by unchecking the setting Allow AI to be used for data transformation. Doing so will disable the AI-powered Data Transformation Builder.


How to Use the AI-powered Data Transformation Builder

How to Generate Transformation Expressions

  1. In your Templafy Admin Center, navigate to the Resources tab.
  2. Go to the External Data Sources tab.
  3. Select the data source you’d like to transform.
  4. Click on the three dots menu and select Transform data.
  5. Expand the section labeled Generate with AI.
  6. Enter a prompt in plain English describing the desired transformation rule.
  7. Click Generate transformation.

    AIGenerate.gif

 
  • When using AI to generate a transformation, the newly generated transformation will replace any previous transformation.
  • To keep your existing transformation rules and make use of this AI tool, copy and save the transformation definition outside of Templafy.

 

How to Explain Transformation Expressions

  1. In your Templafy Admin Center, navigate to the Resources tab.
  2. Got to the External Data Sources tab.
  3. Select the data source you’d like to transform.
  4. Click on the three dots menu and select Transform data.
  5. Expand the section labeled Explain transformation.


    AIExplain.gif

 

 
  • Because of token limits, using Explain transformation on large transformation definitions will likely result in an error.
  • To explain specific parts of the transformation, copy and save the full text elsewhere and leave just the desired line(s) in the Transformation pane. 

 

How to Show or Hide the AI Section in the Transformation Screen

By default, when the AI-powered Data Transformation Builder is enabled, the Explain and Generate options will appear under the Transformation section. These options can be hidden if AI help is not needed.

To hide the AI sections:

  1. In the Transformation screen, click on More options.
  2. Click on Show AI section to uncheck it.

Once the AI section is hidden, it can be shown again when needed by repeating those steps:

  1. In the Transformation screen, click on More options.
  2. Click on Show AI section to enable.

 

Security and Data Privacy Information

  • By enabling and using the AI-powered Data Transformation Builder, the admin consents to external data processing outside of Templafy, specifically by Microsoft Azure OpenAI Services.
  • Prompts and completions data are exchanged between Templafy and Azure OpenAI Services only when the data transformation rules are being configured by by means of actively using the AI-powered widgets in the admin area of Templafy.
  • Only the following information is included in the data exchange:
    • Prompts typed by the admin user
    • Selected external data source schema definition
    • Selected data sample from the selected data source
    • The current data transformation rule
  • If any PII (Personal Identifiable Information) is included in any of the above information, that means that information will be processed by AI services external to Templafy.
  • Any information from the AI-powered Data Transformation Builder is not used for training of the external AI model, and is not shared with any other 3rd parties, fully regulated by terms and conditions of Microsoft Azure OpenAI Services
  • The exchanged data may be temporarily stored by the Azure OpenAI Service. It will be encrypted and only accessible to authorized Microsoft employees for limited purposes. More information can be found here.

 

 
  • There are some limitations in Azure OpenAI services that are outside of Templafy's control. These limitations include:
    • Prompt token limit
    • Large language model hallucinations
    • Inability to answer some questions correctly
    • Inability to process unethical requests 
  • It is a good idea to double-check any answers provided by AI

 

Related articles

 

 

Related articles

Comments

0 comments

Article is closed for comments.