API templating

API Templating

GeWoVa supports arbitrary AI models through API templating. Overriding defaults is also implemented through API templating.

Desired state

Use case 1: Custom API

Use case 2: Chain of templates

In the setup below, the templating workflow is as follows:

• first, run the default openai template,
• second, patch the endpoint to connect to another openai-compatible service,
• third, patch the request to set generation parameters.

Step 1: Get the template code

To learn how to write templates, visit the “templating for API” project. As a quick help, below are examples of patching templates.

openai-proxy template to set the target URL:

local parent = std.parseJson(std.extVar('parent'));

parent + {
  url: 'http://localhost:8080/open-ai-proxy/chat/completions'
}

high-temperature template to patch the request body:

local parent = std.parseJson(std.extVar('parent'));

parent + {
  body+: {
    n: 3,
    temperature: 0.8,
  }
}

Further links:

• OpenAI API reference lists parameters for the “chat” endpoint
• GeWoVa template for OpenAI

Step 2: Put code into GeWoVa

A template in GeWoVa is a normal document with conventions:

• The name is apis/<name>/template
• The “request” template is
  • a paragraph with the word “Request”, followed by
  • a code block
• The “document” template is:
  • a paragraph with the word “Document”, followed by
  • a code block

Note: a code block is created by placing three backticks (```) at the beginning of a line and then pressing the <space> button.

Here is a legend to the screenshot above:

1. The template name is high-temperature, so the document name is apis/high-temperature/template
2. This is a patch-template to add the parameter temperature to the query
3. This is a patch-template to transform an OpenAI response (already transformed by the default template) to a GeWoVa document. The sample code is an identity transformation that does nothing. The block (3) can be deleted from the template.
4. The debug-flag

Step 3: Attach a template to a profile

There is a difference between a template and a profile:

• “profile” is what you select in the top bar of the application
• “profile” is a collection of several things, such as an API template and API secrets
• “template” is a named code
• One template can be used in multiple profiles

Attaching a template to a profile should be obvious. Below is a screenshot showing the main control buttons.

Troubleshooting

Add a paragraph with the text TPLFA_DEBUG to the body of a template. GeWoVa will print the transformation details to the web console.

If this doesn’t help, debug the templates with tools from the “templating for API” and jsonnet projects.