Integrating Your AI Gateway with Cato

Many organizations use AI gateways or similar routing layers to centralize how internal applications, assistants, and development tools connect to AI providers. Cato integrates with your AI gateway so AI application traffic can be forwarded to Cato for inspection, visibility, and policy enforcement.

Note: Currently, Cato supports integration with LiteLLM. Other AI gateway vendors, such as Kong, API-M, Portkey, and more, need to be coordinated with your Cato Networks representative.

By configuring your AI gateway to send AI application traffic to Cato, you can apply Cato AI Security controls to the traffic before requests reach the AI provider and, where supported, before responses are returned to users.

This approach helps organizations enforce consistent AI usage policies across applications that use the AI gateway, while continuing to use their existing AI infrastructure. Cato inspects the forwarded AI traffic according to the configured AI Security policy, helping detect and prevent sensitive data exposure, unsafe prompts, prompt injection attempts, and other non-compliant AI interactions.

Use Case

ABC Company uses an internal customer support AI agent to help support engineers summarize customer tickets, search internal KB articles, and generate suggested responses. The agent sends prompts and responses through LiteLLM before connecting to the AI provider.

Because the agent processes customer support data, the company needs to reduce the risk of exposing sensitive information, such as customer names, account details, access tokens, and troubleshooting logs. The security team is also concerned that a malicious or accidental prompt could instruct the agent to ignore company policies, reveal restricted information, or generate an unsafe response.

To inspect this traffic before it reaches the AI provider, the company integrates LiteLLM with Cato and creates an AI Security Guard of the type AI Gateway in the CMA. The Guard is configured to inspect prompts before they are sent to the AI provider and block requests that contain sensitive data or prompt injection attempts.

For example, if a support engineer asks the AI agent to summarize a ticket and the prompt includes an API token or customer credentials, Cato detects the sensitive data and blocks the request before it leaves the AI gateway. If a prompt attempts to bypass the agent’s instructions, Cato can also block the interaction based on the configured Guard policy. This helps the company continue using its support AI agent while enforcing consistent AI Security controls for sensitive support workflows.

Configuring AI Gateway Traffic Forwarding

Configuring AI gateway traffic forwarding includes two main steps:

Create the AI gateway configuration in Cato and add the generated snippet to your AI gateway.
Create an AI Security Guard of the type AI Gateway in Cato.

Create the AI Gateway Configuration in Cato

In Cato, create the AI gateway configuration that generates the information required for your existing AI gateway to forward AI application traffic to Cato.

Navigate to AI Security > AI Gateway and click New.
Enter the required information:
- Integration name – Provide a descriptive name that will help you identify this integration.
- AI Gateway service provider – Select the AI Gateway service provider. Currently, only LiteLLM is supported.
- Hosting location – Determine if the Guard you will create will be hosted on the Cato Cloud or a cloud-based Outpost.
Click Create.
Copy the generated configuration snippet and add it to your AI gateway configuration.

Example configuration:

guardrails:
  - guardrail_name: ai-sec
    litellm_params:
      guardrail: aim
      mode: [pre_call, post_call]
      api_key: <AI_GATEWAY_API_KEY>
      api_base: "https://api.aisec.catonetworks.com"
      default_on: true

In this configuration, LiteLLM sends AI interactions to Cato before and after the AI call. This allows Cato to inspect and enforce policy on AI traffic before the request is sent to the AI provider and after the response is returned.

API Key Rotation

Each AI gateway integration includes multiple API keys. Only one key is required for an API call, but multiple keys let you rotate credentials without interrupting traffic forwarding to Cato.

Cato recommends regenerating API keys regularly. To rotate keys without downtime, update your AI gateway to use one key while regenerating the other key. After the regenerated key is available, update the AI gateway configuration to use the new key, confirm that traffic continues to be forwarded to Cato, and then regenerate the previous key.

This approach helps maintain continuous AI traffic forwarding while supporting secure credential management.

To rotate API keys:

Navigate to AI Security > AI Gateway and click the AI Gateway entry that you want to edit.
In the AI Gateway API Keys section, click the rotate icon.

Create an AI Gateway Guard in Cato

After configuring your AI gateway to forward AI application traffic to Cato, create an AI Security Guard of the type AI Gateway.

Note: The Guard name must match exactly the name of the LiteLLM virtual key it should protect.