Troubleshooting AI Security For Apps

Overview

This playbook covers the full range of AI Security for Apps issues in CMA, including Guards not inspecting or actioning agent/LLM traffic, AI Gateway integrations not forwarding events, Managed Agent integrations failing to connect or sync, and Guards Policy rules not taking effect as expected.

The affected components span AI Security Guards (Proxy, API, and AI Gateway mode), Guards Policy rules and their Engine Profiles, AI Gateway integrations, Managed Agent Integrations and the underlying AI Security for Applications license. When any of these fail, customers lose visibility into agent/LLM traffic and managed agent activity, policy enforcement cannot occur, and governance and compliance objectives are not met.

Symptoms

• A Guard shows zero invocations in CMA even though the connected application is actively sending traffic
• A Guards Policy rule is configured but never matches, or matches inconsistently across similar requests
• A Block or Anonymize & Block rule is matched, but the request is only monitored, or no action is visibly applied
• An AI Gateway integration (Portkey, LiteLLM, or Kong) is configured, but traffic routed through the gateway never appears as inspected in CMA
• A Managed Agent integration shows as disconnected, fails Test Connection, or fails during its setup script/CloudFormation stack installation
• A Managed Agent integration shows as Active, but no agent configurations or agent sessions appear in CMA

Possible Causes

• AI Security for Apps license expired or not enabled
• The global Guards Policy Enabled toggle is switched off in CMA
• The Guard is created but not actually receiving traffic. The application, or gateway is not pointed at the Guard's endpoint (Proxy/API Mode) or the AI Gateway guardrail is not included in the gateway configuration.
• A Guards Interaction Policy rule is disabled, unpublished, or scoped to the wrong Engine Profile, Direction (User, Assistant, Tool Input, Tool Output), or Action
• A platform-specific setting required for Managed Agent visibility has not been enabled
• Expected data-refresh latency has not yet elapsed.
• The affected LLM provider, agent platform, or gateway type is not currently supported by the Guard or Managed Agent integration in question

Initial Assessment

Complete these checks before deeper troubleshooting. Most issues trace back to one of these prerequisites.

Validate AI Security License

Verify that the AI Security End Users license (this covers AI Security for Agents) is enabled and has not expired. Navigate to CMA > Account > Licenses > Premium Security and confirm the license is present, active, and within its validity period.

If the license is absent, disabled, or expired, AI Security for Apps features, including Guard creation, Guards Interaction Policy enforcement, AI Gateway integrations, and Managed Agent Integrations, will not function regardless of configuration. Resolve any licensing issues before continuing with the steps in this document.

Validate Guards Policy is Enabled

Navigate to CMA > AI Security > Guards Policy and ensure that the toggle Guards Policy Enabled is checked. This is a simple step that is easy to overlook, and no rule will take effect while it is off.

Validate Guard Deployment and Traffic Routing

A Guard only inspects traffic it actually receives. Before assuming a policy or detection problem, confirm the Guard is correctly in the traffic path.

1. Navigate to AI Security > Guards and confirm the affected Guard exists, and is configured with the correct Type (Proxy, API, or AI Gateway) for the deployment in question.
2. Generate a user prompt from the homegrown agent. The invocation count shown in CMA should increase with each prompt. If invocations do not increase, this points to a possible application misconfiguration.
   
3. Open the affected Guard and navigate to the Deployment tab. Depending on the Guard type, specific values from this tab must be present in the application's configuration:
   • For all Guard types, confirm the API key configured in the application matches an active, non-revoked key shown in CMA. You can regenerate the API key in CMA if needed, this invalidates the previous key, so update the application immediately after regenerating.
     
   • For Proxy and API Guards, confirm the application is pointed at the Guard's endpoint URL by editing the application’s configuration file (e.g., config.py) and adding GUARD_API_KEY, BASE_URL, MODEL_ID, and when the guard is proxy type and the LLM key was not provided in CMA, LLM_PROVIDER_API_KEY.
   • Ensure that all required request headers, especially the ones marked with an asterisk (*) symbol, are included in the application’s client code (e.g., client.py). For more information regarding Guard deployment, see Example Implementing Guards for a Chat Bot.
   • For an AI Gateway Guard, confirm the corresponding gateway integration (Portkey, LiteLLM, or Kong) is configured to route traffic through Cato. The Cato-issued API key and guardrail must be added to the gateway's own configuration (for example, config.yaml for LiteLLM).

Validate Managed Agent Integration Connection

Managed Agent Integrations are read-only connections with third-party agent platforms and use a different model than Guards. There is no traffic inspection, only inventory and session polling.

1. Navigate to AI Security > Integrations > Managed Agent Platform and confirm the relevant integration (Amazon Bedrock, Amazon Quick, Microsoft Foundry, Copilot Studio, or Gemini Enterprise Agent Platform) shows an Active status and the integration was fully configured.
2. For Amazon Bedrock or Amazon Quick, confirm the CloudFormation stack completed installation successfully in the correct AWS account (or, for a multi-account deployment, in the AWS Organizations management account) and that all documented AWS prerequisites are met.
3. For Microsoft Foundry or Copilot Studio, confirm every setup step was completed by the correct role holder and that documented prerequisites are met:
   • An Entra ID admin (Application Administrator, Cloud Application Administrator, or Global Administrator) granted tenant-wide consent to the Cato Networks AI Security Integration application via the Cato Networks admin consent link.
   • For Foundry, a subscription Owner or User Access Administrator assigned the Cognitive Services User role to the Cato application on every subscription containing Foundry resources.
   • For Copilot Studio, a Power Platform Administrator (or Global Administrator) registered the Cato application as a management application using the PowerShell registration script in Azure Cloud Shell, and enabled Agent Sessions for every relevant, non-developer environment.
   • A Cato admin entered the correct Microsoft Entra ID Tenant ID in the CMA and successfully clicked Test Connection before saving. A mismatched tenant ID is a frequent cause of Test Connection failures.
4. For Gemini Enterprise Agent Platform, confirm the Google Cloud setup script was run in the correct Google Cloud project (run gcloud config set project PROJECT_ID if needed), that the identity running the script had permission to enable APIs and create IAM/Workload Identity Federation resources, and that the onboarding token had not expired before the script ran. If the connection cannot be verified immediately after a successful script run, allow several minutes for Google Cloud IAM changes to propagate before retrying.
5. Confirm expected data-refresh timing before concluding an integration is broken:
   • Foundry and Copilot Studio fetch data immediately on connection and then hourly
   • Bedrock AgentCore activity depends on AWS delivering CloudWatch traces, which can take time after the first agent invocation.

Troubleshooting the Issue

Steps to troubleshoot the symptoms administrators or users may encounter are listed below. These steps are intended to identify possible causes for the issues faced.

Troubleshooting a Guard Showing No Invocations

This applies when the Guards page (AI Security > Guards) shows zero invocations in CMA.

1. Confirm the AI Security for Apps license is valid, and user prompts have been generated from the affected application.
2. For proxy guards, confirm that the AI provider is supported by Cato as documented in Working with AI Security Guards.
3. Navigate to Home > Events, filter for Event Type = AI Security and confirm whether any guard-related events are generated at all. If none are arriving, ensure that the guard has been deployed to the application’s configuration, as explained in Validate Guard Deployment and Traffic Routing.
   
4. Verify AI Security API reachability by running curl -v https://api.aisec.catonetworks.com via CLI. The output should show FQDN resolution, a successful TLS handshake, and an HTTP response (usually HTTP 404).
5. To confirm that a specific guard is reachable and responsive, you may run a curl command using the request example (include a valid guard key) from AI Security > Guards > Deployment. A successful response should include the matching guard policy rule and the action that would be applied.
   
6. If Guard is correctly configured on the application, the guard is reachable via curl and no CMA events are generated, escalate to Cato Support with the Guard type/configuration (Proxy, API, or AI Gateway), application details and configuration files (redact API keys)

Troubleshooting Guards Policy Rules not Taking Effect

This applies when the Guards page shows invocations, but the guards policy rules are not taking the configured action.

1. Navigate to AI Security > Guards Policy, confirm Guards Policy Enabled is checked and the affected rule is published.
2. Confirm the Engine Profile attached to the rule actually includes the detector expected to catch the content in question. You can use the Engine Playground to validate that the prompt content triggers the detector within the engine profile before troubleshooting further.
3. Confirm the rule's Direction matches where the content actually appears. A detector aimed at Tool Output will not evaluate a User-direction prompt, and vice versa.
4. Confirm the Action configured on the rule is the action being expected. A rule set to Anonymize & Block or Anonymize and Monitor will apply an action based on the type of content detected within the prompt, as explained in Unexpected Action Applied by the Policy.
5. If multiple rules could apply to the same event, confirm which rule actually fired by reviewing the event detail. Rules are evaluated independently and are not order-dependent. When more than one rule matches, the stricter action applies. An event actioned less strictly than expected usually means the stricter rule's scope, Engine Profile, or Direction did not actually match that event.
6. If the rule is enabled, correctly scoped, and the content should match, but the expected action is still not applied, escalate to Cato Support with the event ID, a screenshot of the rule (Engine Profile, Direction, Action), and the Guard or integration through which the traffic was received.

Troubleshooting Missing Agent Configurations or Sessions

This applies when the expected Managed Agent configurations or agent sessions are missing or incomplete in CMA.

1. Confirm the integration itself shows as Active, see Validate Managed Agent Integration Connection before troubleshooting missing data within it.
2. Allow for the platform's normal refresh cadence: at least one hourly cycle for Microsoft Foundry and Copilot Studio, and up to the time needed for AWS to generate and deliver CloudWatch traces for Bedrock AgentCore.
3. Validate that Cato supports agent sessions visibility in CMA for the affected managed application, as documented in Working with Managed Agents.
4. For Amazon Bedrock AgentCore, confirm CloudWatch Transaction Search is enabled (AWS Console > Application Signals (APM) > Transaction search, with span ingestion as structured logs). Without this, Cato can show AgentCore agents in inventory but never their activity.
5. For Copilot Studio, confirm Agent Sessions is enabled for the specific environment the missing agent belongs to (Power Platform admin center > Manage > Environments > [environment] > Settings > Product > Features > Copilot Studio agents), and confirm the environment is not a developer environment, which is unsupported for this setting.
6. For Microsoft Foundry, confirm the specific subscription that contains the missing Foundry resource has the Cognitive Services User role assigned to the Cato application. Role assignment does not apply tenant-wide and must be repeated per subscription.
7. For Gemini Enterprise Agent Platform, confirm expectations: only agent sessions and session events are shown; tool, integration, and external-service usage within a session is not collected by design.
8. If the relevant refresh interval has elapsed and the platform-specific tracing/role/session settings are all confirmed correct, escalate to Cato Support with the integration type, the specific subscription/environment/project affected, and the expected versus actual data observed.

Raising Cases to Cato Support

Before raising a case with Cato Support, ensure that all relevant troubleshooting steps outlined in this playbook have been completed. If the issue remains unresolved, escalate to Cato Support with the following information:

• Issue Description — A clear and concise description of the observed symptom and the expected behavior
• Affected Component — Guard name and type (Proxy, API, or AI Gateway), AI Gateway integration (Portkey, LiteLLM, or Kong), or Managed Agent integration (Amazon Bedrock, Amazon Quick, Microsoft Foundry, Copilot Studio, or Gemini Enterprise Agent Platform)
• Environment Details — For Guards: application details and configuration files (redact API keys). For Managed Agent integrations: confirm that the documented prerequisites are met and provide any configuration errors observed.
• Connection/Policy Details — Guards Policy rule name(s), Engine Profile, Direction, and Action; or the specific integration setup step last completed and its result
• Troubleshooting Steps Completed — A summary of the steps already taken from this playbook and the outcome of each
• Supporting Evidence — links to AI security events, screenshots of the rule or integration configuration, CloudFormation stack status or Test Connection error messages, and exact timestamps (UTC and customer time zone)

The more context and evidence provided upfront, the faster Cato Support will be able to investigate and resolve the issue.