Bring Your Own LLM (BYOLLM)

Catalio’s Bring Your Own LLM (BYOLLM) capability allows organizations to connect their own AI provider accounts. This gives you full control over model selection, usage costs, and data flow while leveraging Catalio’s AI-powered features.

Overview

With BYOLLM, you can:

• Use your existing AI provider accounts - No need for separate Catalio AI billing
• Choose specific models - Select the exact models for chat and embeddings
• Control data flow - Data goes directly to your provider, not through Catalio
• Manage costs - Pay your provider directly with your existing agreements
• Ensure compliance - Use providers that meet your regulatory requirements

Configuration Steps

Step 1: Access LLM Provider Settings

1. Log in to Catalio as an organization administrator
2. Navigate to Settings from the main menu
3. Select LLM Providers from the settings menu

You’ll see a list of your configured providers (if any) and options to add new ones.

Step 2: Add a New Provider

1. Click Add Provider or Configure New Provider
2. Select your provider type from the dropdown:
   • OpenAI
   • Anthropic
   • Azure OpenAI
   • Google Gemini
   • Groq
   • xAI (Grok)
   • Ollama
   • OpenRouter
   • GitHub Copilot

Step 3: Enter Configuration Details

Fill in the required fields for your chosen provider:

Basic Configuration

Display Name (Optional)

A friendly name to identify this configuration (e.g., “Production OpenAI” or “Dev Anthropic”)

API Key (Required)

Your provider’s API key. See provider-specific formats below.

Endpoint URL (Required for Azure OpenAI and Ollama)

The base URL for your provider’s API. Only required for providers that need custom endpoints.

Model Selection

Chat Model

Select the model to use for chat, analysis, and other text generation features.

Embedding Model (If supported)

Select the model to use for semantic search and vector embeddings. Only available for providers that support embeddings (OpenAI, Azure OpenAI, Gemini, Ollama, OpenRouter).

Additional Options

Active Status

Toggle whether this configuration is active and available for use.

Expiration Date (Optional)

Set an optional expiration date for the API key. Catalio will warn administrators as the date approaches.

Step 4: Validate Connection

Before saving, click Validate Connection to verify your configuration:

1. Catalio tests connectivity to the provider
2. Confirms the API key is valid
3. Lists available models
4. Reports any errors

A successful validation shows the model count and marks the provider as validated.

Step 5: Assign to Features

After adding providers, assign them to specific AI features:

1. Return to the LLM Providers settings page

2. Use the Feature Assignments dropdowns:

   • AI Insights Provider: For requirement analysis, quality assessment, summarization
   • AI Chat Provider: For the AI chat assistant
   • Embeddings Provider: For semantic search (must support embeddings)

3. Select the configured provider for each feature

4. Changes take effect immediately

API Key Formats

Each provider uses a specific API key format:

API Key Security

Catalio takes API key security seriously:

Encryption at Rest

All API keys are encrypted using AES-256-GCM encryption before storage. The encryption key is managed separately from the database, ensuring that:

• Database access alone cannot reveal API keys
• Keys are decrypted only when needed for API calls
• Encryption follows industry best practices

Never Logged

API keys are never written to logs, telemetry, or error reports:

• Sensitive fields are automatically sanitized
• Error messages redact credential information
• Audit trails track actions without exposing keys

Secure Transmission

When Catalio calls your AI provider:

• All requests use HTTPS/TLS encryption
• API keys are sent only in authorization headers
• No keys are included in URLs or query parameters

Access Control

Only organization administrators can:

• View provider configurations
• Add or modify API keys
• Change feature assignments
• Delete provider configurations

Regular users can use AI features but cannot access configuration details.

Provider-Specific Configuration

OpenAI

Provider Type: OpenAI
API Key: sk-proj-XXXXXXXXXX
Endpoint URL: (leave blank for default)
Chat Model: gpt-4o
Embedding Model: text-embedding-3-small

Getting your API key:

1. Go to platform.openai.com
2. Navigate to API Keys in your account settings
3. Create a new secret key
4. Copy immediately (shown only once)

Anthropic

Provider Type: Anthropic
API Key: sk-ant-api03-XXXXXXXXXX
Endpoint URL: (leave blank for default)
Chat Model: claude-sonnet-4-5-20250929
Embedding Model: (not supported)

Azure OpenAI

Provider Type: Azure OpenAI
API Key: [Your Azure key]
Endpoint URL: https://your-resource.openai.azure.com
Chat Model: [Your deployment name, e.g., gpt-4o-deployment]
Embedding Model: [Your embedding deployment name]

Important: Azure OpenAI requires:

1. An Azure OpenAI resource in your Azure subscription
2. Model deployments created in Azure OpenAI Studio
3. The exact deployment names (not model names) in Catalio

Google Gemini

Provider Type: Google Gemini
API Key: AIzaSy...
Endpoint URL: (leave blank for default)
Chat Model: gemini-2.0-pro
Embedding Model: gemini-embedding-001

Groq

Provider Type: Groq
API Key: gsk_XXXXXXXXXX
Endpoint URL: (leave blank for default)
Chat Model: llama3-70b-8192
Embedding Model: (not supported)

xAI (Grok)

Provider Type: xAI
API Key: xai-XXXXXXXXXX
Endpoint URL: (leave blank for default)
Chat Model: grok-2-1212
Embedding Model: (not supported)

Ollama (Self-Hosted)

Provider Type: Ollama
API Key: (leave blank for local Ollama)
Endpoint URL: http://localhost:11434 (or your server address)
Chat Model: llama3
Embedding Model: nomic-embed-text

Note: For remote Ollama servers, ensure the endpoint is accessible from Catalio’s servers and configure any necessary authentication.

OpenRouter

Provider Type: OpenRouter
API Key: sk-or-v1-XXXXXXXXXX
Endpoint URL: (leave blank for default)
Chat Model: openai/gpt-4o
Embedding Model: openai/text-embedding-3-small

Note: OpenRouter model names include the provider prefix (e.g., openai/gpt-4o, anthropic/claude-3-sonnet).

GitHub Copilot

Provider Type: GitHub Copilot
API Key: [GitHub Copilot Enterprise token]
Endpoint URL: (leave blank for default)
Chat Model: gpt-4o
Embedding Model: (not supported)

Requirements:

• GitHub Enterprise Cloud subscription
• GitHub Copilot Enterprise license
• Token with appropriate permissions

Managing Configurations

Viewing Configuration Status

The LLM Providers page shows each configuration’s status:

• Active: Configuration is enabled and available
• Validation Status: Valid, Invalid, or Unknown
• Last Validated: When the connection was last tested
• Cache Status: Age of cached model list

Refreshing Model Lists

Provider model lists are cached for 24 hours. To manually refresh:

1. Find the provider configuration
2. Click Refresh Models
3. Wait for the refresh to complete

This is useful when providers add new models or you want to ensure the latest options.

Editing Configurations

1. Click the Edit button on the configuration card
2. Modify settings as needed
3. Optionally update the API key (leave blank to keep existing)
4. Click Save

Note: Changing the API key requires re-validation.

Deleting Configurations

1. Click the Delete button on the configuration card
2. Confirm the deletion in the modal

Best Practices

Use Descriptive Display Names

Instead of relying on provider type alone, use clear display names:

• Good: “Production OpenAI - GPT-4o”
• Good: “Development Anthropic”
• Good: “Azure OpenAI (HIPAA Compliant)”

Set API Key Expiration

For security best practices:

1. Create API keys with limited lifetimes at your provider
2. Set matching expiration dates in Catalio
3. Document key rotation procedures

Test Before Production

Before using a provider for production workloads:

1. Configure in a test organization first
2. Validate connection and model access
3. Test each AI feature manually
4. Verify costs and response quality
5. Then replicate configuration in production

Monitor Usage

Check your provider’s usage dashboard regularly:

• Track token consumption
• Monitor costs
• Identify unusual usage patterns
• Ensure you’re within rate limits

Configure Multiple Providers

For redundancy and optimization:

1. Configure a primary provider (e.g., OpenAI for chat)
2. Configure a secondary provider (e.g., Anthropic for long documents)
3. Assign different providers to different features based on strengths

Troubleshooting

“Invalid API Key” Error

• Verify the key was copied correctly (no extra spaces)
• Check that the key hasn’t been revoked at the provider
• Ensure the key has permissions for the models you’re trying to use
• For Azure: verify the key matches the endpoint URL

“Model Not Found” Error

• For Azure OpenAI: use deployment names, not model names
• For OpenRouter: include the provider prefix (e.g., openai/gpt-4o)
• Verify the model is available in your provider account
• Refresh the model list and check available options

“Endpoint URL Required” Error

Azure OpenAI and Ollama require endpoint URLs:

• Azure: https://your-resource.openai.azure.com
• Ollama: http://localhost:11434 or your server address

Connection Timeout

• Check network connectivity to the provider
• Verify firewall allows outbound HTTPS to provider endpoints
• For Ollama: ensure the server is running and accessible
• Try increasing timeout in advanced settings

Embeddings Provider Not Available

If the “Embeddings Provider” dropdown shows no options:

• Verify you have at least one provider that supports embeddings
• Supported: OpenAI, Azure OpenAI, Gemini, Ollama, OpenRouter
• Not supported: Anthropic, Groq, xAI, GitHub Copilot

Data Flow with BYOLLM

When you use BYOLLM, data flows directly between Catalio and your provider:

User Request
    |
    v
Catalio Application
    |
    | (HTTPS + API Key)
    v
Your AI Provider
    |
    v
AI Response
    |
    v
Catalio Application
    |
    v
User Interface

Key points:

• Catalio sends requirement text to your provider for analysis
• Responses are returned directly to Catalio
• Your provider’s data handling policies apply
• Catalio does not store provider responses except as needed for features
• API calls are logged for audit purposes (without sensitive content)

Next Steps

• Supported AI Providers - Detailed provider comparison
• Setting Up LLM API Keys - Complete setup walkthrough
• AI Features and Data Privacy - Privacy policies and data handling

Last Updated: December 2025