Integrating Connect Agents into Custom Software

This guide explains how to integrate Connect agents into your custom software using our WebSocket-based communication system. The WebSocket approach provides real-time, bidirectional communication between your application and the Connect platform, enabling a more responsive and interactive user experience.

Overview
Prerequisites
WebSocket Connection
Authentication
Starting a Session
Sending Messages
Handling Responses
Tool Execution
Error Handling
Session Management
Complete Example
Best Practices

Overview

Connect's WebSocket-based agent integration allows you to:

Establish real-time communication with Connect agents
Send user messages and file attachments
Receive agent responses and tool execution requests
Execute tools on the client side and send results back
Maintain session state across multiple interactions

This approach offers several advantages over traditional REST API calls:

Lower latency for a more responsive user experience
Streaming responses for immediate feedback
Persistent connections that reduce overhead
Better handling of long-running conversations

Prerequisites

Before integrating with Connect, you'll need:

A Connect account with access to agents
The agent ID of the agent you want to integrate (for published agents)
An authentication token (for accessing private agents or user-specific tools)
A WebSocket client library for your platform

WebSocket Connection

To connect to the Connect WebSocket server, use the following endpoint:

wss://api.connect.example.com

Most programming languages and frameworks have WebSocket client libraries available. Here's a basic example using JavaScript:

const socket = new WebSocket('wss://api.connect.example.com');

socket.onopen = () => {
  console.log('Connected to Connect WebSocket server');
};

socket.onclose = () => {
  console.log('Disconnected from Connect WebSocket server');
};

socket.onerror = (error) => {
  console.error('WebSocket error:', error);
};

socket.onmessage = (event) => {
  const data = JSON.parse(event.data);
  handleMessage(data);
};

Authentication

For accessing private agents or user-specific tools, you'll need to authenticate with a JWT token. This token should be sent when starting a session:

// Get the authentication token from your auth system
const token = getAuthToken();

// Send it when starting a session
socket.send(JSON.stringify({
  type: 'start_session',
  agentId: 'agent-123',
  token: token
}));

For public agents, authentication is optional.

Starting a Session

To start a new session with an agent, send a start_session message:

socket.send(JSON.stringify({
  type: 'start_session',
  agentId: 'agent-123', // Optional: Specific agent ID
  token: 'your-auth-token' // Optional: Authentication token
}));

The server will respond with a session_started message containing a session ID:

{
  type: 'session_started',
  sessionId: 'session-456'
}

Store this session ID as you'll need it for all subsequent messages.

Sending Messages

To send a user message to the agent, use the user_message event type:

socket.send(JSON.stringify({
  type: 'user_message',
  sessionId: 'session-456',
  message: 'Hello, can you help me with something?',
  files: [] // Optional: Array of file attachments
}));

Attaching Files

You can attach files to your messages for the agent to process:

// Read a file and convert to base64
const fileContent = await readFileAsBase64('document.pdf');

socket.send(JSON.stringify({
  type: 'user_message',
  sessionId: 'session-456',
  message: 'Can you analyze this document?',
  files: [
    {
      name: 'document.pdf',
      content: fileContent, // Base64-encoded content
      type: 'application/pdf'
    }
  ]
}));

The server will extract text from supported file types and include it in the context for the agent.

Handling Responses

The agent will respond with one or more agent_response messages. These can be of different types:

Text Messages

{
  type: 'agent_response',
  responseType: 'message',
  message: 'Hello! I'd be happy to help you with that.'
}

Tool Calls

When the agent needs to use a tool, it will send a tool call that should be executed on the client side:

{
  type: 'agent_response',
  responseType: 'tool_calls',
  toolCalls: [
    {
      id: 'call-789',
      type: 'function',
      function: {
        name: 'author-12345-search',
        arguments: '{"query":"climate change","limit":5}'
      }
    }
  ]
}

Tool Execution

When you receive a tool call, you should:

Execute the tool on the client side
Send the result back to the server

// Execute the tool
const toolCall = data.toolCalls[0];
const args = JSON.parse(toolCall.function.arguments);
const result = await executeTool(toolCall.function.name, args);

// Send the result back
socket.send(JSON.stringify({
  type: 'tool_execution_result',
  sessionId: 'session-456',
  toolCallId: toolCall.id,
  result: JSON.stringify(result)
}));

If there's an error during tool execution:

socket.send(JSON.stringify({
  type: 'tool_execution_result',
  sessionId: 'session-456',
  toolCallId: toolCall.id,
  error: 'Failed to execute search: Network error'
}));

Error Handling

The server may send error messages in case of issues:

{
  type: 'error',
  message: 'Session not found'
}

Your client should handle these errors appropriately, potentially by retrying operations or notifying the user.

Session Management

Sessions have a timeout period (default: 10 minutes). If a session expires due to inactivity, you'll receive a session_expired message:

{
  type: 'session_expired',
  sessionId: 'session-456'
}

When this happens, you should start a new session if the user wants to continue the conversation.

You can also explicitly close a session by disconnecting from the WebSocket server.

Complete Example

Here's a complete example in JavaScript:

class ConnectAgentClient {
  constructor(serverUrl) {
    this.serverUrl = serverUrl;
    this.socket = null;
    this.sessionId = null;
    this.messageHandlers = {
      onMessage: () => {},
      onToolCall: () => {},
      onError: () => {},
      onSessionExpired: () => {}
    };
  }

  connect() {
    this.socket = new WebSocket(this.serverUrl);

    this.socket.onopen = () => {
      console.log('Connected to Connect WebSocket server');
    };

    this.socket.onclose = () => {
      console.log('Disconnected from Connect WebSocket server');
    };

    this.socket.onerror = (error) => {
      console.error('WebSocket error:', error);
      this.messageHandlers.onError('Connection error');
    };

    this.socket.onmessage = (event) => {
      const data = JSON.parse(event.data);
      this.handleMessage(data);
    };
  }

  handleMessage(data) {
    switch (data.type) {
      case 'session_started':
        this.sessionId = data.sessionId;
        console.log(`Session started: ${this.sessionId}`);
        break;

      case 'agent_response':
        if (data.responseType === 'message' && data.message) {
          this.messageHandlers.onMessage(data.message);
        } else if (data.responseType === 'tool_calls' && data.toolCalls) {
          for (const toolCall of data.toolCalls) {
            this.messageHandlers.onToolCall(toolCall);
          }
        }
        break;

      case 'error':
        console.error('Server error:', data.message);
        this.messageHandlers.onError(data.message);
        break;

      case 'session_expired':
        console.log('Session expired');
        this.sessionId = null;
        this.messageHandlers.onSessionExpired();
        break;

      default:
        console.warn('Unknown message type:', data.type);
    }
  }

  startSession(agentId, token) {
    if (!this.socket || this.socket.readyState !== WebSocket.OPEN) {
      throw new Error('WebSocket not connected');
    }

    this.socket.send(JSON.stringify({
      type: 'start_session',
      agentId,
      token
    }));
  }

  sendMessage(message, files = []) {
    if (!this.sessionId) {
      throw new Error('No active session');
    }

    this.socket.send(JSON.stringify({
      type: 'user_message',
      sessionId: this.sessionId,
      message,
      files
    }));
  }

  sendToolResult(toolCallId, result, error) {
    if (!this.sessionId) {
      throw new Error('No active session');
    }

    this.socket.send(JSON.stringify({
      type: 'tool_execution_result',
      sessionId: this.sessionId,
      toolCallId,
      result: error ? undefined : JSON.stringify(result),
      error
    }));
  }

  setEventHandlers(handlers) {
    this.messageHandlers = { ...this.messageHandlers, ...handlers };
  }

  disconnect() {
    if (this.socket) {
      this.socket.close();
      this.socket = null;
      this.sessionId = null;
    }
  }
}

// Usage example
const client = new ConnectAgentClient('wss://api.connect.example.com');

client.setEventHandlers({
  onMessage: (message) => {
    console.log('Agent says:', message);
    document.getElementById('responses').innerHTML += `<p>${message}</p>`;
  },
  
  onToolCall: async (toolCall) => {
    console.log('Tool call:', toolCall);
    
    try {
      const args = JSON.parse(toolCall.function.arguments);
      const result = await executeTool(toolCall.function.name, args);
      
      client.sendToolResult(toolCall.id, result);
    } catch (error) {
      client.sendToolResult(toolCall.id, null, error.message);
    }
  },
  
  onError: (error) => {
    console.error('Error:', error);
    document.getElementById('errors').innerHTML = `<p class="error">${error}</p>`;
  },
  
  onSessionExpired: () => {
    console.log('Session expired, starting new session');
    client.startSession('agent-123');
  }
});

client.connect();

// When the user wants to start a conversation
document.getElementById('start-button').addEventListener('click', () => {
  client.startSession('agent-123');
});

// When the user sends a message
document.getElementById('send-button').addEventListener('click', () => {
  const messageInput = document.getElementById('message-input');
  const message = messageInput.value.trim();
  
  if (message) {
    client.sendMessage(message);
    messageInput.value = '';
  }
});

// When the user attaches a file
document.getElementById('file-input').addEventListener('change', async (event) => {
  const file = event.target.files[0];
  if (file) {
    const reader = new FileReader();
    reader.onload = (e) => {
      const base64Content = e.target.result.split(',')[1];
      client.sendMessage('I\'m attaching a file', [
        {
          name: file.name,
          content: base64Content,
          type: file.type
        }
      ]);
    };
    reader.readAsDataURL(file);
  }
});

// Tool execution function
async function executeTool(name, args) {
  // Implement your tool execution logic here
  // This is where you would call your actual tool implementations
  console.log(`Executing tool ${name} with args:`, args);
  
  // Example implementation for a search tool
  if (name.endsWith('-search')) {
    return {
      results: [
        { title: 'Result 1', url: 'https://example.com/1' },
        { title: 'Result 2', url: 'https://example.com/2' }
      ]
    };
  }
  
  throw new Error(`Unknown tool: ${name}`);
}

Best Practices

Error Handling: Implement robust error handling for WebSocket connections, including reconnection logic.
Message Queuing: If the WebSocket connection is temporarily lost, queue messages to be sent when the connection is restored.
Timeouts: Implement timeouts for tool executions to prevent hanging operations.
User Feedback: Provide clear feedback to users about the connection status and ongoing operations.
Security: Store authentication tokens securely and never expose them in client-side code.
Rate Limiting: Implement rate limiting to prevent abuse of the WebSocket connection.
Testing: Test your integration with different agents and tools to ensure compatibility.
Logging: Implement comprehensive logging to help debug issues in production.
Graceful Degradation: Have a fallback mechanism if WebSockets are not supported or blocked.
Session Management: Handle session expiration gracefully and provide a smooth experience for starting new sessions.

By following this guide, you should be able to successfully integrate Connect agents into your custom software using our WebSocket-based communication system. If you encounter any issues or have questions, please contact our support team.

Table of Contents​

Overview​

Prerequisites​

WebSocket Connection​

Authentication​

Starting a Session​

Sending Messages​

Attaching Files​

Handling Responses​

Text Messages​

Tool Calls​

Tool Execution​

Error Handling​

Session Management​

Complete Example​

Best Practices​

Table of Contents