Agnost

# Agnost Analytics SDK (TypeScript) [](https://badge.fury.io/js/agnost) [](https://opensource.org/licenses/MIT) Analytics SDK for tracking and analyzing Model Context Protocol (MCP) server interactions. ## Installation ```bash npm install agnost ``` ## Setup Example ```typescript import { trackMCP, createConfig } from 'agnost'; import { Server } from '@modelcontextprotocol/sdk/server/index.js'; // Create your MCP server instance const server = new Server( { name: "my-server", version: "1.0.0" }, { capabilities: { tools: {} } } ); // Configure analytics const config = createConfig({ endpoint: "https://api.agnost.ai", disableInput: false, disableOutput: false }); // Enable analytics tracking const trackedServer = trackMCP(server, "your-organization-id", config); ``` ## Configuration Example ```typescript import { trackMCP, createConfig } from 'agnost'; // Create a custom configuration const config = createConfig({ endpoint: "https://api.agnost.ai", disableInput: false, // Set to true to disable input tracking disableOutput: false, // Set to true to disable output tracking disableLogs: false // Set to true to completely disable all SDK logs }); // Apply the configuration trackMCP( server, "your-organization-id", config ); ``` ### Disabling All Logs To completely disable all SDK logs (including error logs), you can use the `disableLogs` option: ```typescript import { trackMCP, createConfig } from 'agnost'; // Configuration with all logs disabled const config = createConfig({ endpoint: "https://api.agnost.ai", disableLogs: true // This will disable ALL SDK logs }); trackMCP(server, "your-organization-id", config); ``` Alternatively, you can use environment variables: ```bash # Disable all logs via environment variable export AGNOST_DISABLE_LOGS=true # Or control log level (debug, info, warning, error) export AGNOST_LOG_LEVEL=error ``` ## User Identification The SDK supports user identification to track analytics per user. This is especially useful for understanding usage patterns across different users and roles. ### Basic User Identification ```typescript import { trackMCP, createConfig } from 'agnost'; // Enable user identification trackMCP(server, 'your-org-id', { // .. other config like disableInput, disableOutput identify: (request, env) => ({ userId: request?.headers?.['x-user-id'] || env?.USER_ID || 'anonymous', email: request?.headers?.['x-user-email'] || env?.USER_EMAIL, role: request?.headers?.['x-user-role'] || env?.USER_ROLE || 'user' }) }); ``` ### Advanced User Identification ```typescript import { trackMCP, createConfig } from 'agnost'; // Complex identification logic with async operations trackMCP(server, 'your-org-id', { identify: async (request, env) => { try { // Extract token from headers const token = request?.headers?.['authorization']?.replace('Bearer ', ''); if (!token) { return { userId: 'anonymous' }; } // You could validate token and fetch user info // const userInfo = await validateTokenAndGetUser(token); // Return user identity with custom fields return { userId: 'user-123', email: 'user@example.com', role: 'admin', organization: 'acme-corp', subscription: 'premium' }; } catch (error) { console.warn('User identification failed:', error); return { userId: 'anonymous' }; } } }); ``` ### User Identity Interface The identify function should return a `UserIdentity` object or `null`: ```typescript interface UserIdentity { userId: string; // Required: Unique user identifier [key: string]: any; // Optional: Any additional user properties } type IdentifyFunction = ( request?: any, // MCP request object with headers, params, etc. env?: Record<string, string | undefined> // Environment variables (process.env) ) => UserIdentity | null | Promise<UserIdentity | null>; ``` ### Identify Function Parameters - **`request`**: The incoming MCP request object containing: - `headers`: HTTP-style headers (e.g., `x-user-id`, `authorization`) - `params`: Request parameters including tool name and arguments - Other request metadata from the MCP protocol - **`env`**: Environment variables from `process.env`, useful for: - Reading user info from environment variables - Accessing configuration secrets - Getting deployment-specific user context ### Common Usage Patterns #### 1. Header-based Identification ```typescript identify: (request, env) => ({ userId: request?.headers?.['x-user-id'] || 'anonymous', role: request?.headers?.['x-user-role'] || 'user' }) ``` #### 2. Environment Variable Identification ```typescript identify: (request, env) => ({ userId: env?.USER_ID || env?.LOGGED_IN_USER || 'anonymous', workspace: env?.WORKSPACE_ID }) ``` #### 3. Token-based Identification ```typescript identify: async (request, env) => { const authHeader = request?.headers?.['authorization']; if (authHeader?.startsWith('Bearer ')) { const token = authHeader.replace('Bearer ', ''); const decoded = await decodeJWT(token); return { userId: decoded.sub, email: decoded.email, role: decoded.role }; } return { userId: 'anonymous' }; } ``` ### Important Notes - The `userId` field is **required** in the returned `UserIdentity` object - If identification fails, return `null` or `{ userId: 'anonymous' }` - User identification happens once per session and is cached - Any errors in the identify function are logged and fallback to anonymous tracking - Additional fields beyond `userId` are included in analytics for segmentation ### Configuration Options | Option | Type | Default | Description | |--------|------|---------|-------------| | `endpoint` | `string` | `"https://api.agnost.ai"` | API endpoint URL | | `disableInput` | `boolean` | `false` | Disable tracking of input arguments | | `disableOutput` | `boolean` | `false` | Disable tracking of output results | | `disableLogs` | `boolean` | `false` | Completely disable all SDK logs | | `identify` | `IdentifyFunction` | `undefined` | Function to identify users from request context | ## Performance Monitoring with Checkpoints The TypeScript SDK provides a powerful `checkpoint()` function for detailed latency breakup of tool calls. Checkpoints allow you to track specific points in your tool's execution, providing granular observability into where time is being spent. ### Overview When analyzing tool performance, knowing the total execution time is often not enough. The `checkpoint()` function lets you mark specific points in your execution flow to understand: - Which operations are slow - Where bottlenecks occur - How time is distributed across different phases - Performance impact of external API calls, database queries, or processing steps All checkpoint data is automatically captured and visualized in the Agnost AI dashboard with interactive timeline charts. ### Function Signature ```typescript import { checkpoint } from 'agnost'; checkpoint(name: string, metadata?: any): void ``` **Parameters:** - `name` (string): A descriptive name for the checkpoint (e.g., "database_query_start", "api_call_complete") - `metadata` (optional): Any additional context to attach to this checkpoint (e.g., row counts, response sizes, status codes) ### Basic Usage ```typescript import { checkpoint } from 'agnost'; import { z } from 'zod'; // Define your tool server.tool( 'get_user_data', 'Fetches and processes user data from the database', { userId: z.string().describe('The user ID to fetch') }, async ({ userId }) => { // Mark the start of input validation checkpoint('input_validation_start'); if (!userId || userId.length === 0) { throw new Error('Invalid user ID'); } checkpoint('input_validation_complete'); // Mark the start of database query checkpoint('database_query_start'); const userData = await db.query('SELECT * FROM users WHERE id = ?', [userId]); checkpoint('database_query_complete', { rowCount: userData.length }); // Mark the start of data processing checkpoint('data_processing_start'); const processed = await processUserData(userData); checkpoint('data_processing_complete', { recordsProcessed: processed.length }); return { content: [ { type: 'text', text: JSON.stringify(processed) } ] }; } ); ``` ### Advanced Example: API Call Monitoring ```typescript import { checkpoint } from 'agnost'; server.tool( 'fetch_weather', 'Fetches weather data from external API', { city: z.string() }, async ({ city }) => { // Track input normalization checkpoint('input_normalization_start'); const normalizedCity = city.trim().toLowerCase(); checkpoint('input_normalization_complete'); // Track cache lookup checkpoint('cache_lookup_start'); const cached = await cache.get(`weather:${normalizedCity}`); checkpoint('cache_lookup_complete', { cacheHit: !!cached }); if (cached) { checkpoint('returning_cached_data'); return cached; } // Track external API call checkpoint('api_call_start'); const response = await fetch(`https://api.weather.com/v1/${normalizedCity}`); checkpoint('api_call_complete', { statusCode: response.status, responseSize: response.headers.get('content-length') }); // Track response parsing checkpoint('response_parsing_start'); const data = await response.json(); checkpoint('response_parsing_complete'); // Track cache update checkpoint('cache_update_start'); await cache.set(`weather:${normalizedCity}`, data, 3600); checkpoint('cache_update_complete'); return { content: [ { type: 'text', text: JSON.stringify(data) } ] }; } ); ``` ### How Checkpoints Work 1. **Automatic Context Tracking**: When a tool is called, the SDK automatically creates an execution context using AsyncLocalStorage 2. **Relative Timestamps**: Each checkpoint records the time elapsed since the tool execution started (in milliseconds) 3. **Metadata Capture**: Optional metadata is stored with each checkpoint for additional context 4. **Safe Operation**: Checkpoints called outside of tool execution are safely ignored (no errors thrown) 5. **Zero Performance Impact**: Checkpoints are optimized for minimal overhead and won't affect your tool's performance ### Checkpoint Data Structure Each checkpoint is recorded with the following structure: ```typescript interface Checkpoint { name: string; // The checkpoint name timestamp: number; // Milliseconds since execution start metadata?: any; // Optional metadata object } ``` ### Dashboard Visualization Checkpoints are automatically visualized in the Agnost AI dashboard with: - **Timeline Bar Chart**: Visual representation of time spent between checkpoints - **Detailed Breakdown**: List of all checkpoints with: - Absolute timestamp (ms from start) - Duration since previous checkpoint - Percentage of total latency - Metadata display - **Remaining Time Analysis**: Shows overhead/time not covered by explicit checkpoints Example timeline visualization: ``` [0ms--------50ms][50ms---------200ms][200ms----250ms] Input Valid DB Query Processing ``` ### Best Practices 1. **Use Descriptive Names**: Make checkpoint names clear and specific ```typescript // Good checkpoint('database_query_complete'); checkpoint('external_api_call_start'); // Avoid checkpoint('step1'); checkpoint('done'); ``` 2. **Track Start and End**: For operations you want to measure, add both start and end checkpoints ```typescript checkpoint('operation_start'); await expensiveOperation(); checkpoint('operation_complete'); ``` 3. **Add Useful Metadata**: Include context that helps debug performance issues ```typescript checkpoint('query_complete', { rowCount: results.length, queryTime: Date.now() - startTime, cacheHit: false }); ``` 4. **Focus on Expensive Operations**: Add checkpoints around: - Database queries - External API calls - File I/O operations - Heavy computation - Network requests 5. **Don't Over-checkpoint**: Too many checkpoints can make analysis harder. Focus on meaningful boundaries ```typescript // Good: Major operation boundaries checkpoint('fetch_data_start'); checkpoint('fetch_data_complete'); checkpoint('process_data_complete'); // Avoid: Too granular checkpoint('variable_declared'); checkpoint('loop_iteration_1'); checkpoint('loop_iteration_2'); ``` ### Common Patterns #### Pattern 1: Database Operations ```typescript checkpoint('db_connection_start'); const connection = await pool.getConnection(); checkpoint('db_connection_acquired'); checkpoint('db_query_start'); const results = await connection.query(sql); checkpoint('db_query_complete', { rowCount: results.length }); ``` #### Pattern 2: Multi-Step Processing Pipeline ```typescript checkpoint('fetch_raw_data'); const raw = await fetchData(); checkpoint('transform_data'); const transformed = transform(raw); checkpoint('validate_data'); const validated = validate(transformed); checkpoint('store_data'); await store(validated); checkpoint('pipeline_complete'); ``` #### Pattern 3: Parallel Operations ```typescript checkpoint('parallel_operations_start'); const [result1, result2, result3] = await Promise.all([ operation1(), operation2(), operation3() ]); checkpoint('parallel_operations_complete', { operation1Time: result1.duration, operation2Time: result2.duration, operation3Time: result3.duration }); ``` ### Troubleshooting **Checkpoints not appearing in dashboard:** - Ensure you're calling `checkpoint()` inside a tracked tool handler - Verify analytics tracking is enabled with `trackMCP()` - Check that your organization ID is correct **Timestamps seem incorrect:** - Timestamps are relative to tool execution start (not absolute time) - Ensure you're not calling `checkpoint()` outside of tool execution context **Performance concerns:** - Checkpoints have minimal overhead (< 1ms per checkpoint) - They use object pooling and efficient timestamp recording - Safe to use even in high-frequency tools