Callbacks & Events

Centralized event handling for LLM, tools, agents, and chains for responsive AI applications.

OrkaJS provides a powerful CallbackManager for centralized event handling across your entire application. Track token streaming, tool execution, agent actions, and more with a unified API.

Key Features

Centralized event management across all components
Token-level callbacks (onTokenStart, onToken, onTokenEnd)
Tool execution tracking (onToolStart, onToolEnd, onToolError)
Agent lifecycle events (onAgentAction, onAgentFinish)
Chain and retrieval events
Async callback support with error isolation
Event filtering by type

Quick Start

Create a callback handler and add it to the global manager:

import { 
  getCallbackManager, 
  createCallbackHandler 
} from '@orka-js/core';
 
// Create a handler with shortcut callbacks
const myHandler = createCallbackHandler('my-handler', {
  onToken: (token, index, content) => {
    process.stdout.write(token);
  },
  onToolStart: (event) => {
    console.log(`Tool ${event.toolName} started`);
  },
  onToolEnd: (event) => {
    console.log(`Tool ${event.toolName} completed in ${event.durationMs}ms`);
  },
  onAgentAction: (event) => {
    console.log(`Agent action: ${event.action}`);
  },
  onAgentFinish: (event) => {
    console.log(`Agent finished in ${event.durationMs}ms`);
  },
});
 
// Add to global manager
const manager = getCallbackManager();
manager.addHandler(myHandler);

#CallbackManager

The CallbackManager is the central hub for all events:

import { CallbackManager } from '@orka-js/core';
 
// Create a new manager
const manager = new CallbackManager({
  asyncMode: true,   // Fire-and-forget callbacks (default)
  verbose: false,    // Log event emissions
});
 
// Add handlers
manager.addHandler({
  name: 'logger',
  eventTypes: ['tool_start', 'tool_end'], // Only these events
  handleEvent(event) {
    console.log(`[${event.type}]`, event);
  },
});
 
// Emit events programmatically
const runId = await manager.emitToolStart('search', { query: 'test' });
await manager.emitToolEnd(runId, 'search', { query: 'test' }, { results: [] }, 150);
 
// Remove handler
manager.removeHandler('logger');
 
// Clear all handlers
manager.clearHandlers();

Event Types

Available callback event types:

System Domain	Monitoring Context	Event Hooks
Token	Real-time generation & latency	`token_starttokentoken_end`
Tool	External function execution	`tool_starttool_endtool_error`
Agent	High-level reasoning & decisions	`agent_actionagent_finishagent_error`
Chain	Workflow orchestration	`chain_startchain_endchain_error`
LLM	Model inference cycles	`llm_startllm_endllm_error`
Retrieval	Vector search & context injection	`retrieval_startretrieval_end`

// Token/Streaming events
type TokenEvents = 'token_start' | 'token' | 'token_end';
 
// Tool events
type ToolEvents = 'tool_start' | 'tool_end' | 'tool_error';
 
// Agent events  
type AgentEvents = 'agent_action' | 'agent_observation' | 'agent_finish' | 'agent_error';
 
// Chain events
type ChainEvents = 'chain_start' | 'chain_end' | 'chain_error';
 
// LLM events
type LLMEvents = 'llm_start' | 'llm_end' | 'llm_error';
 
// Retrieval events
type RetrievalEvents = 'retrieval_start' | 'retrieval_end';
 
// All events
type CallbackEventType = 
  | TokenEvents 
  | ToolEvents 
  | AgentEvents 
  | ChainEvents 
  | LLMEvents 
  | RetrievalEvents;

#Creating Handlers

Use createCallbackHandler for a simpler API:

import { createCallbackHandler } from '@orka-js/core';
 
const handler = createCallbackHandler('analytics', {
  // Token events
  onTokenStart: (event) => {
    console.log('Generation started for:', event.prompt.slice(0, 50));
  },
  onToken: (token, index, content) => {
    // Track token count
  },
  onTokenEnd: (event) => {
    console.log(`Generated ${event.tokenCount} tokens in ${event.durationMs}ms`);
  },
 
  // Tool events
  onToolStart: (event) => {
    analytics.track('tool_invoked', { tool: event.toolName });
  },
  onToolEnd: (event) => {
    analytics.track('tool_completed', { 
      tool: event.toolName, 
      duration: event.durationMs 
    });
  },
  onToolError: (event) => {
    analytics.track('tool_error', { 
      tool: event.toolName, 
      error: event.error.message 
    });
  },
 
  // Agent events
  onAgentAction: (event) => {
    console.log(`Agent thinking: ${event.thought}`);
    console.log(`Action: ${event.action}`);
  },
  onAgentFinish: (event) => {
    console.log(`Agent completed in ${event.durationMs}ms`);
  },
 
  // LLM events
  onLLMStart: (event) => {
    console.log(`Calling ${event.model}...`);
  },
  onLLMEnd: (event) => {
    console.log(`LLM used ${event.usage.totalTokens} tokens`);
  },
});

#Agent Integration

Agents automatically emit events to the CallbackManager:

import { ReActAgent } from '@orka-js/agent';
import { OpenAIAdapter } from '@orka-js/openai';
import { getCallbackManager, createCallbackHandler } from '@orka-js/core';
 
const llm = new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! });
 
// Set up callback handler
const manager = getCallbackManager();
manager.addHandler(createCallbackHandler('agent-monitor', {
  onToolStart: (event) => {
    console.log(`🔧 Tool: ${event.toolName}`);
  },
  onAgentAction: (event) => {
    console.log(`🤔 Thought: ${event.thought}`);
    console.log(`⚡ Action: ${event.action}`);
  },
  onAgentFinish: (event) => {
    console.log(`✅ Completed in ${event.durationMs}ms`);
  },
}));
 
// Create agent - it automatically uses the global CallbackManager
const agent = new ReActAgent(
  {
    goal: 'Help users with calculations',
    tools: [
      {
        name: 'calculate',
        description: 'Perform math calculations',
        execute: async ({ expression }) => ({ output: eval(expression) }),
      },
    ],
  },
  llm
);
 
// Or pass a specific CallbackManager
const customManager = new CallbackManager();
const agentWithCustomManager = new ReActAgent(
  {
    goal: 'Help users',
    tools: [],
    callbackManager: customManager, // Use custom manager
  },
  llm
);
 
// Run agent - events are automatically emitted
const result = await agent.run('What is 25 * 4?');

Console Handler

Use the built-in console handler for debugging:

import { getCallbackManager, consoleCallbackHandler } from '@orka-js/core';
 
// Add built-in console handler for debugging
const manager = getCallbackManager();
manager.addHandler(consoleCallbackHandler);
 
// Now all events will be logged to console:
// [2024-01-15T10:30:00.000Z] [token_start] Starting generation...
// hello world
// [2024-01-15T10:30:01.000Z] [token_end] Completed in 1000ms (15 tokens)
// [2024-01-15T10:30:01.100Z] [tool_start] Tool: search
// [2024-01-15T10:30:01.500Z] [tool_end] Tool search completed in 400ms

Global vs Local Managers

You can use the global manager or create local instances:

import { 
  CallbackManager,
  getCallbackManager, 
  setCallbackManager,
  resetCallbackManager 
} from '@orka-js/core';
 
// Get the global singleton
const global = getCallbackManager();
 
// Replace global manager
const custom = new CallbackManager({ verbose: true });
setCallbackManager(custom);
 
// Reset to a fresh global manager
resetCallbackManager();
 
// Create child manager (inherits parent handlers)
const parent = getCallbackManager();
parent.addHandler({ name: 'parent', handleEvent: () => {} });
 
const child = parent.createChild();
child.addHandler({ name: 'child', handleEvent: () => {} });
 
// child has both 'parent' and 'child' handlers
// parent only has 'parent' handler

Best Practices

Use async mode (default) for non-blocking callbacks
Filter events by type to reduce overhead
Handle errors in callbacks to prevent silent failures
Use child managers for isolated scopes