# OrkaJS - Production-Ready AI Framework

> TypeScript-first framework for building LLM applications with agents, RAG, workflows, evaluation, and multi-model orchestration.

## Quick Start

Install the full framework:
```bash
npm install orkajs
```

Or install selective packages:
```bash
npm install @orka-js/core @orka-js/openai @orka-js/memory
```

## Core Concepts

- **Intent-based API**: `orka.ask()`, `orka.agent()`, `orka.workflow()`, `orka.graph()`, `orka.test()`, `orka.prompts`
- **Modular Architecture**: 21+ packages, install only what you need
- **Provider Agnostic**: OpenAI, Anthropic, Mistral, Ollama, Google Gemini
- **Production Ready**: Built-in retry, fallback, observability, evaluation
- **Type-Safe**: Full TypeScript support with IDE autocomplete
- **Framework Agnostic**: Works with Express, NestJS, Fastify, Hono, AdonisJS, etc.

## Package Structure

### Core & Adapters
- `@orka-js/core` - Core types, errors, utilities, Orka class, Knowledge, Chunker
- `@orka-js/openai` - OpenAI adapter (GPT-4, GPT-4o, GPT-3.5)
- `@orka-js/anthropic` - Anthropic adapter (Claude 3.5 Sonnet, Claude 3 Opus)
- `@orka-js/mistral` - Mistral AI adapter (Mistral Large, Medium, Small)
- `@orka-js/ollama` - Ollama adapter (Local models: Llama, Mistral, etc.)
- `@orka-js/google` - Google Gemini adapter (Gemini Pro, Gemini Ultra)

### Vector Databases
- `@orka-js/memory` - In-memory vector store (dev/testing)
- `@orka-js/pinecone` - Pinecone adapter (production-grade)
- `@orka-js/qdrant` - Qdrant adapter (self-hosted/cloud)
- `@orka-js/chroma` - ChromaDB adapter (open-source)

### Agent System
- `@orka-js/agent` - Complete agent system with multiple strategies:
  - **ReActAgent**: Reasoning + Acting loop (Thought → Action → Observation)
  - **PlanAndExecuteAgent**: Plan-first execution with replanning on failure
  - **OpenAIFunctionsAgent**: Native OpenAI function calling format
  - **StructuredChatAgent**: JSON-based communication with schema validation
  - **HITLAgent**: Human-in-the-loop with approval checkpoints
  - **Agent Toolkits**:
    - SQLToolkit: `sql_query`, `sql_schema`, `sql_list_tables` (read-only mode, auto LIMIT)
    - CSVToolkit: `csv_info`, `csv_search`, `csv_filter`, `csv_aggregate`

### Data Processing
- `@orka-js/tools` - Comprehensive data processing toolkit:
  - **Document Loaders**: TextLoader, CSVLoader, JSONLoader, MarkdownLoader, PDFLoader, DirectoryLoader
  - **Text Splitters**: RecursiveCharacterTextSplitter, MarkdownTextSplitter, CodeTextSplitter, TokenTextSplitter
  - **Retrievers**: MultiQueryRetriever, ContextualCompressionRetriever, EnsembleRetriever, VectorRetriever, ParentDocumentRetriever, SelfQueryRetriever, BM25Retriever
  - **Output Parsers**: JSONParser, StructuredOutputParser (Zod), ListOutputParser, AutoFixingParser, XMLParser, CSVParser, CommaSeparatedListParser
  - **Pre-built Chains**: RetrievalQAChain, ConversationalRetrievalChain, SummarizationChain, QAChain

### Workflows & Orchestration
- `@orka-js/workflow` - Multi-step workflows with built-in steps:
  - `plan`: Generate execution plan
  - `retrieve`: Fetch relevant context from knowledge base
  - `generate`: Generate response with LLM
  - `verify`: Validate output quality
  - `improve`: Iterative refinement
  - Custom steps support
- `@orka-js/graph` - Graph-based workflows:
  - Node-based execution with conditional branching
  - Parallel execution support
  - State management across nodes
  - Mermaid diagram export for visualization
- `@orka-js/orchestration` - Multi-model orchestration strategies:
  - **RouterLLM**: Route requests by complexity/topic
  - **FallbackLLM**: Auto-healing failover (OpenAI → Anthropic → Ollama)
  - **ConsensusLLM**: Best-of-N majority voting
  - **RaceLLM**: Lowest latency wins
  - **LoadBalancerLLM**: Distribute load across providers

### Caching & Performance
- `@orka-js/cache` - Caching layer for cost optimization:
  - **MemoryCache**: In-memory caching (single instance)
  - **RedisCache**: Distributed caching (production)
  - **CachedLLM**: Wrap any LLM with automatic caching
  - **CachedEmbeddings**: Cache embedding generations
  - DJB2 hash replaced with SHA-256 for security

### Resilience & Reliability
- `@orka-js/resilience` - Production-grade resilience:
  - **Retry**: Exponential backoff with jitter
  - **FallbackLLM**: Multi-provider failover
  - **ResilientLLM**: Wrapper combining retry + fallback
  - **Timeout**: AbortController-based timeouts for all adapters

### Memory Management
- `@orka-js/memory-store` - Conversation memory:
  - **Single-session memory**: Buffer, sliding window, summary strategies
  - **Multi-session memory**: TTL-based session management
  - **Summary strategy**: Real implementation that compresses old messages

### Evaluation & Testing
- `@orka-js/evaluation` - Automated quality testing:
  - **Built-in Metrics**: Faithfulness, relevance, coherence, latency, token usage
  - **Custom Metrics**: Define your own evaluation criteria
  - **Test Runner**: CI/CD integration with `orka.test()`
  - **Assertions**: `expectFaithfulness()`, `expectLatency()`, `expectTokens()`
  - **Reporters**: ConsoleReporter, JSONReporter, JUnitReporter

### Observability & Monitoring
- `@orka-js/observability` - Production monitoring:
  - **Tracer**: Distributed tracing with hooks
  - **Logging**: Configurable log levels (debug, info, warn, error)
  - **Hooks**: `onStart`, `onSuccess`, `onError`, `onComplete`
  - **Memory leak fix**: `maxTraces` (default 1000) + TTL-based eviction

### Prompt Management
- `@orka-js/prompts` - Prompt versioning and management:
  - **Registry**: Centralized prompt storage
  - **Versioning**: Track prompt changes over time
  - **Diff**: Compare prompt versions
  - **Rollback**: Revert to previous versions
  - **Persistence**: File-based storage

### Advanced Features
- `@orka-js/multimodal` - Vision and Audio agents:
  - **VisionAgent**: Image analysis with GPT-4 Vision, Claude 3
  - **AudioAgent**: Audio processing
  - **ContentPart types**: text, image_url, image_base64, audio
- `@orka-js/mcp` - Model Context Protocol:
  - Connect to standardized MCP servers
  - Access external tools and data sources
  - Filesystem, GitHub, Database integrations
- `@orka-js/finetuning` - Model fine-tuning:
  - Dataset preparation and validation
  - Training job management
  - Evaluation and metrics
  - OpenAI fine-tuning API integration
- `@orka-js/ocr` - Optical Character Recognition:
  - Document text extraction
  - Layout analysis
  - Multi-language support
- `@orka-js/devtools` - Development tools:
  - Interactive dashboard
  - Request/response inspection
  - Performance profiling

### Templates & Utilities
- `@orka-js/templates` - Prompt templates:
  - **PromptTemplate**: Simple variable substitution
  - **ChatPromptTemplate**: Multi-message templates
  - **FewShotPromptTemplate**: Example-based prompting

### Meta Package
- `orkajs` - Meta-package re-exporting all packages for convenience

## Security Features

### PII Protection (GDPR-compliant)
- **PIIGuard**: Detect and redact sensitive information
- **Patterns**: Email, phone, SSN, credit card, IP address, etc.
- **Custom patterns**: Define your own PII patterns
- **Redaction modes**: Replace with `[EMAIL]`, `[PHONE]`, or custom placeholders

### SQL Injection Prevention
- **SQLToolkit**: Read-only mode by default
- **Query validation**: `validateReadOnlyQuery()` blocks dangerous keywords
- **Identifier validation**: `isValidIdentifier()` validates table/column names
- **Multi-statement protection**: Blocks multiple SQL statements

### SSRF Protection
- **Knowledge URL fetch**: Protocol validation (http/https only)
- **Timeout**: 30s default timeout
- **Size limit**: 50MB max response size
- **Response validation**: Check `response.ok` before processing

## Architecture Patterns

### BaseAgent Abstract Class
All agents extend `BaseAgent` which provides:
- Event emitter (`on`, `off`, `emit`)
- Standard lifecycle hooks
- Error handling
- Memory management

### Error Hierarchy
- `OrkaError`: Base error class with error codes
- `LLMError`: LLM-specific errors
- `VectorDBError`: Vector database errors
- `ValidationError`: Input validation errors
- `isRetryable()`: Helper to determine if error is retryable

## Example Usage

### Basic RAG
```typescript
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { MemoryVectorAdapter } from '@orka-js/memory';

const orka = createOrka({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
  vectorDB: new MemoryVectorAdapter(),
});

// Index documents
await orka.knowledge.create({
  name: 'docs',
  source: ['Your company documentation...'],
});

// Ask questions with RAG
const result = await orka.ask({
  knowledge: 'docs',
  question: 'How does authentication work?',
});

console.log(result.answer);
```

### Agent with Tools
```typescript
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';

const searchTool = {
  name: 'search',
  description: 'Search the web',
  execute: async (query: string) => {
    // Your search implementation
    return `Results for: ${query}`;
  },
};

const orka = createOrka({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
});

const agent = orka.agent({
  model: 'gpt-4o',
  tools: [searchTool],
});

const result = await agent.run('Find the latest news about AI');
```

### Multi-Step Workflow
```typescript
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';

const orka = createOrka({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
});

const flow = orka.workflow()
  .step('plan', { prompt: 'Create a plan for: {{input}}' })
  .step('execute', { prompt: 'Execute the plan: {{plan}}' })
  .step('verify', { prompt: 'Verify the result: {{execute}}' });

const result = await flow.run({ input: 'Build a REST API' });
```

### Multi-Model Orchestration
```typescript
import { FallbackLLM, RouterLLM } from '@orka-js/orchestration';
import { OpenAIAdapter } from '@orka-js/openai';
import { AnthropicAdapter } from '@orka-js/anthropic';
import { OllamaAdapter } from '@orka-js/ollama';

// Fallback: OpenAI → Anthropic → Ollama
const resilient = new FallbackLLM({
  adapters: [
    new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
    new AnthropicAdapter({ apiKey: process.env.ANTHROPIC_API_KEY! }),
    new OllamaAdapter({ baseURL: 'http://localhost:11434' }),
  ],
});

// Route complex prompts to GPT-4o
const smart = new RouterLLM({
  routes: [
    { 
      condition: (prompt) => prompt.length > 500, 
      adapter: new OpenAIAdapter({ model: 'gpt-4o' }),
    },
  ],
  defaultAdapter: new OpenAIAdapter({ model: 'gpt-4o-mini' }),
});

// Use anywhere an LLM is expected
const orka = createOrka({ llm: smart, vectorDB });
```

### Evaluation & Testing
```typescript
import { createOrka } from '@orka-js/core';

const orka = createOrka({ llm, vectorDB });

// Run automated tests
await orka.test({
  name: 'RAG Quality Test',
  cases: [
    {
      input: { question: 'What is OrkaJS?' },
      expectations: {
        faithfulness: { threshold: 0.8 },
        latency: { max: 2000 },
      },
    },
  ],
});

// Or evaluate a single result
const result = await orka.ask({ knowledge: 'docs', question: '...' });
await orka.evaluate(result, {
  metrics: ['faithfulness', 'relevance', 'latency'],
  threshold: 0.8,
});
```

### Graph Workflow
```typescript
import { createOrka } from '@orka-js/core';

const orka = createOrka({ llm });

const graph = orka.graph()
  .addNode('analyze', async (state) => {
    const analysis = await llm.generate(`Analyze: ${state.input}`);
    return { ...state, analysis };
  })
  .addNode('decide', async (state) => {
    const decision = state.analysis.includes('complex') ? 'detailed' : 'simple';
    return { ...state, decision };
  })
  .addNode('detailed', async (state) => {
    const result = await llm.generate(`Detailed response for: ${state.input}`);
    return { ...state, result };
  })
  .addNode('simple', async (state) => {
    const result = await llm.generate(`Simple response for: ${state.input}`);
    return { ...state, result };
  })
  .addEdge('analyze', 'decide')
  .addConditionalEdge('decide', (state) => state.decision)
  .setEntryPoint('analyze');

const result = await graph.run({ input: 'Explain quantum computing' });

// Export Mermaid diagram
const diagram = graph.toMermaid();
```

### Human-in-the-Loop
```typescript
import { HITLAgent } from '@orka-js/agent';
import { OpenAIAdapter } from '@orka-js/openai';

const agent = new HITLAgent({
  llm: new OpenAIAdapter({ model: 'gpt-4o' }),
  tools: [transferMoneyTool, sendEmailTool],
  requiresApproval: ['transfer_money'], // Only this tool needs approval
  onInterrupt: async (action) => {
    console.log(`Approval needed for: ${action.tool}`);
    console.log(`Arguments: ${JSON.stringify(action.args)}`);
    
    // Wait for user approval (e.g., from UI, CLI, webhook)
    const approved = await waitForUserApproval();
    
    if (approved) {
      return agent.resume(); // Continue execution
    } else {
      return agent.reject('User rejected the action');
    }
  },
});

const result = await agent.run('Transfer $1000 to account XYZ');
```

### MCP Integration
```typescript
import { createOrka } from '@orka-js/core';
import { MCPClient } from '@orka-js/mcp';

const mcp = new MCPClient({
  servers: [
    { name: 'filesystem', url: 'http://localhost:3000' },
    { name: 'github', url: 'http://localhost:3001' },
  ],
});

const orka = createOrka({ llm, mcp });

const agent = orka.agent({
  model: 'gpt-4o',
  tools: await mcp.getTools(['read_file', 'search_code']),
});

const result = await agent.run('Find all TypeScript files in the project');
```

### Fine-Tuning
```typescript
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';

const orka = createOrka({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
});

// Prepare training data
const trainingData = [
  { prompt: 'What is OrkaJS?', completion: 'OrkaJS is a TypeScript framework...' },
  { prompt: 'How to install?', completion: 'npm install orkajs' },
  // ... more examples
];

// Start fine-tuning job
const job = await orka.fineTune({
  model: 'gpt-4o-mini',
  dataset: trainingData,
  epochs: 3,
  validationSplit: 0.2,
});

// Monitor progress
console.log(`Job ID: ${job.id}`);
console.log(`Status: ${job.status}`);

// Use fine-tuned model
const fineTunedLLM = new OpenAIAdapter({ model: job.fineTunedModel });
```

## Best Practices

### Error Handling
```typescript
import { OrkaError, LLMError } from '@orka-js/core';

try {
  const result = await orka.ask({ knowledge: 'docs', question: '...' });
} catch (error) {
  if (error instanceof LLMError && error.isRetryable()) {
    // Retry logic
  } else if (error instanceof OrkaError) {
    console.error(`Orka error [${error.code}]:`, error.message);
  } else {
    throw error;
  }
}
```

### Caching for Cost Optimization
```typescript
import { CachedLLM } from '@orka-js/cache';
import { RedisCache } from '@orka-js/cache';
import { OpenAIAdapter } from '@orka-js/openai';

const cache = new RedisCache({
  host: 'localhost',
  port: 6379,
  ttl: 3600, // 1 hour
});

const llm = new CachedLLM({
  llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
  cache,
});

// Identical prompts will be cached
const result1 = await llm.generate('What is AI?'); // API call
const result2 = await llm.generate('What is AI?'); // Cached (no API call)
```

### Observability
```typescript
import { Tracer } from '@orka-js/observability';

const tracer = new Tracer({
  maxTraces: 1000,
  traceTtlMs: 3600000, // 1 hour
});

tracer.on('trace:start', (trace) => {
  console.log(`Started: ${trace.operation}`);
});

tracer.on('trace:complete', (trace) => {
  console.log(`Completed: ${trace.operation} in ${trace.duration}ms`);
});

const orka = createOrka({ llm, vectorDB, tracer });
```

### PII Protection
```typescript
import { PIIGuard } from '@orka-js/security';

const guard = new PIIGuard({
  patterns: ['email', 'phone', 'ssn', 'credit_card'],
});

const userInput = 'My email is john@example.com and phone is 555-1234';
const sanitized = guard.redact(userInput);
// Output: 'My email is [EMAIL] and phone is [PHONE]'

// Use with LLM
const result = await llm.generate(sanitized);
```

## Development Workflow

1. **Create feature branch**: `git checkout -b feature/my-feature`
2. **Make changes**: Edit files in `packages/*/src/`
3. **Run tests**: `pnpm test`
4. **Create changeset**: `pnpm changeset` (documents changes)
5. **Commit and PR**: `git commit -m "feat: my feature" && git push`
6. **After merge**: 
   - `pnpm version-packages` (bump versions)
   - `pnpm release` (publish to npm)

## TypeScript Configuration

OrkaJS works with all `moduleResolution` modes:
- `node` (legacy, supported)
- `node16` (modern)
- `nodenext` (modern)
- `bundler` (Vite, Webpack, etc.)

## Environment Variables

```bash
# LLM Providers
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
MISTRAL_API_KEY=...
GOOGLE_API_KEY=...

# Vector Databases
PINECONE_API_KEY=...
PINECONE_ENVIRONMENT=...
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=...

# Caching
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=...

# Observability
ORKA_LOG_LEVEL=info
ORKA_TRACE_ENABLED=true
```

## Links

- **Documentation**: https://orkajs.com
- **GitHub**: https://github.com/orka-ai/orkajs
- **npm**: https://www.npmjs.com/package/orkajs
- **Discord**: [Community Discord]
- **Twitter**: [@OrkaJS]

## Version Information

- **Current Version**: v3.4.5
- **Package Count**: 21 modular packages
- **License**: MIT
- **Node.js**: >= 18.0.0
- **TypeScript**: >= 5.0.0

## Scope & Organization

- **npm organization**: `@orka-js` (with hyphen)
- **Meta package**: `orkajs` (no scope)
- **All scoped packages**: `@orka-js/*`

## Community & Support

- Report bugs on GitHub Issues
- Join Discord for community support
- Follow on Twitter for updates
- Contribute via Pull Requests (see CONTRIBUTING.md)

## Roadmap Status

- ✅ V1: Core RAG + Basic Agents
- ✅ V2: Workflows + Evaluation + Memory
- ✅ V3: Orchestration + Observability + Prompts
- ✅ V4: Loaders + Splitters + Retrievers + Parsers
- ✅ V5: Multimodal + Caching + Templates
- ✅ V6: Advanced Retrievers + Pre-built Chains
- ✅ V7: Advanced Agents + Toolkits + New Parsers
- ✅ V8: Security (PII Guard, SQL Injection, SSRF)
- ✅ V9: MCP + Fine-tuning + OCR + DevTools
- 🚧 V10: GraphRAG + Multi-Agent Systems

## Key Differentiators

1. **TypeScript-First**: Not a Python port, built for TypeScript from day one
2. **Modular**: Install only what you need, tree-shakeable
3. **Production-Ready**: Built-in resilience, observability, evaluation
4. **Provider Agnostic**: Swap any LLM or VectorDB with one line
5. **Intent-Based API**: Code that reads like English
6. **Framework Agnostic**: Works with any Node.js framework
7. **Batteries Included**: Everything you need in one ecosystem
8. **Security-First**: PII protection, SQL injection prevention, SSRF protection
9. **Developer Experience**: Full TypeScript support, excellent docs, active community

---

Last Updated: 2026-03-20