Reduce Token Usage & Optimize Costs

Agent thinking → LLM prompt + 100 tool schemas (50k tokens) → LLM response
Cost: $0.75 per request

With Agent-CoreX:
Agent thinking → Query Agent-CoreX → Get 5 relevant tools (5k tokens) → LLM response
Cost: $0.20 per request (73% savings!)

// ❌ Vague (uses more tokens)
await agent.retrieveTools({
  query: "do something with GitHub"
});

// ✅ Specific (fewer tokens)
await agent.retrieveTools({
  query: "Create a pull request on GitHub with code changes"
});

// ❌ Request all tools (200 tokens)
await agent.retrieveTools({
  query: "Deploy",
  topK: 50
});

// ✅ Request what you need (50 tokens)
await agent.retrieveTools({
  query: "Deploy",
  topK: 3
});

// ❌ Search all 100+ servers (higher cost)
await agent.retrieveTools({
  query: "Deploy"
});

// ✅ Filter to AWS only (lower cost)
await agent.retrieveTools({
  query: "Deploy",
  filter: {
    server: "aws-mcp"
  }
});

const toolCache = new Map();

async function getCachedTools(query) {
  // Check cache (1 hour TTL)
  if (toolCache.has(query)) {
    const cached = toolCache.get(query);
    if (Date.now() - cached.time < 3600000) {
      return cached.tools;
    }
  }

  // Not in cache, fetch
  const tools = await agent.retrieveTools({ query });
  toolCache.set(query, { tools, time: Date.now() });
  return tools;
}

// ❌ Three separate queries
const deployTools = await agent.retrieveTools({
  query: "Deploy"
});
const monitorTools = await agent.retrieveTools({
  query: "Monitor"
});
const notifyTools = await agent.retrieveTools({
  query: "Notify"
});

// ✅ One combined query
const allTools = await agent.retrieveTools({
  query: "Deploy, monitor, and notify"
});

Scenario: ChatGPT agent with 100 tools

Per request:
- Prompt (2k tokens): $0.01
- 100 tool schemas (48k tokens): $0.72
- Completion (1k tokens): $0.03
Total: $0.76

1000 requests/month: $760

Per request:
- Prompt (2k tokens): $0.01
- Retrieve tools (200 tokens): $0.002
- 5 tool schemas (3k tokens): $0.04
- Completion (1k tokens): $0.03
Total: $0.082

1000 requests/month: $82
Savings: 89%!