Guides

LLM

Bring your own provider keys and call ctx.llm.complete or ctx.llm.stream from handlers.

Every defineApi, defineJob, and defineSocket handler receives an injected llm client. Keys are per-project, encrypted at rest, and chosen on the dashboard AI page. They are not project Secrets and are never copied into ctx.env. Starbase decrypts the key at call time and talks to the provider from the control plane — tenant bundles never see the secret.

If you omit provider, Starbase uses the first configured key in this order: grok, gemini, openai, anthropic. If none are configured, the call throws llm_not_configured. Per-project rate limits are 60 complete/stream starts per minute and 10 in flight; over that throws llm_rate_limited (HTTP 429).

llm.complete

import { defineApi, z } from "@robodev-ai/sdk";
export const post = defineApi({
  body: z.object({ prompt: z.string().min(1) }),
  handler: async ({ llm, body }) => {
    const result = await llm.complete({
      provider: "grok",
      messages: [{ role: "user", content: body.prompt }],
    });
    return { text: result.text, model: result.model };
  },
});

llm.stream

import { defineSocket } from "@robodev-ai/sdk";
export default defineSocket({
  onMessage: async ({ llm, send }, data) => {
    const prompt = typeof data === "string" ? data : "";
    for await (const chunk of llm.stream({
      messages: [{ role: "user", content: prompt }],
    })) {
      if (chunk.text) send({ text: chunk.text });
      if (chunk.done) send({ done: true });
    }
  },
});