LLM Summarizer

Fetch a document, hand it to a model, return the gist: the pattern behind most "read this for me" agents.

An agent runs inside the VM; the host is the JavaScript program that embeds the VM and starts the run. The host passes in capabilities (IO functions it injects, such as a model), sets policy for the VM's built-in IO (which domains a fetch may reach), and sets a fuel budget (a meter of the work the agent does; the run stops when it is spent).

Every example on this page calls a model, which this static site does not have, so none of them run here. Each prints this instead, which is expected, not a broken example:

This example needs an LLM, which the static site does not provide. Run it locally with
tjs-lang and LM Studio (see the AJS guide).

Deleting the llmPredict line to watch the fetch on its own does not help either: the site's fetch allowlist does not include tjs.tosijs.net. Run them locally with the host below.

function summarize({ source = 'coffee-origins' }) {
  // The sample documents served by this site
  let sources = ['coffee-origins', 'ai-history', 'renewable-energy']
  if (!sources.includes(source)) {
    return { source, error: 'Unknown source; try one of: ' + sources.join(', ') }
  }
  let url = 'https://tjs.tosijs.net/texts/' + source + '.txt'
  let text = httpFetch({ url })

  let prompt = 'Summarize the following text in 2-3 sentences:\n\n' + text
  let summary = llmPredict({ prompt })
  return { source, summary }
}

What to notice

Running it locally

The host transpiles the agent on its own side and hands the VM only the AST: tjs-lang/vm-ast contains no parser, so nothing in the sandbox parses source. The source string holds one function, and that function is the agent.

The agent's llmPredict call becomes a call to the capability's llm.predict(prompt, options), which must return a string. Any object of that shape works; the llm in LLM Chat talks to any OpenAI-compatible server. A stand-in is enough to watch the plumbing:

import { transpile } from 'tjs-lang/lang'
import { AgentVM } from 'tjs-lang/vm-ast'

const agentSource = `
function summarize({ source = 'coffee-origins' }) {
  // ...the agent above...
}`

// Replace with the llm from LLM Chat to call a real model
const llm = {
  async predict(prompt, options) {
    return `(a summary of ${prompt.length} characters)`
  },
}

const { ast } = transpile(agentSource)
const run = await new AgentVM().run(
  ast,
  { source: 'ai-history' },
  {
    fuel: 1000, // plenty: a run uses about 107
    capabilities: { llm },
    context: { allowedFetchDomains: ['tjs.tosijs.net'] }, // where httpFetch may go
    quotas: { llmPredict: 1 }, // at most one model call per run
    membraneMaxBytes: 1024 * 1024, // the largest value that may cross in or out (default 4 MB)
  }
)
if (run.error) console.log('run failed:', run.error.message)
else if (run.result.error) console.log('agent refused:', run.result.error)
else console.log(run.result.summary)

With the stand-in, this prints (a summary of 1332 characters). With a real model, run.result looks like this:

{ source: 'ai-history', summary: 'Artificial intelligence began as…' }

An agent can fail in two ways, and the host checks for both. The agent can return an error as data: with { source: 'nope' }, the run succeeds and run.result.error is 'Unknown source; try one of: coffee-origins, ai-history, renewable-energy'. Or the run can fail: any error the agent does not catch ends the run, and vm.run still resolves normally, with the failure in run.error. Your code never sees a thrown exception.

A run uses about 107 units of fuel: 100 for the model call, 5 for the fetch, the rest for expressions. The budget of 1000 is headroom, not a target; set it well above what a normal run uses, and low enough to stop a runaway one. If fuel runs out, the run ends with run.error whose message is Out of Fuel. Leave out allowedFetchDomains and the fetch is refused with Fetch blocked: no allowedFetchDomains configured; leave out llm and the model call fails with Capability 'llm.predict' missing. This agent catches neither, so both arrive in run.error.

Long documents

Long documents cost tokens and can exceed a model's context. Cut the text before you send it, and tell the model what you did:

function summarizeUrl({
  url = 'https://tjs.tosijs.net/texts/renewable-energy.txt',
  maxChars = 600,
}) {
  let text = httpFetch({ url })
  let truncated = text.length > maxChars
  let excerpt = text.slice(0, maxChars)
  let note = truncated ? ' (the text was cut short)' : ''
  let prompt = 'Summarize in 3 bullet points' + note + ':\n\n' + excerpt
  let summary = llmPredict({ prompt, options: { temperature: 0.2 } })
  return { url, truncated, summary }
}

The sample document is about 1,300 characters, so the default maxChars of 600 cuts it and truncated is true; pass maxChars: 8000 and it is sent whole. Counting characters is a rough stand-in for counting tokens, but it needs nothing from the host.

options is handed to the host's llm.predict as written; what it accepts depends on the host (see LLM Chat). A low temperature gives steadier, less inventive summaries.

This agent takes any url, so it cannot check its input against a known list as summarize does. The host's allowedFetchDomains is what keeps a caller from pointing it anywhere, and a URL on an allowed domain that does not exist still returns a 404 page, which the model will summarize.

When the world fails

A fetch can be refused or fail, and a model can be down. Both are failures of the world, so try/catch handles them, and the agent can return a readable reason instead of ending the run with run.error. The rule for httpFetch: a refused domain, a network failure or a body over the size cap throws; an HTTP error status does not, and returns the body.

function summarizeSafely({ url = 'https://tjs.tosijs.net/texts/ai-history.txt' }) {
  let text = ''
  try {
    text = httpFetch({ url })
  } catch (e) {
    return { url, error: 'Could not fetch: ' + e }
  }
  try {
    let summary = llmPredict({ prompt: 'Summarize in one sentence:\n\n' + text })
    return { url, summary }
  } catch (e) {
    return { url, error: 'The model failed: ' + e }
  }
}

The catch receives the error's message as a string, so it is + e, not + e.message (see Error Handling). Run it with the host above and a url on another domain, such as https://example.com/x, and error is "Could not fetch: Fetch blocked: domain not in allowlist. Allowed: tjs.tosijs.net". With no llm capability it is "The model failed: Capability 'llm.predict' missing".

What try does not catch: a URL that does not exist on an allowed domain. The fetch succeeds, returns the 404 page, and summarizeSafely summarizes it. Some failures also end the run whatever the agent does, try or not: running out of fuel or time, and bugs in the code such as calling an atom with the wrong input. The full list is in Error Handling.

Limits