parallel()

Basics#

parallel(id, { branches, join? }) hands the same input to every branch at once and waits for all of them. Branches are any nodes: asks, routes, whole chains. Without a join, the output is an object keyed exactly like branches, each value typed as that branch's output.

import { parallel, ask, choice, noul } from "jevchain";

const vibes = parallel("vibes", {
  branches: {
    tone: ask("tone", { questions: { tone: choice("Tone?", ["warm", "cold", "unhinged"]) } }),
    spam: ask("spam", { questions: { spam: noul("Is this spam?") } }),
  },
});

// OutputOf<typeof vibes>:
// {
//   tone: { tone: ChoiceAnswer<"warm" | "cold" | "unhinged"> };
//   spam: { spam: NoulAnswer };
// }

Every branch has to accept the parallel's input, so its input type is the intersection of what the branches want. Pass a string to a parallel whose branches expect { messages } and the compiler says so.

Joining results#

Give it a join to turn the pile of answers into one value. It runs after every branch finishes, receives the typed results, the original input and a StepContext, and its (awaited) return value becomes the parallel's output.

const vibes = parallel("vibes", {
  branches: { tone, spam },
  // results is typed from the branches. May be async; gets the input and a StepContext.
  join: async (results, input, ctx) => {
    ctx.log("joined", { spam: results.spam.spam.noul });
    return results.spam.spam.noul > 0.8 ? "block" : results.tone.tone.choice;
  },
});
// OutputOf<typeof vibes>: "block" | "warm" | "cold" | "unhinged"

const tribunal = parallel("sandwich-tribunal", {
  title: "The Sandwich Tribunal",
  branches: {
    taxonomy: ask("taxonomy", {
      questions: {
        is: choice("Structurally, what is this food?", {
          sandwich: "filling between two separate pieces of bread",
          taco: "filling in a single folded carrier",
          "soup-with-extra-steps": "mostly liquid, bread is a formality",
        }),
      },
    }),
    crime: ask("crime", { questions: { crime: noul("Would a reasonable chef call this a crime?") } }),
    structure: ask("structure", {
      questions: {
        holds: score("Will it survive being eaten with one hand?", ["collapses", "wobbles", "holds", "load-bearing"]),
      },
    }),
  },
  // Same state for all three asks → the client sends ONE request.
  join: (r) => ({
    ruling: r.taxonomy.is.choice,
    guilty: r.crime.crime.noul > 0.5,
    oneHanded: r.structure.holds.score >= 2,
  }),
});

Automatic batching#

Jev ingests the state once and answers every question about it in parallel, so three questions in one request is strictly cheaper and faster than three requests. The client exploits that for you: asks with the same model and the same state that are issued in the same tick are merged into one request. States are compared with a key-order-stable stringify, so { a, b } and { b, a } batch together.

Branches of a parallel all start synchronously, so any branch whose first move is a Jev call joins the batch. You don't configure anything; it's on by default.

In the trace#

Each merged call carries a batch record. usage.requests counts a merged request once, and the request's tokens are split evenly across the calls that shared it, so per-node cost still adds up.

// trace.spans[…].calls[0], for each of the three tribunal asks
"batch": { "id": "batch_1", "size": 3, "questions": 3 }

// trace.usage
{ "calls": 3, "requests": 1, "inputTokens": 212, "outputTokens": 3, "costUsd": 0.0000089 }

Tuning it#

// Off for a whole client…
const jev = createJev({ batch: false });

// …tuned…
const jev = createJev({ batch: { windowMs: 5, maxQuestions: 32 } });

// …or off for one direct call.
await jev.ask(state, questions, { batch: false });

When a branch fails#

Siblings share an AbortController. The first branch to fail aborts the rest, and the whole parallel fails with that first error. The rest are cancelled, not left running in the background.

• The run ends with status: "error", and error is a NodeError pointing at the innermost failing node (see Errors).
• Spans that were still running are closed in the trace with a cancelled error, so you can see what got cut off.
• A branch that halts (a gate with no otherwise) halts the whole run too. If you'd rather it didn't, give that gate an otherwise.