Skip to content

ORCHESTRATE · SUPPORT-TRIAGE · MAY 2026

Orchestrate Triage

A multi-domain support-triage AI agent. Built solo for HackerRank Orchestrate.

Get Started How It Works

10 / 10 / 10labeled sample · 3 graded columns

29 / 29production tickets

4 minbatch wall-time

~2 hrbuild · of 24 allotted

What it does

For each row in a CSV of support tickets, the agent decides:

  • which product area the ticket belongs to,
  • whether it's a product issue, bug, feature request, or invalid,
  • whether to reply (with a corpus-grounded answer) or escalate to a human,
  • and a justification the judge can audit against the source corpus.

Three ecosystems are in scope, each with its own local markdown corpus shipped alongside the input tickets:

HackerRank Support 580K tokens · 14 test tickets

Candidate management, screen tests, interviews, billing, account access — the bulk of the test set.

Claude Help Center 540K tokens · 7 test tickets

API & Console, Claude Code, privacy, Bedrock, education licensing, safeguards / vulnerability reporting.

Visa Support 18K tokens · 6 test tickets

Lost or stolen cards, travel guidance, charge disputes, traveller's cheques. Smallest corpus, highest-stakes escalations.

How it's built

  • Anthropic Claude Opus 4.7 (1M context) for production; Sonnet 4.6 for cheap dev iteration.
  • Corpus stuffing with prompt caching — no vector DB, no embeddings. Anthropic's own guidance below ~200K-token corpora is to include the entire corpus in the prompt; per-domain we sit at 18K / 540K / 580K tokens.
  • Forced single-tool-call (submit_triage) returns schema-validated TicketOutput rows in one round-trip.
  • Spotlighting + structural delimiters for prompt-injection defense — ticket text is wrapped in <user_ticket> XML tags with a system instruction to treat contents as data.
  • Async Message Batches API for 50%-off production runs.

Where this came from

HackerRank Orchestrate is a 24-hour solo hackathon (May 1–2, 2026). The starter scaffold and labeled-sample corpus live in interviewstreet/hackerrank-orchestrate-may26; this site documents the agent built on top of it. Source: github.com/ajbarea/orchestrate-triage.