AI Agent Engineer hiring kit

Every step to hire the engineer who builds agents that plan, act and recover on their own — and can prove the loop is safe to run.

About this kit — the 5Ws

An AI agent engineer builds systems where a model runs in a loop. It plans a step, calls a tool, checks what happened, then decides what to do next — rather than answering once and stopping. Knowing a popular agent framework matters less here than what happens when the loop goes wrong: limiting what the agent is allowed to touch, tracing a failure back to where it really started, and recognising when a simpler, fixed process would have been safer.

Calibrated for:: Someone who has already shipped production software — the bounding and tracing judgement this job needs is hard to build without that base.
Timeline:: Set an approval-to-offer target at intake, allowing for panel availability, candidate adjustments, references and any lawful checks.

Who: Engineering leaders and recruiters hiring their first or next AI agent engineer — the person who owns agents that plan, call tools and act over many steps, not just teams that already run one.
What: Use this kit to define evidence for safe agent ownership, test failure tracing and permission design, interview consistently, and measure how the hiring process performed once someone accepts.
When: Use it when a team needs someone to build or operate multi-step agents safely, running from role approval all the way through timing how the hire actually moved through each step.
Where: Technology teams operating or planning production agents.
Why: The work sample, structured questions and rubric let you compare job-related evidence consistently. Rather than compare framework buzzwords on a CV, you compare how each candidate traced a real failure back to its cause and decided what to bound before the agent ran again.

The competency model · 6

Use these competencies to compare job-related evidence. Review any apparent gap in a must-have before deciding.

Orchestration and loop designMust-have · 20
Can describe how an agent decides what to do next, when it calls a tool, and what makes it stop — and has built a loop that terminates reliably instead of wandering.
Boundary and permission designMust-have · 20
Gives each agent only the access its job needs (least privilege — the fewest permissions that still let it work), tests risky actions in an isolated practice environment first (sandboxing), and sets spend limits — all before the agent goes near anything that can't be undone.
Failure tracing and observabilityMust-have · 20
Records every step of a run — what the agent decided, which tool it called, what came back (this is called observability) — and can trace a failure back to where it actually started, even when the symptom shows up several steps later.
Multi-step evaluation designMust-have · 15
Builds an evaluation harness — a repeatable set of test scenarios that runs automatically and checks the results, instead of a person eyeballing each one — that judges an agent by what it actually did: the outcome, which tools it called, what it changed, whether it followed the rules, and whether it recovered when something went wrong, checked across many runs rather than one.
Cost and reliability engineeringCan learn · 10
Tracks spend and step counts per run, and sets retries that wait longer after each failed attempt (backoff) plus a hard cap on steps, rather than letting a stuck agent retry itself into a large bill.
Architecture judgement and communicationCan learn · 15
Can say plainly when a workflow beats an agent, and explains that call — and what an agent is doing — to people who aren't engineers.

What's required and what's optional

Your hiring journey for AI Agent Engineer. Select a stage to jump to it.

Required for every hire
Optional — use when it adds evidence
AI boost — a big drafting win
Colour = phase

Phase 1
Plan
1. 01Plan the roleRequired
2. 02Agree successRequiredAI boost
Phase 2
Attract
1. 03Write the jobRequiredAI boost
2. 04Find candidatesOptionalAI boost
Phase 3
Select
Phase 4
Decide
Phase 5
Onboard & learn
1. 12Help them settle inRequired
2. 13Measure time to hireRequiredAI boost

AI drafts, people decide: Use AI to draft, rewrite and summarise your own material — never to rank, score, screen out or decide on candidates, and never with a candidate's personal data.

Plan

Attract

Select

Decide

Onboard & learn

Solid = required · dashed = optional · arrow keys move between steps

Stage 1 of 13Required

0 of 3 done

Plan the role

Workforce planning and requisition approval

Who runs it: Hiring manager + finance / HR business partner
Done when: An approved requisition with a written business case, budget and a named decision-maker.

Key tool

Answer these before the role is approved

1Which agents will this person own in their first year, and which of them can already take real actions — sending something, changing a record, spending money?
2How would we find out today if one of those agents went wrong for a week before anyone noticed?
3Could an existing backend or platform engineer grow into owning an agent within two quarters?

AI as an advantage4D framework

Delegate: Hand AI the first draft of the business case and the build / borrow / buy comparison. The decision, the budget and the sign-off stay with people.

Show the prompt and the checks

Describe: You are helping a hiring manager write a business case for an AI Agent Engineer role. Context: I will paste the problem the hire solves, the work it involves, the capacity we already have and the time we have. Produce: (1) a one-sentence problem statement in business terms; (2) a build / borrow / buy comparison with the strongest argument for each; (3) the questions a finance partner would ask; and (4) a short recommendation with the evidence it rests on. Use British English and plain language. Do not invent figures — leave [brackets] where I need to supply numbers. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check every number and claim against your own notes and strike anything the draft assumed. Make sure 'buy' is argued from ongoing work, not from a job title.
Diligence: The hiring manager owns the final case and the budget request. No candidate information belongs here; AI drafts the case, people decide whether to open the role.

Don't miss: Opening this role because a demo agent impressed leadership, with no production agent yet worth the title.

Framework

Build · Borrow · Buy

Before opening a role, decide whether to build the capability in your current team, borrow it (contractor, agency, partner), or buy it (a permanent hire).

Write the problem the hire solves in one sentence, in business terms — not the job title.
Test Build: could an existing person grow into this within the time you have?
Test Borrow: is the need time-boxed, or does it end once a system is shipped?
Choose Buy only when the work is ongoing, core to the product and nobody inside can absorb it.
Record the decision, the budget and the one person who signs off the final hire.

Why it works: Writing the build, borrow or buy decision down exposes a vague or temporary need before it becomes a permanent requisition — and gives every later stage an anchor to check against.

• Name the final decision-maker — for example, whoever owns reliability or platform engineering for the team the agent sits in.

Is this a dedicated role?

Decide whether the work is sustained and substantial enough for a dedicated owner by writing down the workload, the risk, the capacity you already have and the outcomes you expect.

Answer these before the role is approved

1.Which agents will this person own in their first year, and which of them can already take real actions — sending something, changing a record, spending money?
Listen for: Named agents with real side effects today, or committed on the roadmap.
Red flag: 'We want agents to do more' with nothing specific.
2.How would we find out today if one of those agents went wrong for a week before anyone noticed?
Listen for: A specific detection method — alerting, a dashboard, a customer complaint — or an honest admission that nothing would catch it, which is the strongest case for this hire.
3.Could an existing backend or platform engineer grow into owning an agent within two quarters?
Listen for: A named engineer, a realistic path to operating an agent solo, or a clear reason none exists yet.
4.Is the need ongoing, or does it end once one agent ships?
Listen for: If new agents keep appearing and each needs bounding and evaluation built fresh, that's ongoing work for a permanent hire; a single agent that just needs hardening once may suit a contractor.
5.What does it cost us each month this role stays open?
Listen for: A concrete consequence: a stalled launch, a support backlog, spend nobody is watching.

Build, borrow or buy?

Build: give an existing backend or platform engineer a ring-fenced learning budget and a mentor who has operated an agent before.
Borrow: bring in a specialist contractor to bound and instrument your riskiest agent first, with a written hand-off plan so the knowledge doesn't leave with them after three months.
Buy: hire a permanent agent engineer once more than one agent needs continuous bounding, tracing and evaluation — not just a single agent hardened once and left alone.

Watch out for

Confusing this role with an AI engineer. If the hardest question is 'does our one model call return a good answer?', hire an AI engineer; if it's 'what happens when the loop runs off course?', hire an agent engineer.

Stage 2 of 13Required

0 of 2 done

Agree success

Intake meeting and competency model

Who runs it: Recruiter + hiring manager
Done when: A signed-off competency model: 5–7 competencies, each marked must-have or trainable, with weights.

Key tool

Ask the hiring manager

1Which of the systems or areas this person will own worries you most, and why?
2When something there went wrong recently, how did you find out?
3In 12 months, what will be shipped, fixed or running for you to call this hire a success? Agree two or three measurable outcomes.
4Who will they work with most closely?
5What could a strong candidate learn in their first three months? Those capabilities are 'trainable' — never use them to filter applications.

AI as an advantageAI boost4D framework

Delegate: Hand AI your raw intake notes to turn into a draft competency model. Choosing the must-haves and the weights stays with the hiring manager and recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter and a hiring manager build a competency model for an AI Agent Engineer role. Context: I will paste the notes from our intake meeting, including the 12-month outcomes and what the hiring manager worries about most. Produce a table of 5–7 observable competencies with, for each: a one-line definition; whether it is a must-have or trainable, with the reason; what evidence of it looks like in real work; and a suggested weight so the weights total 100. Use British English and inclusive wording, avoid years-of-experience or named employers as evidence, and keep every competency job-related. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each competency is observable and job-related, that nothing describes a type of person rather than the work, and that trainable items haven't been marked must-have.
Diligence: The hiring manager signs off the final model and weights before anyone meets a candidate. No candidate data goes into the prompt; AI drafts, people decide.

Don't miss: Listing a specific orchestration framework as a must-have. Frameworks turn over quickly; the underlying discipline doesn't.

Framework

Competency model (must-have vs trainable)

A short list of the observable capabilities the job needs, split into what the hire must bring on day one and what you will teach.

Start from the 12-month outcomes, then ask which capabilities produce them.
Keep 5–7 competencies. With more, interviewers struggle to score each one separately.
Mark each must-have or trainable. Only must-haves can count against a candidate.
Give each a weight so they sum to 100, agreed before anyone meets a candidate.
Write what evidence of each looks like, so interviewers know what they are listening for.

Why it works: Agreeing the criteria before meeting candidates stops the bar moving from one candidate to the next, and makes it harder for 'I'll know it when I see it' to decide the hire.

Core intake questions (ask for every role)

Book 45 minutes with the hiring manager before anything is advertised, and write the answers down.
Which of the systems or areas this person will own worries you most, and why?
When something there went wrong recently, how did you find out?
In 12 months, what will be shipped, fixed or running for you to call this hire a success? Agree two or three measurable outcomes.
Who will they work with most closely?
What could a strong candidate learn in their first three months? Those capabilities are 'trainable' — never use them to filter applications.
Who is on the interview panel, and is their time held for the next four to six weeks?

Questions specific to this role

1.Which of your current or planned agents can already take an action with real consequences — sending something, changing a record, spending money — and which are still read-only or supervised?
Listen for: A clear map of which agents carry real risk today. It shapes how urgent bounding and tracing work is, and which agent becomes the work sample.
2.Is this person taking over agents someone else already built, building new ones from scratch, or both?
Listen for: Scope of the job — inheriting undocumented systems calls for stronger tracing skill; building fresh calls for stronger loop-design judgement.
3.Do you already log what an agent decided at each step, or would this person be starting that from nothing?
Listen for: Whether observability already exists. It changes how much of the first months goes to instrumentation rather than new capability.

What success looks like in 12 months

Every agent that can take a real action has a spend ceiling, a step limit and a named owner.
At least one agent's failure rate is measurably reduced, with before-and-after numbers from a real evaluation set.
A run that fails does so loudly, with a trace that shows where it actually went wrong.
The team can say, for each agent, what it is allowed to do without asking a human first — and what it isn't.

Watch out for

Letting 'has trained or fine-tuned a model' sneak onto the must-have list — that's model-training work, not operating agents built on pre-built models.

Stage 3 of 13Required

0 of 3 done

Write the job

Job description and pay band

Who runs it: Hiring manager (content) + recruiter (publishing)
Done when: A published job description that states outcomes, the must-haves, how AI tools are used and the pay information the law requires.

Key tool

AI Agent Engineer

We're looking for an AI agent engineer to own the systems behind [product] that plan, call tools and act over many steps. You'll design the loop, bound what it's allowed to touch, build the evaluation harness that tells us when a run went wrong, and be the person we call when one does — working closely with [reliability/platform] and product.

What you'll do

•Design the agent's control loop: how it plans, when it calls a tool, and what makes it decide it's finished.
•Limit what each agent can do: give it only the access its job needs (least privilege), test risky actions in an isolated practice environment first (sandboxing), and set spend ceilings.
•Add logging to every run (instrument it) so a failure can be traced back to where it actually started, not just where it surfaced.
•Build evaluation sets — repeatable test scenarios that run automatically — for multi-step tasks, checking the outcome, the tool calls and the state changes across many runs.
•Control cost with step limits, backoff (retries that wait longer each time) and loop detection, and track spend per run.
•Advise the team on when an agent is the wrong tool and a plain workflow would be safer.

What you need

•You have shipped and operated at least one agent in production — a loop that ran against real traffic, not only a demo.
•You can show how you traced a failure back to its real cause, including one that surfaced several steps after it started.
•You have designed tool or permission boundaries so an agent can't take an action it shouldn't.
•You write tested, production-grade [Python/TypeScript] — the language matters less than whether it ships safely.

Nice to have

•Experience building evaluation harnesses for multi-step or non-deterministic systems.
•Experience with hand-offs between agents, or between an agent and a human reviewer.
•On-call or reliability ownership for a system with unpredictable load.
•Some awareness of AI-specific regulation, such as the EU AI Act, and how it might touch an autonomous agent.

How we use AI tools

•Where company policy permits, use approved AI tools and verify their output — including when you're building the very agents this role owns.
•You'll recognise when handing a task to an agent — instead of a simpler scripted step — adds risk without adding value, and say so before building it.
•For the work sample, you're free to reach for the AI tools you'd use on the job — note in your submission which ones and where.

Pay

Salary range: [currency, minimum–maximum], plus [equity/bonus/benefits]. The range reflects [location/working pattern]. We place every offer within it using the same pre-set, job-related, gender-neutral factors — level, scope of agent ownership — how many agents, how much autonomy each is given, depth of production-incident experience — never previous salary or negotiating style.

AI as an advantageAI boost4D framework

Delegate: Hand AI the first draft of the job description from your signed-off competency model. The must-haves, the pay range and the final wording stay with people.

Show the prompt and the checks

Describe: You are helping write an outcome-based job description for an AI Agent Engineer role. Context: I will paste the competency model (must-have and trainable, with evidence), the 12-month outcomes, how the team uses AI tools, the location and working pattern, and the pay range. Produce: a summary paragraph that leads with the problem and what the hire will own; 'What you'll do' as outcomes, not tasks; 'What you need' containing only the must-haves; 'Nice to have' for the trainable items; 'How we use AI tools'; and a pay section that states the range. Use British English and inclusive, gender-neutral wording, avoid hype, and replace any 'X years' with the capability it stands for. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that 'What you need' contains only the must-haves, that no degree, employer or years requirement crept in, and that the pay range and location are exactly what was approved.
Diligence: The hiring manager owns the content and the recruiter owns publishing, including the pay information the law requires for each location. AI drafts the advert; people decide what it says.

Don't miss: Asking for years of experience with a named agent framework. Most of these frameworks are younger than the requirement implies.

Stage 4 of 13Optional

0 of 3 done

Find candidates

Sourcing strategy

Who runs it: Recruiter / sourcer
Done when: A shortlist-ready pipeline from at least three channels, with outreach tracked.

Use when:: Inbound applications won't give you enough qualified candidates.
Skip when:: Your shortlist already comes from inbound applications or referrals.

Key tool

Search query

("agent" OR "agentic" OR "AI agent" OR "autonomous agent") AND ("orchestration" OR "tool calling" OR "multi-step" OR "agent loop") AND ("production" OR "shipped" OR "operated") AND (Python OR TypeScript) NOT ("intern" OR "student")

First outreach message

AI as an advantageAI boost4D framework

Delegate: Hand AI the search queries and the first draft of the outreach template. Choosing channels, personalising to a candidate's public work and sending stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter source candidates for an AI Agent Engineer role. Context: I will paste the must-have competencies, the kind of work we want evidence of, the channels we are using and the pay range. Produce: (1) one search query per channel, using OR for alternatives, AND for things that must all appear and NOT to exclude, with a one-line note on what each query looks for; and (2) a first-outreach template of under 120 words that opens with a [placeholder for something the candidate built], states the problem we are hiring to solve and the pay range, and ends with a low-pressure question. Use British English and inclusive wording. Do not refer to any specific person. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the queries search for evidence of the work rather than job titles alone, and that the template claims nothing about the role or the pay you can't back up.
Diligence: The recruiter personalises every message to something the candidate actually built, and sends it. No names or profile data go into the prompt; AI drafts, people decide whom to contact.

Don't miss: Search for agent-operation evidence across backend, platform and agent-engineer titles.

Framework

Channel mix + evidence-first outreach

Spread sourcing across channels where the work is visible, and open outreach with the problem, not the perks.

Pick channels where candidates show their work, not only where they list job titles.
Write one search query per channel. OR joins alternatives, AND joins things that must all appear, NOT excludes. Keep the version that works best.
Personalise the first line to something the candidate actually built or wrote.
Lead with the problem and the pay range. Save the perks for later.
Track reply rates by channel each week and drop the channel that isn't working.

Why it works: Searching for evidence of the work can surface candidates whose current title differs from the vacancy, or whose CV undersells what they have done.

Sourcing basics

Adapt the search query for each platform — the same words behave differently on each site.
Candidates with adjacent titles may have relevant evidence. Search for the work and outcomes the role needs, not the current title.
Channel activity is a lead, not evidence. Review linked work and examples against the written competencies.

Where to look

Public code repositories and incident write-ups

A public post-mortem lets you review how the person traced and fixed a real agent failure, not just which framework they list.

Conference talks and engineering blogs on agent reliability

A talk or write-up on bounding or debugging agents lets you review the specific failure modes the person has dealt with.

Your own engineering team's referrals

A colleague who ran agent incidents alongside the candidate can point you to specific traces or write-ups to review.

LinkedIn and similar networks, searched for agent operations and incident language rather than job titles

Search by the operational work described in a profile rather than the job title, since the title alone will not show it.

Communities around orchestration and agent tooling

In these spaces, look for a name attached to a specific fix or a working example, not just active participation.

Boolean search string

("agent" OR "agentic" OR "AI agent" OR "autonomous agent") AND ("orchestration" OR "tool calling" OR "multi-step" OR "agent loop") AND ("production" OR "shipped" OR "operated") AND (Python OR TypeScript) NOT ("intern" OR "student")

First outreach message

Hi [name] — [specific detail from their post, repo or incident write-up] caught my eye. At [company], our agents [describe your specific failure mode — e.g. 'occasionally repeat an irreversible step before anyone notices']. We're building out the role that owns bounding and fixing that, paying [range]. Open to a 20-minute conversation about what that problem actually looks like day to day?

Stage 5 of 13Required

0 of 3 done

Check applications

Application screening

Who runs it: Recruiter
Done when: Every application reviewed against the same written knockouts within five working days.

Key tool

Reject only if one of these is missing

•Can meet the role's work-authorisation requirements by the start date, with sponsorship where the employer offers it. Don't ask for documents at this stage.
•Describes at least one agent — a system that plans, calls tools and acts over multiple steps — that reached real production traffic.
•Has shipped production software in any language — the agent stack matters less than the engineering discipline.

AI as an advantage4D framework

Delegate: Hand AI the job of turning your written knockouts into a consistent checklist with reason codes. Reviewing each application stays with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter prepare an application-screening checklist for an AI Agent Engineer role. Context: I will paste the must-have competencies and our written knockout criteria. Produce: (1) each knockout as a yes/no question answerable from an application alone; (2) a short list of reason codes for a 'no'; and (3) a note of any criterion that looks like a proxy — a degree, employer brand or employment gap — rather than a job-related requirement, with a suggested rewrite. Use British English. Do not evaluate any applicant and do not ask for applicant details. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each question is binary and job-related, and that every flagged proxy has been removed or justified in writing.
Diligence: A person reviews every application against the checklist and records the reason. Never paste CVs or applicant details into an AI tool, and never let a tool screen anyone out — in New York City and the EU that can bring automated-decision rules into play.

Don't miss: Scoring a candidate down for having built agents with a different orchestration framework than the one your team uses — ask whether they can explain the failure modes, not whether they know this specific tool.

Framework

Knockout criteria

A small set of yes/no requirements, written in advance, applied to every applicant the same way.

Derive knockouts only from must-have competencies and genuine legal requirements.
Keep them binary and answerable from the application itself.
Apply them in the same order to every application and log the reason for each rejection.
Do not use proxies such as university, employer brand or employment gaps as knockouts.
Anything ambiguous gets a second review or moves forward to the skills stage.

Why it works: Early screening rejects people before richer evidence exists, so its criteria need the most care. Written criteria make decisions reviewable — but each one must also be job-related, applied consistently and checked for unjustified adverse impact.

Screening rules

Do not use degrees, employment gaps or certifications as filters unless they are demonstrably job-related and justified for this role.
Judge the claims in a CV, not how polished it is — a well-written CV may or may not have had AI help, and either way it is the evidence that matters.

Knockout criteria

Can meet the role's work-authorisation requirements by the start date, with sponsorship where the employer offers it. Don't ask for documents at this stage.
Describes at least one agent — a system that plans, calls tools and acts over multiple steps — that reached real production traffic.
Has shipped production software in any language — the agent stack matters less than the engineering discipline.

Green flags

Describes a specific failure and how they traced it back to its real cause.
Mentions bounding what an agent could do before it went live — permissions, sandboxing or spend limits.
Links to code, an incident write-up or a demo you can check.

Red flags

Rattles off five orchestration frameworks by name but can't describe a single run that actually failed in front of a real user.
Describes only prompt-chaining — a fixed sequence of model calls — as if it were an agent loop.
General claims about agents the candidate can't tie back to one specific project.

Watch out for

Treating five listed orchestration frameworks as a strong signal on its own — breadth of framework names does not show whether any of them were operated under real load.

Compliance checkpointNew York CityEU

For a New York City employment decision, work out whether a tool is an 'automated employment decision tool' (AEDT) under Local Law 144 — one that uses covered techniques to substantially assist or replace a discretionary decision. Before using a covered tool, make sure an independent bias audit was completed no more than a year earlier, publish the required summary, and give covered candidates or employees the required notice at least ten business days before use. In the EU, the AI Act (Annex III) lists AI used to recruit, filter applications or evaluate candidates as high-risk, subject to the Article 6(3) exception; the related obligations apply from 2 December 2027 under the amended timetable. A manual screen does not exempt an AI tool used elsewhere in the process.

Sources: NYC DCWP — Automated employment decision tools · EU AI Act — consolidated Regulation (EU) 2024/1689 · checked 2026-09-25

Stage 6 of 13Optional

0 of 3 done

Recruiter call

Recruiter screen

Who runs it: Recruiter
Done when: A 20–30 minute call that confirms logistics, motivation and one or two must-have signals.

Use when:: You are hiring at volume, or pay and logistics need confirming early.
Skip when:: You have only a handful of applicants — fold the pay and logistics questions into application screening instead.

Key tool

Ask every candidate, in this order

1Tell me about the most recent agent you built or operated. What could it do on its own?
2Tell me about a time one of your agents went wrong. How did you find out?
3What's the most autonomous action you've let an agent take without a human checking it first? What made that safe?
4Our range for this role is [range]. Does that work for you?

AI as an advantage4D framework

Delegate: Hand AI the first draft of the screening-call script and its evidence anchors. Running the call and taking the notes stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter write a 20–30 minute structured screening call for an AI Agent Engineer role. Context: I will paste the must-have competencies, the logistics we need to confirm (location, working pattern, notice period) and the pay range. Produce: an opening that explains the process, the timeline and the pay range; 4–6 questions in a fixed order, each with one line on what a strong answer contains; and a closing that asks whether the published range works. Use British English and inclusive wording. Do not include questions about salary history, age, family, health or any other protected characteristic. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that every question is job-related and lawful, that the pay question asks whether the range works rather than for history, and that the anchors describe evidence, not impressions.
Diligence: The recruiter asks the same questions of everyone and writes the notes. No candidate details go into the prompt; AI drafts the script, people run the call and decide.

Don't miss: Expecting the recruiter to judge deep technical depth. Their job here is motivation, logistics and whether the candidate can describe their own work clearly — the trace-debugging round does the rest.

Framework

Structured screening call

The same short set of questions for every candidate, with notes taken against what a strong answer contains.

Open by explaining the process, timeline and pay range. No surprises later.
Confirm logistics: location, working pattern, notice period and whether the range works for them.
Ask the same 4–6 questions in the same order, and note evidence, not impressions.
Leave time for the candidate's questions. Don't score their questions unless they test a written, job-related competency.
Decide within 48 hours and tell the candidate either way.

Why it works: Using the same questions and written evidence anchors makes answers easier to compare, and a short, predictable call respects the candidate's time.

Screening call basics

Set the decision timetable before sourcing and tell candidates when to expect each update.
Ask every candidate what draws them to this particular role, and listen for interest in the work described rather than the title.
Confirm notice period and whether they are in other processes, so you can plan the timetable.
End every call by asking whether the published pay range works — never ask for salary history.

Recruiter screen questions

1.Tell me about the most recent agent you built or operated. What could it do on its own?
Listen for: A specific agent, real side effects and the candidate's own part in it.
Red flag: Talks only about 'the team' and can't point to a decision they personally made on the agent.
2.Tell me about a time one of your agents went wrong. How did you find out?
Listen for: Any structured signal — an alert, a trace, a dashboard — not 'a customer complained and we guessed'.
Red flag: 'It looked fine in testing' or 'users didn't complain'.
3.What's the most autonomous action you've let an agent take without a human checking it first? What made that safe?
Listen for: A concrete, bounded decision — least-privilege access, a spend cap, a sandboxed trial — not 'we trusted the model'.
Red flag: Can't name a specific action, or describes giving an agent broad access with no limit.
4.Our range for this role is [range]. Does that work for you?
Listen for: A clear answer on whether the published range works. Don't ask for salary history — this process sets pay from the published band and pre-set, job-related factors.

Compliance checkpointUS (state and city)EU

Don't ask for salary history. In the EU, national laws implementing Article 5(2) of Directive (EU) 2023/970 must prohibit the question. US rules vary, but jurisdictions including California and New York prohibit employers from seeking salary history. Ask whether the published range works instead.

Sources: EU Pay Transparency Directive 2023/970, Article 5 · California Labor Commissioner — salary history · New York State DOL — salary history ban · checked 2026-09-25

Stage 7 of 13Required

0 of 3 done

Work sample

Skills assessment / work sample

Who runs it: Hiring manager (design) + recruiter (logistics)
Done when: Every shortlisted candidate completes the same job-shaped task, marked against criteria written in advance.

Key tool

Bound and fix a looping onboarding agent

You're given a small agent that walks new users through account setup: it creates an account, sends a verification email, and provisions a starter workspace, calling each tool in turn. A run log shows it sometimes repeats the email step several times before finishing, and one run never finishes at all. Read the trace, find where the loop actually goes wrong (it isn't where it finally gives up), fix it, and add one boundary so a future bug like this can't send a real user ten emails.

Planned for up to three hours (adjustments available on request). At the agreed stopping time, write a short note on what you would examine next, and what you deliberately left out.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the work-sample brief and its marking sheet. Choosing the task, setting the timebox and marking stay with the hiring manager.

Show the prompt and the checks

Describe: You are helping a hiring manager design a work sample for an AI Agent Engineer role. Context: I will paste the competency model, a real problem the team solved recently with anything confidential removed, the timebox, and our AI-tool policy for candidates. Produce: a candidate-facing brief (what to do, what to hand in, the time cap, how to request an adjustment, which AI tools are allowed and how to report their use); and a marking sheet with 4–6 criteria tied to named competencies, each with what a 1, a 3 and a 5 look like. Use British English and plain, inclusive wording. Do not design anything that needs more than the stated time. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the task fits the timebox, that every criterion maps to a competency, and that the brief gives all candidates the same starting materials.
Diligence: The hiring manager owns the task and the criteria, and people mark every submission. Do not paste candidate submissions into an AI tool for scoring; AI drafts the brief, people mark and decide.

H-EvaluateH-Evaluate can generate a job-specific assessment from your job description, so you don't have to build the task by hand.How H-Evaluate helps →

Don't miss: Asking candidates to design a brand-new agent from a blank page. It rewards whoever has the most free evenings, not whoever is best at bounding and fixing a loop that's already misbehaving — which is the actual job.

Framework

Work-sample test

A small, realistic slice of the job, marked against criteria written in advance. Give candidates equivalent materials and arrange reasonable adjustments where required.

Take a real problem the team solved recently and shrink it to fit the timebox.
Write the marking criteria first, tied to the competency model.
Timebox honestly — for take-home work, a few hours at most — and say so in the brief.
State the AI-tool policy in writing. Where the job permits these tools, use equivalent conditions in the assessment and evaluate how candidates verify their output.
Mark without CV details where you can, and have two people mark borderline submissions.

Why it works: A well-designed work sample gives direct evidence of job-relevant skill rather than a proxy for it, and gives candidates a realistic preview of the work.

Assessment rules for every candidate

State the planned time cap and how to request an adjustment. At the agreed stopping time, ask candidates to note what they would examine next and what they left out — never reward unpaid overtime.
Give every candidate the same starting materials, and use a second marker for borderline submissions.
Say in writing which AI tools are allowed and ask candidates to tell you how they used them.
Invite every candidate to ask for adjustments, and answer each request promptly.
Pay for anything the task costs the candidate to run, such as model or cloud usage.

The work sample

Bound and fix a looping onboarding agent

Timebox: Planned for up to three hours (adjustments available on request). At the agreed stopping time, write a short note on what you would examine next, and what you deliberately left out.

What the candidate hands in

Your account of where the loop actually goes wrong, and why the symptom appears later than the cause.
The corrected code, with a short note explaining why each specific change was needed to stop the loop repeating.
One boundary you added so the same class of bug can't repeat an irreversible action.

Marking criteria (write these first)

The root cause is correctly identified, not just the step where the run finally failed.
The fix addresses that cause rather than papering over the symptom.
The added boundary is concrete and would actually stop the repeat action, not just log it.
The candidate's reasoning is visible — they explain why, not only what.
A teammate could pick up the fixed code and extend it to the next agent without needing the candidate to explain it in person.
Their closing note about what's still unchecked shows genuine judgement about priority, not just a list of leftover tasks.

Adjustments: Candidates can ask for extra time, or to work through it live with an interviewer instead. We cover the model usage — the starter repository ships with a spend-limited key.

Watch out for

Banning AI tools outright. Assess whether the candidate checks what the tools produce, not whether they used any at all.

Compliance checkpointUSGreat BritainEU

In the US, covered employers must provide a reasonable accommodation to a qualified applicant with a disability unless it would cause undue hardship (ADA). In Great Britain, the Equality Act 2010 requires reasonable adjustments in recruitment. In the EU, the duty comes from national equality law. Don't ask for more medical information than the law allows.

Sources: EEOC — prohibited practices · Equality Act 2010 · checked 2026-09-25

Stage 8 of 13Required

0 of 3 done

Structured interviews

Structured interview loop

Who runs it: Interview panel (named in advance)
Done when: Every competency has a lead interviewer and at least one second source of evidence, with the same questions for every candidate.

Key tool

Interview questions by pillar

Free in this kit: Domain questions for every round. The other three pillars and the AI-fluency check are measured by H-Evaluate.

Free in this kit

Senior engineer who marked the work sample · Failure tracing and observability · Multi-step evaluation design

1.Three traces come back from the same agent: two fail after a tool timeout, and one returns a stale customer record instead of failing outright. Which would you dig into first, and what in each trace would change your first guess at the cause?
Failure tracing and observability
Good answerSeparates the two failure classes instead of treating them as one bug, picks a starting point based on evidence (the stale-record case, since it fails silently), and names a specific signal in the trace — not just 'look at the logs' — that would redirect the investigation.
Weak answerTreats all three as the same kind of failure, or jumps straight to a fix before describing what they'd check.
2.An onboarding agent fails intermittently and you can't reproduce it on demand. What would you check first, and what would you add to the system so the next occurrence is easier to catch?
Failure tracing and observability
Good answerChecks or adds instrumentation before changing behaviour, looks for a pattern across the failed runs — timing, input shape, which tool was involved — and proposes a concrete addition, such as a per-step trace ID, rather than 'add more logging' in general.
Weak answerStarts changing code to see if the failure stops, or can't name anything they'd add to catch it next time.
3.Pick one multi-step task — say, the support-triage agent deciding whether to answer or hand off. How would you build an evaluation for it, given more than one handling of a ticket could be acceptable?
Multi-step evaluation design
Good answerScores runs on what's observable — the final state, which tools were called and in what order, whether the hand-off rule was followed — rather than a single expected transcript, and describes how they'd automate that check across many runs.
Weak answerAssumes one correct transcript exists and checks for an exact match, or can't say how the scoring would actually run without a person reading each transcript.

Optional follow-ups (1)

4.Here's an evaluation set that scores the triage agent only on its final reply. The agent also calls tools and updates ticket records along the way. What would you add to the evaluation, and why those additions specifically?
Multi-step evaluation design
Good answerNames concrete additions — checking which tools were called, whether record updates matched policy, whether an irreversible action happened without the required hand-off — each tied to a specific thing the current scoring would miss.
Weak answerSays the evaluation 'needs more test cases' without naming what the new checks would actually examine.

AI as an advantageAI boost4D framework

Delegate: Hand AI the drafting of probe questions and strong / weak anchors for one competency. Asking, listening and scoring stay with the interviewer.

Show the prompt and the checks

Describe: You are helping an interviewer prepare a structured technical interview for an AI Agent Engineer role. Context: I will paste one competency, its evidence description and the core question we ask every candidate. Produce: four probes in this fixed order — what would you do, why that choice, what else did you consider, how would you know it worked — tailored to the question; a one-line description of a strong answer and of a weak answer; and two follow-ups to use only if time allows. Use British English and inclusive wording. Keep everything about the work, nothing about the person. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the probes test the named competency rather than trivia, that the anchors describe observable evidence, and that no question touches a protected characteristic.
Diligence: Interviewers ask the same core questions of every candidate and score alone against the rubric. Interview notes and candidate names never go into an AI tool; AI drafts the questions, people interview and decide.

H-EvaluateH-Evaluate can add evidence on Cognitive, Situational judgement, Behavioural and AI fluency for your panel to review alongside these interviews.How H-Evaluate helps →

Don't miss: Scoring the trace-debugging round on whether the candidate found the exact same root cause you did, rather than whether their diagnostic process held up.

Framework

Structured technical interview

Every candidate gets the same job-specific questions, each interviewer leads on specific competencies, and every answer is probed with the same four follow-ups: what would you do, why, what else did you consider, and how would you know it worked.

Give each competency a lead interviewer, plus a second source of evidence — the work sample, another round or an independent marker. Avoid asking the same thing twice, but never let one person's view be the only evidence for a critical competency.
Ask the three core questions for your round, in order, for every candidate. Use an optional follow-up only if there is time.
Probe each answer: why that choice, what else they considered, and how they would know it worked — until you hear the candidate's own technical decision.
Take notes on evidence during the interview. Don't rate until it's over.
Fill in your interviewer sheet and submit it before you talk to any other interviewer.

Why it works: Structured interviews make answers comparable: every candidate faces the same job-related questions and is scored against the same anchors.

Interview rules

Use the same core questions in the same order for every candidate.
In the work-sample review, understand the candidate's decisions against the rubric — don't turn it into a debate about your preferred solution.
Avoid trivia disconnected from a real job decision. Use a realistic debugging, design or evaluation problem instead.

Watch out for

Letting an answer drift into a general career story — keep steering back to the specific decision the candidate made, the evidence behind it, and what they'd check or change next.

Compliance checkpointUSUKEU

Don't ask about protected characteristics — such as age, disability, pregnancy or family plans, religion, race or nationality — unless applicable law permits the question for a specific, necessary, job-related purpose, and HR or counsel has confirmed that basis beforehand. Ask only whether the candidate can meet the role's work-authorisation requirements. If a candidate volunteers protected information, don't probe; record only what is lawfully needed, keep it separate and restrict access.

Sources: EEOC — prohibited practices · Equality Act 2010 · checked 2026-09-25

Where to find more interview questions

Use web question banks for inspiration only: ask every candidate the same job-related questions, check each one against this kit's competencies, and never ask about protected characteristics. Don't reuse this kit's work sample from a public bank — candidates can find questions that circulate online, so a widely posted question tests preparation more than skill.

Show 7 sites

US OPM — Structured interviews(opens in a new tab)
US federal guidance on building structured interviews, with a guide, training material and sample questions with rating scales.
FreeFor interviewers
Acas — Interviewing job applicants(opens in a new tab)
UK public guidance on preparing fair interviews, asking applicants the same questions and avoiding discriminatory ones.
FreeFor interviewers
Workable — Interview questions library(opens in a new tab)
Sample interview questions for employers, listed by job role, including machine learning, data and engineering roles.
FreeFor interviewers
Indeed for Employers — Interview questions(opens in a new tab)
Sample interview questions for employers by job title and skill, with notes on what to look for in answers.
FreeFor interviewers
Tech Interview Handbook(opens in a new tab)
Open-source software-engineering interview preparation: coding, system design and behavioural questions.
FreeFor candidates and interviewers
Chip Huyen — Machine Learning Interviews Book(opens in a new tab)
Free online book with knowledge questions on maths, computer science and machine learning, grouped by difficulty.
FreeFor candidates and interviewers
Blind — Interview questions channel(opens in a new tab)
Anonymous posts by tech workers about questions they report being asked. Unverified; some content needs sign-in.
Free, some content needs sign-inCandidate reports, unverified

Links checked 2026-10-05. External sites; we don't control their content.

Stage 9 of 13Required

0 of 2 done

Compare evidence

Scoring, calibration and debrief

Who runs it: Hiring manager (chairs) + panel
Done when: A documented hire / no-hire decision traceable to scored evidence against the competency model.

Key tool

Interactive scorecard

Score one candidate against the rubric. Everything stays in your browser. Download it as a spreadsheet for your records.

Orchestration and loop design Must-have

Weight

What do 1, 3 and 5 look like?

1: Can wire tool calls together but can't say how the agent decides it's finished; loops or stops arbitrarily.
3: Builds a control loop with basic termination conditions; needs prompting to consider edge cases.
5: Designs termination, retries and hand-offs deliberately, and can explain what happens when the model disagrees with itself.

Boundary and permission design Must-have

Weight

What do 1, 3 and 5 look like?

1: Lets an agent act with whatever access it was given, with no separate limits.
3: Adds permission checks when asked; covers the obvious irreversible actions.
5: Designs least-privilege access, spend ceilings and an irreversible-action list unprompted, before the agent goes live.

Failure tracing and observability Must-have

Weight

What do 1, 3 and 5 look like?

1: Fixes the step where the run stopped without checking whether that's where the problem started.
3: Can trace a failure with guidance; catches issues that are close to where they surface.
5: Instruments runs so failures are traceable by design, and reads backwards from the symptom to the real cause on their own.

Multi-step evaluation design Must-have

Weight

What do 1, 3 and 5 look like?

1: No evaluation, or checks only whether the final output 'looks right'.
3: Puts together an evaluation covering the obvious failure mode once asked, but doesn't think to check less common paths through the loop.
5: Builds the evaluation before being asked, judges runs on outcome, tool calls and state changes across many repeats, and reports regressions honestly.

Cost and reliability engineering Can learn

Weight

What do 1, 3 and 5 look like?

1: No idea what a run costs or how long it takes.
3: Tracks cost and step count; knows the common levers for reducing them.
5: Designs backoff, step limits and loop detection on purpose, and checks any cost change against the evaluation.

Architecture judgement and communication Can learn

Weight

What do 1, 3 and 5 look like?

1: Can't explain agent behaviour without jargon; treats every problem as agent-shaped.
3: Gives clear explanations when asked; occasionally questions whether an agent is the right tool.
5: Explains limits honestly to any audience, and argues against building an agent when a workflow would do — unprompted.

Weighted score– / 5

AI as an advantage4D framework

Delegate: Hand AI the drafting of a blank debrief worksheet and agenda from the competency rubric alone. People add the evidence locally; the scores and the decision stay with the panel.

Show the prompt and the checks

Describe: You are helping a hiring manager prepare a debrief for an AI Agent Engineer role. Context: I will paste the competency rubric only — the competencies, their weights and their 1, 3 and 5 anchors. Produce: (1) a blank debrief worksheet with one row per competency and empty columns for each interviewer's independent score, the evidence they cite, the spread of scores and an 'evidence gap?' box; (2) a 45-minute agenda: confirm every scorecard is in and reveal all scores at once, discuss the widest spreads using evidence only, review any must-have scored 1, then look at the weighted total and decide (hire, no hire or one targeted follow-up), and record the reasons; and (3) neutral questions the chair can ask to draw out evidence rather than adjectives. Leave every evidence and score cell empty. Do not suggest a score, a rating or a recommendation. Use British English. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the worksheet follows your rubric exactly — same competencies, weights and anchors — and that the agenda keeps independent scores hidden until the reveal.
Diligence: Interviewers fill in the evidence and scores locally, alone, before the debrief, and the panel decides together. Candidate notes and personal data never go into an AI tool; AI never scores, ranks or compares candidates.

H-EvaluateH-Evaluate gives you a report on each candidate's assessment evidence to bring into this debrief.How H-Evaluate helps →

Don't miss: Averaging away a weak score on boundary and permission design because the weighted total still clears the bar.

Stage 10 of 13Optional

0 of 2 done

Check references

References and background checks

Who runs it: Recruiter (references) + HR (checks)
Done when: At least two references that test specific competencies, plus only the checks that are lawful and relevant to the role.

Use when:: It is lawful where you hire, and specific evidence from a referee would change the decision.
Skip when:: Local law or company policy rules it out.

Key tool

Ask each referee

1What's one thing [candidate] built or fixed on an agent that's still running in production without anyone touching it since?
2When they changed how an agent behaved, how did they check it was actually better and not just different?
3How did they handle limits on what an agent was allowed to do?

AI as an advantage4D framework

Delegate: Hand AI the drafting of reference questions aimed at the competency the panel was least sure about. Choosing referees, making the calls and interpreting the answers stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter write competency-based reference questions for an AI Agent Engineer role. Context: I will paste the competency model and the one or two competencies where our evidence is thinnest — no candidate details. Produce: 3–4 questions per competency that ask for specific examples ('tell me about a time…') rather than ratings; an opening question about how the referee worked with the person and for how long; and a closing question about what their next manager could do to help them do their best work. Use British English and inclusive wording. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the questions ask for examples rather than opinions, that they stay on job-related competencies, and that none asks about health, family or other protected matters.
Diligence: References are taken with the candidate's consent and interpreted by people. Referee answers and candidate details never go into an AI tool; AI drafts the questions, people decide.

Don't miss: Asking 'would you hire them again?' and stopping there — a friendly yes isn't evidence about how they'd handle a live agent incident.

Framework

Competency-based referencing

Treat references as a last structured conversation about the candidate's work, aimed at the competencies where your evidence is thinnest.

With the candidate's consent, ask for two people who saw their relevant work directly — ideally a former manager and a peer.
Accept suitable alternatives if a manager or peer isn't available, and never penalise a candidate for a former employer's no-reference policy.
Aim your questions at the one or two competencies the debrief was least sure about.
Ask for examples, not ratings. 'Tell me about a time they…' works here too.
Set the timing of each background check by the law and policy that apply.

Why it works: Generic reference questions tend to produce little job-related evidence. Asking for specific examples tied to the role's competencies turns referencing into something you can use.

Core reference questions and checks

Open with: 'How did you work together, and for how long?' — it tells you how much weight the rest can carry.
Ask: 'Tell me about a time something they owned went wrong. What did they do?'
Close every reference with: 'What should their next manager do to help them do their best work?'
Verify only claims that were material to the decision — don't verify a degree the role never required.
Run criminal-record checks only where the role justifies them and local law allows; many roles don't need one.
Schedule formal work-authorisation checks at the legally required time: before employment starts in the UK; for US Form I-9, Section 1 by the first day of paid work and Section 2 within three business days.

Reference questions specific to this role

1.What's one thing [candidate] built or fixed on an agent that's still running in production without anyone touching it since?
Listen for: Concrete, lasting work — not general praise.
2.When they changed how an agent behaved, how did they check it was actually better and not just different?
Listen for: A second source confirming they actually tested changes before calling them improvements.
3.How did they handle limits on what an agent was allowed to do?
Listen for: Examples of proactively bounding scope, not just reacting to an incident.

Compliance checkpointUS

If a consumer reporting agency provides a background report, the Fair Credit Reporting Act requires a clear, stand-alone written disclosure and the candidate's written authorisation before you order it. Before rejecting someone because of the report, give them a copy and the FCRA summary of rights and a reasonable opportunity to review and dispute it before any adverse action; if you still reject, send a final adverse-action notice. State and local 'fair chance' laws may delay criminal-history questions or checks until after a conditional offer.

Sources: FTC — Background checks: what employers need to know · FTC — Using consumer reports: what employers need to know · checked 2026-09-25

Stage 11 of 13Required

0 of 2 done

Agree the offer

Offer, negotiation and close

Who runs it: Hiring manager + recruiter
Done when: An accepted written offer within the approved band, with a start date.

Key tool

Build the offer around

•The problem: they'll own what happens when an agent's loop goes wrong, not just when the demo goes right.
•Budget for experimentation: a monthly model and tool-call allowance they control.
•Visibility: budget to speak at a conference on agent reliability, and time to write it up publicly.
•A clear level rationale: why they're slotted here and exactly what the next level up would need to see.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the offer message built around what the candidate said matters to them. The figure, the approval and the conversation stay with the hiring manager.

Show the prompt and the checks

Describe: You are helping a hiring manager draft an offer message for an AI Agent Engineer role. Context: I will paste three general priority categories (for example: learning, flexibility, scope) — not the candidate's words, notes or personal details; the parts of the offer; and the start-date options — without the salary figure. Produce: a warm, plain-English message that links each part of the offer to one of their priorities, says what happens next and when the written offer follows, and invites questions. Use British English and inclusive wording, and leave [brackets] for figures and names. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that nothing promised is outside policy or the approved band, that every claim about the role is true, and that the message reads as personal rather than templated.
Diligence: The hiring manager places the offer in the band using the same pre-set factors for everyone and makes the verbal offer personally. Figures and personal details are filled in by people, never pasted into AI.

Don't miss: Leaving ownership and autonomy unclear in the offer.

Framework

Close plan (motivators → offer → objections)

Build the offer around what the candidate told you they care about, anticipate their objections, and make sure the verbal and written offers match.

List the candidate's top three motivators from your notes at every stage.
Make the verbal offer personally, and link each part of it to a motivator.
Ask what else they are weighing and when they have to decide.
Place the offer in the band using the same pre-set, job-related factors for everyone, and record why.
Send the written offer promptly after the verbal one.

Why it works: A close plan ties the offer to the candidate's stated priorities and makes approvals explicit before the final interview.

Offer checklist and negotiation

Stay inside the approved band. If the candidate is worth more, re-level them with evidence rather than stretching the band.
Trade in things that cost less than salary where policy allows: start date, learning budget, working pattern.
Before the written offer, ask: 'What would make this an easy yes?'
Agree approval and the pay band before the final interview, so the offer isn't held up.
The hiring manager makes the verbal offer; the written offer follows within a working day.
Record which pre-set factors placed the offer where it is in the band.
Tell unsuccessful finalists personally, without avoidable delay, with brief feedback.

What will close this candidate

The problem: they'll own what happens when an agent's loop goes wrong, not just when the demo goes right.
Budget for experimentation: a monthly model and tool-call allowance they control.
Visibility: budget to speak at a conference on agent reliability, and time to write it up publicly.
A clear level rationale: why they're slotted here and exactly what the next level up would need to see.

Compliance checkpointEUUS

Use objective, gender-neutral, pre-agreed factors to place candidates within the band, apply them consistently and record the decision as an internal pay-equity control. Check the national law implementing Directive (EU) 2023/970 and any local record-keeping rules; this recommendation is good practice, not a universal statutory documentation rule.

Sources: EU Pay Transparency Directive 2023/970 · checked 2026-09-25

Stage 12 of 13Required

0 of 2 done

Help them settle in

Pre-boarding and 30-60-90 onboarding

Who runs it: Hiring manager + buddy + HR
Done when: A new hire who has worked through a written 30-60-90 plan tied to the 12-month outcomes, with a check-in at each milestone.

Key tool

Before day one
Laptop, account access and the agent repositories set up before day one.
Days 1–30: learn
Meets platform, reliability and product colleagues who depend on the agents they'll own.
Days 31–60: contribute
Adds or hardens instrumentation for the highest-risk agent.
Days 61–90: own
Owns reliability for one agent end to end, with an evaluation set running before each change.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the 30-60-90 plan from the 12-month outcomes. Agreeing the plan with the new hire stays with the manager.

Show the prompt and the checks

Describe: You are helping a hiring manager draft a 30-60-90 day onboarding plan for a new AI Agent Engineer. Context: I will paste the 12-month outcomes from the intake meeting, the systems and people the hire will work with, and anything they will need before day one. Produce: a pre-boarding checklist (equipment, access, a named buddy, a first-week schedule); then three sections — days 1–30 learn, days 31–60 contribute, days 61–90 own — each with 3–4 concrete, checkable expectations and one milestone check-in question. Use British English and plain wording. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each expectation is realistic for a new starter, ties back to a 12-month outcome, and offers the hire support rather than a test.
Diligence: The manager agrees the plan with the new hire in week one and owns the check-ins. The new hire's personal details stay out of the prompt.

Don't miss: Handing a new hire a brand-new agent to build in week one, before they've read how the existing ones are bounded, traced and watched — they'll repeat mistakes the team already fixed.

Framework

30-60-90 day plan

Three phases — learn, contribute, own — with written expectations for each, agreed with the new hire in week one.

Before day one: equipment, access, a named buddy and a first-week schedule.
Days 1–30, learn: systems, people and the current state of the work.
Days 31–60, contribute: a first scoped piece of work shipped with support.
Days 61–90, own: an area of responsibility with clear outcomes.
Hold a check-in at each milestone against the written plan.

Why it works: Hiring doesn't end at the signature. A written onboarding plan makes expectations and support visible to the new hire and their manager.

Before day one
- Laptop, account access and the agent repositories set up before day one.
- Model and tool-call API keys with a sensible spending limit.
- A named buddy from the engineering team.
- A short reading list for before day one: how the agent platform is architected, which agents are live today, and the incident log for each.
Days 1–30: learn
- Meets platform, reliability and product colleagues who depend on the agents they'll own.
- Maps every live agent, what it's allowed to do, and how each is currently bounded and watched.
- Ships one small production fix in their first month, to learn how a change reaches a live agent.
Days 31–60: contribute
- Adds or hardens instrumentation for the highest-risk agent.
- Finds and reports at least one boundary gap or failure mode nobody had measured.
- Adds cost and step tracking per agent.
Days 61–90: own
- Owns reliability for one agent end to end, with an evaluation set running before each change.
- Delivers one measured improvement — fewer failures, tighter bounds, lower cost — with before-and-after numbers.
- Presents findings to the wider team in plain language.

Compliance checkpointUSUK

US: the employee completes Form I-9 Section 1 no later than their first day of paid work, and the employer completes Section 2 within three business days after that day (on the first day if the job lasts less than three business days). UK: complete the prescribed right-to-work check before employment starts, and apply it the same way to everyone.

Sources: USCIS — Form I-9 instructions · GOV.UK — Check a job applicant's right to work · checked 2026-09-25

Stage 13 of 13Required

0 of 3 done

Measure time to hire

Measure the process: mean time to hire

Who runs it: Recruiter / talent lead + hiring manager
Done when: A stage-by-stage timing record for this hire, the slowest step named, and one change agreed before the next requisition opens.

Key tool

Record when the hire accepts

•Time to hire: days from the successful candidate entering the pipeline (application or first contact) to the accepted offer. Average it across hires for mean time to hire.
•Time to fill, recorded separately: days from requisition approval to the accepted offer.
•Days per stage, from the stage's first action to its last — and whether the time was spent waiting or working.
•Offer-acceptance rate: offers accepted divided by offers made.
•Where candidates dropped out, and the reason you were given.
•How many days it took to build and get security sign-off on the sandboxed task environment before the work sample could go out.

Check in at 90 days whether the hire is meeting the outcomes in the job description — that tells you whether the process picked well.

Read: mean time to hire Read: time to hire vs time to fill

AI as an advantageAI boost4D framework

Delegate: Hand AI the job of building a blank timing worksheet and the instructions for reading it. Entering dates, working out the timings and deciding what to change stay with the recruiter and the hiring manager.

Show the prompt and the checks

Describe: You are helping a recruiter and a hiring manager set up a timing review for an AI Agent Engineer hire. Context: I will list the stage names we used, in order (for example requisition approved, sourcing, screening, recruiter call, work sample, interviews, debrief, references, offer accepted). Produce: (1) a blank worksheet with one row per stage and columns for start date, end date, days in stage, days spent waiting (scheduling, approvals, no reply) and number of candidates who entered and left; (2) step-by-step instructions for us to work out, from our own records, time to hire (the successful candidate's entry to the pipeline, by application or first contact, to the accepted offer; average it across hires for mean time to hire) and, separately, time to fill (requisition approval to the accepted offer); (3) five questions we should ask about the slowest stage, separating waiting from working; (4) a short template for recording one change to make before the next hire. Use British English. Do not include benchmarks or example figures. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the worksheet covers every stage you actually ran, that the two definitions are not mixed up, and that the questions point at the process rather than at any person.
Diligence: People fill in the dates and do the arithmetic from their own records, choose the one thing to fix and write it down. No dates, names or details about any candidate go into the prompt.

Don't miss: Blaming the security team for a slow sign-off without first checking whether the sandboxed task was actually ready to submit for review on time.

Framework

Mean time to hire, by stage

Time to hire is the number of days from the successful candidate entering your pipeline (their application or your first contact) to their accepted offer; mean time to hire is that figure averaged across hires. Time to fill — requisition approval to acceptance — is a separate measure. Record both per stage, so you can see where the time went rather than only the total.

When the hire accepts, write down the date each stage started and ended, and the day the successful candidate entered the pipeline.
Work out the days per stage, the time to hire and the time to fill, and mark the slowest stage.
Note the offer-acceptance rate and the stage where most candidates dropped out.
Separate waiting time (scheduling, approvals, silence) from working time, and decide which one to shorten.
Fix one thing before the next hire, and write down what you changed.

Why it works: A single total hides where the delay sits. Per-stage timing points at the one step to fix, and the acceptance rate and drop-out points show whether candidates kept pace with the process.

Record for every hire

Time to hire: days from the successful candidate entering the pipeline (application or first contact) to the accepted offer. Average it across hires for mean time to hire.
Time to fill, recorded separately: days from requisition approval to the accepted offer.
Days per stage, from the stage's first action to its last — and whether the time was spent waiting or working.
Offer-acceptance rate: offers accepted divided by offers made.
Where candidates dropped out, and the reason you were given.
Check in at 90 days whether the hire is meeting the outcomes in the job description — that tells you whether the process picked well.

What to log for this role

How many days it took to build and get security sign-off on the sandboxed task environment before the work sample could go out.
Whether the agreed monthly model and tool-call budget held once the hire started running real experiments, or needed a mid-process increase.
How many candidates the trace-debugging work sample removed, and whether most dropped off before starting it or partway through.

Where this role's hiring slows down

Watch for the sandbox build turning into the critical path — a task environment rebuilt from nothing for every search instead of reused.
Watch for the security sign-off queue sitting behind unrelated reviews with no dedicated priority for a hiring decision.
Watch for the architecture-and-reliability round depending on one staff engineer's calendar with no second qualified interviewer named.

Compliance checkpointUKEU

Under the EU and UK GDPR, tell candidates how long you will keep their data, or the criteria used to decide. Give that notice when you collect data from them directly, or within the Article 14 time limit when it comes from another source (such as a referral or a sourcing tool). Delete or anonymise data when it is no longer needed, keeping only what a documented lawful purpose still requires.

Sources: GDPR — Regulation (EU) 2016/679, Articles 5, 13 and 14 · ICO — UK GDPR guidance · checked 2026-09-25

General information, not legal advice. Rules depend on the employer, the role and where the work is done — confirm what applies with HR or counsel.

Frequently asked questions

What does an AI agent engineer do?

They build and operate systems where a model runs in a loop — planning, calling tools, checking results and deciding again — rather than answering once. Day to day that means designing the loop, bounding what it can touch, instrumenting runs so failures are traceable, and building evaluation sets for tasks that take many steps.

AI engineer vs AI agent engineer — what's actually different?

The titles overlap, and plenty of companies use them interchangeably. 'AI engineer' is the broader title and can already include building and running agents. Reach for 'AI agent engineer' specifically once most of the job is multi-step work: designing the loop, bounding what it's allowed to do, tracing failures across many steps and evaluating whole runs rather than single answers. Judge the actual scope of the job, not the label on the posting.

How do you hire an AI agent engineer?

Run a structured process: a competency model agreed before you see candidates, a work sample that has them debug a real agent failure, and structured interviews scored against a shared rubric. This kit gives you all three, filled in, so you can adapt rather than build from nothing.

What should an AI agent engineer job description include?

Outcomes the person will own, the few requirements that actually gate the job, an explicit AI-fluency section, and a genuine pay range. Skip the long list of orchestration frameworks — it filters for people who've read the docs, not people who've operated a real agent.

How do you interview an AI agent engineer?

Give them a real failure to debug, not a framework quiz. Pair a job-shaped work sample with structured questions on boundary design, multi-step evaluation, and when an agent is the wrong tool. Score every candidate against the same rubric before comparing notes.

Do we need an AI agent engineer or is an AI engineer enough?

Write down how many agents you run, what real-world actions they can take, who looks after them today and what you expect a dedicated owner to change. If that picture shows sustained, risky work nobody has capacity for, it justifies a dedicated agent engineer.

Go deeper

Related hiring kits

Guides

Skills tests

See your first AI-generated assessment in 5 minutes

No credit card. No question library to learn. Paste a job description and watch H-Evaluate build.

Request early access