AI Engineer hiring kit

Every step to hire the engineer who ships features on top of large language models — and can prove they work.

About this kit — the 5Ws

An AI engineer builds product features on top of models someone else trained: chat, search, summarisation, agents. The job is less about clever prompts and more about knowing whether the feature actually works — building evaluation sets (a standing collection of test questions and correct answers used to check every change before it ships), catching regressions, and keeping cost and latency — how fast the feature responds — under control.

Calibrated for:: Mid-level through senior, individual-contributor track — this kit isn't calibrated for a first hire building an entire AI capability from scratch.
Timeline:: Set an approval-to-offer target at intake, allowing for panel availability, candidate adjustments, references and any lawful checks.

Who: Hiring managers and recruiters at software companies hiring their first or next AI engineer — including people who have never hired for this role before.
What: This kit turns 'we need an AI engineer' into a defined competency model, a job-shaped work sample built around evaluation, and consistent interview questions. It also gives you a way to check, once the hire has settled in, whether their evaluation habits actually showed up in production.
When: Use it from the moment someone says 'we need an AI engineer' — before the job is approved — and carry it through to recording how many days the hire actually took to land.
Where: Software teams shipping model-backed features.
Why: A CV lists frameworks and tools; it doesn't show whether someone built and used an evaluation set before shipping a change. The work sample and interview questions in this kit are designed to surface that specific habit directly, and applying them the same way to every candidate keeps the comparison job-related.

The competency model · 6

Use these competencies to compare job-related evidence. Review any apparent gap in a must-have before deciding.

Evaluation disciplineMust-have · 25
Builds an evaluation set before trusting a change; can say how they knew a feature worked and which failure modes they measured.
LLM application engineeringMust-have · 25
Has shipped features that use retrieval (finding and pulling in the right source material before the model answers), call external tools, or return answers in a required format — and can explain the trade-offs they made.
Production software engineeringMust-have · 20
Writes tested, observable, maintainable code; handles timeouts, retries, rate limits and bad model output gracefully.
Cost, latency and reliability judgementCan learn · 10
Knows what a request costs and how long it takes; picks a model size and caching approach on purpose rather than by default.
Product sense and communicationMust-have · 10
Explains model behaviour and its limits to non-engineers in plain language; ties technical choices to user outcomes.
Responsible AI and safety awarenessCan learn · 10
Spots risks such as prompt injection (hidden instructions in content the model reads, designed to hijack what it does), leaked private data, and harmful output — and designs guardrails (checks and limits that stop these problems) without being asked.

What's required and what's optional

Your hiring journey for AI Engineer. Select a stage to jump to it.

Required for every hire
Optional — use when it adds evidence
AI boost — a big drafting win
Colour = phase

Phase 1
Plan
1. 01Plan the roleRequired
2. 02Agree successRequiredAI boost
Phase 2
Attract
1. 03Write the jobRequiredAI boost
2. 04Find candidatesOptionalAI boost
Phase 3
Select
Phase 4
Decide
Phase 5
Onboard & learn
1. 12Help them settle inRequired
2. 13Measure time to hireRequiredAI boost

AI drafts, people decide: Use AI to draft, rewrite and summarise your own material — never to rank, score, screen out or decide on candidates, and never with a candidate's personal data.

Plan

Attract

Select

Decide

Onboard & learn

Solid = required · dashed = optional · arrow keys move between steps

Stage 1 of 13Required

0 of 3 done

Plan the role

Workforce planning and requisition approval

Who runs it: Hiring manager + finance / HR business partner
Done when: An approved requisition with a written business case, budget and a named decision-maker.

Key tool

Answer these before the role is approved

1Which model-backed features will this person own in their first year?
2How do we know today whether those features give correct answers?
3Could an engineer already on the team grow into this role within the next two quarters?

AI as an advantage4D framework

Delegate: Hand AI the first draft of the business case and the build / borrow / buy comparison. The decision, the budget and the sign-off stay with people.

Show the prompt and the checks

Describe: You are helping a hiring manager write a business case for an AI Engineer role. Context: I will paste the problem the hire solves, the work it involves, the capacity we already have and the time we have. Produce: (1) a one-sentence problem statement in business terms; (2) a build / borrow / buy comparison with the strongest argument for each; (3) the questions a finance partner would ask; and (4) a short recommendation with the evidence it rests on. Use British English and plain language. Do not invent figures — leave [brackets] where I need to supply numbers. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check every number and claim against your own notes and strike anything the draft assumed. Make sure 'buy' is argued from ongoing work, not from a job title.
Diligence: The hiring manager owns the final case and the budget request. No candidate information belongs here; AI drafts the case, people decide whether to open the role.

Don't miss: Posting an 'AI engineer' requisition to match a competitor's job board, with no specific model-backed feature already waiting for this person to own.

Framework

Build · Borrow · Buy

Before opening a role, decide whether to build the capability in your current team, borrow it (contractor, agency, partner), or buy it (a permanent hire).

Write the problem the hire solves in one sentence, in business terms — not the job title.
Test Build: could an existing person grow into this within the time you have?
Test Borrow: is the need time-boxed, or does it end once a system is shipped?
Choose Buy only when the work is ongoing, core to the product and nobody inside can absorb it.
Record the decision, the budget and the one person who signs off the final hire.

Why it works: Writing the build, borrow or buy decision down exposes a vague or temporary need before it becomes a permanent requisition — and gives every later stage an anchor to check against.

• Confirm who signs off the hire in writing — for this role, that's typically the engineering manager who will own the model-backed features day to day.

Is this a dedicated role?

Decide whether the work is sustained and substantial enough for a dedicated owner by writing down the workload, the risk, the capacity you already have and the outcomes you expect.

Answer these before the role is approved

1.Which model-backed features will this person own in their first year?
Listen for: Named features with users today, or committed on the roadmap.
Red flag: 'We want to do more with AI' with nothing specific.
2.How do we know today whether those features give correct answers?
Listen for: If the honest answer is 'we don't know', that gap — not a wish list of tools — is the actual business case for this hire.
3.Could an engineer already on the team grow into this role within the next two quarters?
Listen for: Either a specific person with a credible learning plan, or a clear reason nobody on the team can absorb this.
4.Does the need continue after the first launch, or does it end once this project ships?
Listen for: Recurring evaluation and cost-control work across several features points to a permanent hire; a single one-off launch may suit a contractor instead.
5.What's the real monthly cost of leaving this role open — in delayed launches, support load or uncontrolled model spend?
Listen for: A concrete consequence: a delayed launch, support tickets, model spend nobody controls.

Build, borrow or buy?

Build: pair a backend engineer with a mentor and a ring-fenced learning budget so they pick up evaluation and retrieval work over one or two quarters.
Borrow: bring in a contractor to stand up the first evaluation set and retrieval pipeline, then hand the running of it to the team within about three months.
Buy: hire a permanent AI engineer once more than one feature needs ongoing evaluation, cost tracking and improvement work.

Watch out for

Confusing this role with a machine learning engineer. If the hardest question is 'how do we make the model better?', you need an ML engineer; if it is 'how do we know the feature works?', you need an AI engineer.

Stage 2 of 13Required

0 of 2 done

Agree success

Intake meeting and competency model

Who runs it: Recruiter + hiring manager
Done when: A signed-off competency model: 5–7 competencies, each marked must-have or trainable, with weights.

Key tool

Ask the hiring manager

1Which of the systems or areas this person will own worries you most, and why?
2When something there went wrong recently, how did you find out?
3In 12 months, what will be shipped, fixed or running for you to call this hire a success? Agree two or three measurable outcomes.
4Who will they work with most closely?
5What could a strong candidate learn in their first three months? Those capabilities are 'trainable' — never use them to filter applications.

AI as an advantageAI boost4D framework

Delegate: Hand AI your raw intake notes to turn into a draft competency model. Choosing the must-haves and the weights stays with the hiring manager and recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter and a hiring manager build a competency model for an AI Engineer role. Context: I will paste the notes from our intake meeting, including the 12-month outcomes and what the hiring manager worries about most. Produce a table of 5–7 observable competencies with, for each: a one-line definition; whether it is a must-have or trainable, with the reason; what evidence of it looks like in real work; and a suggested weight so the weights total 100. Use British English and inclusive wording, avoid years-of-experience or named employers as evidence, and keep every competency job-related. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each competency is observable and job-related, that nothing describes a type of person rather than the work, and that trainable items haven't been marked must-have.
Diligence: The hiring manager signs off the final model and weights before anyone meets a candidate. No candidate data goes into the prompt; AI drafts, people decide.

Don't miss: Listing specific frameworks (a particular orchestration library or vector database) as must-haves. Tools change every few months; the discipline doesn't.

Framework

Competency model (must-have vs trainable)

A short list of the observable capabilities the job needs, split into what the hire must bring on day one and what you will teach.

Start from the 12-month outcomes, then ask which capabilities produce them.
Keep 5–7 competencies. With more, interviewers struggle to score each one separately.
Mark each must-have or trainable. Only must-haves can count against a candidate.
Give each a weight so they sum to 100, agreed before anyone meets a candidate.
Write what evidence of each looks like, so interviewers know what they are listening for.

Why it works: Agreeing the criteria before meeting candidates stops the bar moving from one candidate to the next, and makes it harder for 'I'll know it when I see it' to decide the hire.

Core intake questions (ask for every role)

Book 45 minutes with the hiring manager before anything is advertised, and write the answers down.
Which of the systems or areas this person will own worries you most, and why?
When something there went wrong recently, how did you find out?
In 12 months, what will be shipped, fixed or running for you to call this hire a success? Agree two or three measurable outcomes.
Who will they work with most closely?
What could a strong candidate learn in their first three months? Those capabilities are 'trainable' — never use them to filter applications.
Who is on the interview panel, and is their time held for the next four to six weeks?

Questions specific to this role

1.Which model provider and stack does this feature run on today, and will the hire need to work across more than one?
Listen for: A clear answer — it shapes the work sample's starting repository and the system-design interview.
2.Does the team need this person to build new model-backed features, improve evaluation on ones that already exist, or both?
Listen for: A concrete split between new build and improvement work — it changes how much weight to put on evaluation discipline versus shipping speed.
3.What's the monthly budget for this hire's model-usage experiments, and who approves an increase?
Listen for: A number and a named approver. A vague answer often means cost control will fall entirely to the new hire to invent.

What success looks like in 12 months

Every model-backed feature has an evaluation set that runs before each release.
At least one specific failure mode named at intake — wrong answers, made-up citations, ignored instructions — is reduced, with before-and-after numbers from the evaluation set.
Model spend per request is tracked and has an owner.
Support and product colleagues can describe, without an engineer translating for them, what each model-backed feature will and won't do for a customer.

Watch out for

Letting a 'must have trained or fine-tuned a model' requirement sneak into the must-haves. Model development and production ML may sit with a machine learning engineer, and model-backed application engineering with this role — state which one the work actually needs.

Stage 3 of 13Required

0 of 3 done

Write the job

Job description and pay band

Who runs it: Hiring manager (content) + recruiter (publishing)
Done when: A published job description that states outcomes, the must-haves, how AI tools are used and the pay information the law requires.

Key tool

AI Engineer

We're looking for an AI engineer to own the quality of the language-model features in [product]. You'll build the evaluation sets that tell us whether features work, improve retrieval and tool use, and keep cost and speed under control — working closely with product and support.

What you'll do

•Build and maintain evaluation sets for each model-backed feature, and run them before every release.
•Improve retrieval, prompting and tool calling so answers are accurate and grounded in the right information.
•Ship production code: tested, monitored, and resilient to slow or wrong model responses.
•Track cost and latency per request and choose models and caching deliberately.
•Explain what our AI features can and cannot do to product, support and customers.
•Design guardrails against prompt injection, data leaks and harmful output.

What you need

•You have shipped at least one feature built on a large language model to real users.
•You can show how you measured whether that feature worked — an evaluation set, metrics, or a structured review.
•You write production-quality [Python/TypeScript] code, with tests — this isn't a research or notebook-only role.
•You can explain technical trade-offs clearly to people who aren't engineers.

Nice to have

•Experience with retrieval systems (search, embeddings, re-ranking).
•Experience with agents or multi-step tool calling.
•Experience reducing model cost or latency in production.
•Some awareness of AI-specific regulation, such as the EU AI Act, even if you're not the one who owns compliance.

How we use AI tools

•Where company policy permits, use approved AI tools and verify their output — you're being hired to catch what they get wrong, not to trust them by default.
•You'll recognise the cases where a rules-based approach beats a model call, and say so.
•In the work sample, use whichever AI coding or research tools you'd normally reach for on a task like this — just tell us which ones and how you checked their output.

Pay

Salary range: [currency, minimum–maximum], plus [equity/bonus/benefits]. The range reflects [location/working pattern]. We place every offer within it using the same pre-set, job-related, gender-neutral factors — depth of shipped, measured LLM feature experience, scope of evaluation ownership, production software engineering experience, experience with retrieval or tool-calling systems — never previous salary or negotiating style.

AI as an advantageAI boost4D framework

Delegate: Hand AI the first draft of the job description from your signed-off competency model. The must-haves, the pay range and the final wording stay with people.

Show the prompt and the checks

Describe: You are helping write an outcome-based job description for an AI Engineer role. Context: I will paste the competency model (must-have and trainable, with evidence), the 12-month outcomes, how the team uses AI tools, the location and working pattern, and the pay range. Produce: a summary paragraph that leads with the problem and what the hire will own; 'What you'll do' as outcomes, not tasks; 'What you need' containing only the must-haves; 'Nice to have' for the trainable items; 'How we use AI tools'; and a pay section that states the range. Use British English and inclusive, gender-neutral wording, avoid hype, and replace any 'X years' with the capability it stands for. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that 'What you need' contains only the must-haves, that no degree, employer or years requirement crept in, and that the pay range and location are exactly what was approved.
Diligence: The hiring manager owns the content and the recruiter owns publishing, including the pay information the law requires for each location. AI drafts the advert; people decide what it says.

Don't miss: Asking for 'five or more years of LLM experience' — the technology has not existed in its current form for that long, so the requirement only filters for people willing to stretch the truth.

Stage 4 of 13Optional

0 of 3 done

Find candidates

Sourcing strategy

Who runs it: Recruiter / sourcer
Done when: A shortlist-ready pipeline from at least three channels, with outreach tracked.

Use when:: Inbound applications won't give you enough qualified candidates.
Skip when:: Your shortlist already comes from inbound applications or referrals.

Key tool

Search query

("LLM" OR "large language model" OR "generative AI" OR "GenAI") AND ("evaluation" OR "evals" OR "RAG" OR "retrieval") AND ("production" OR "shipped" OR "deployed") AND (Python OR TypeScript) NOT ("intern" OR "student")

First outreach message

AI as an advantageAI boost4D framework

Delegate: Hand AI the search queries and the first draft of the outreach template. Choosing channels, personalising to a candidate's public work and sending stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter source candidates for an AI Engineer role. Context: I will paste the must-have competencies, the kind of work we want evidence of, the channels we are using and the pay range. Produce: (1) one search query per channel, using OR for alternatives, AND for things that must all appear and NOT to exclude, with a one-line note on what each query looks for; and (2) a first-outreach template of under 120 words that opens with a [placeholder for something the candidate built], states the problem we are hiring to solve and the pay range, and ends with a low-pressure question. Use British English and inclusive wording. Do not refer to any specific person. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the queries search for evidence of the work rather than job titles alone, and that the template claims nothing about the role or the pay you can't back up.
Diligence: The recruiter personalises every message to something the candidate actually built, and sends it. No names or profile data go into the prompt; AI drafts, people decide whom to contact.

Don't miss: Spending outreach time on profiles that list many AI frameworks and tools but link to no checkable project — look for a specific feature or repository instead.

Framework

Channel mix + evidence-first outreach

Spread sourcing across channels where the work is visible, and open outreach with the problem, not the perks.

Pick channels where candidates show their work, not only where they list job titles.
Write one search query per channel. OR joins alternatives, AND joins things that must all appear, NOT excludes. Keep the version that works best.
Personalise the first line to something the candidate actually built or wrote.
Lead with the problem and the pay range. Save the perks for later.
Track reply rates by channel each week and drop the channel that isn't working.

Why it works: Searching for evidence of the work can surface candidates whose current title differs from the vacancy, or whose CV undersells what they have done.

Sourcing basics

Adapt the search query for each platform — the same words behave differently on each site.
Candidates with adjacent titles may have relevant evidence. Search for the work and outcomes the role needs, not the current title.
Channel activity is a lead, not evidence. Review linked work and examples against the written competencies.

Where to look

Public code repositories

Contributions to evaluation or LLM tooling are reviewable evidence of the skills this role needs.

Technical blogs and conference talks

A public write-up on a failed feature lets you review how the person measured the failure and what they changed.

Your own engineering team's referrals

Engineers who built evaluation sets with the candidate can name specific features and results you can then review.

LinkedIn and similar networks, searched for shipped LLM work rather than the job title 'AI Engineer'

Candidates with adjacent titles may have relevant evidence — search by capability rather than filtering on 'AI engineer' as the title.

Communities around open-source AI tools

Answers posted in these communities are things you can check directly — a concrete alternative to a CV's list of tools.

Boolean search string

("LLM" OR "large language model" OR "generative AI" OR "GenAI") AND ("evaluation" OR "evals" OR "RAG" OR "retrieval") AND ("production" OR "shipped" OR "deployed") AND (Python OR TypeScript) NOT ("intern" OR "student")

First outreach message

[name] — [one detail] in your [post/repo] on [specific thing] caught my eye. At [company] we ship [n] model-backed features and can't yet tell you, with confidence, when any of them are wrong — that's the problem we want an AI engineer to own, at [range]. Worth a 20-minute conversation about what that problem actually looks like day to day?

Stage 5 of 13Required

0 of 3 done

Check applications

Application screening

Who runs it: Recruiter
Done when: Every application reviewed against the same written knockouts within five working days.

Key tool

Reject only if one of these is missing

•Can meet the role's work-authorisation requirements by the start date, with sponsorship where the employer offers it. Don't ask for documents at this stage.
•Describes at least one LLM-backed feature that reached real users.
•Shows hands-on production software engineering experience, in any language or stack.

AI as an advantage4D framework

Delegate: Hand AI the job of turning your written knockouts into a consistent checklist with reason codes. Reviewing each application stays with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter prepare an application-screening checklist for an AI Engineer role. Context: I will paste the must-have competencies and our written knockout criteria. Produce: (1) each knockout as a yes/no question answerable from an application alone; (2) a short list of reason codes for a 'no'; and (3) a note of any criterion that looks like a proxy — a degree, employer brand or employment gap — rather than a job-related requirement, with a suggested rewrite. Use British English. Do not evaluate any applicant and do not ask for applicant details. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each question is binary and job-related, and that every flagged proxy has been removed or justified in writing.
Diligence: A person reviews every application against the checklist and records the reason. Never paste CVs or applicant details into an AI tool, and never let a tool screen anyone out — in New York City and the EU that can bring automated-decision rules into play.

Don't miss: Rejecting a candidate whose CV lists frameworks rather than a shipped, measurable feature — ask for the specific feature and how they know it works, not the tool list.

Framework

Knockout criteria

A small set of yes/no requirements, written in advance, applied to every applicant the same way.

Derive knockouts only from must-have competencies and genuine legal requirements.
Keep them binary and answerable from the application itself.
Apply them in the same order to every application and log the reason for each rejection.
Do not use proxies such as university, employer brand or employment gaps as knockouts.
Anything ambiguous gets a second review or moves forward to the skills stage.

Why it works: Early screening rejects people before richer evidence exists, so its criteria need the most care. Written criteria make decisions reviewable — but each one must also be job-related, applied consistently and checked for unjustified adverse impact.

Screening rules

Do not use degrees, employment gaps or certifications as filters unless they are demonstrably job-related and justified for this role.
Judge the claims in a CV, not how polished it is — a well-written CV may or may not have had AI help, and either way it is the evidence that matters.

Knockout criteria

Can meet the role's work-authorisation requirements by the start date, with sponsorship where the employer offers it. Don't ask for documents at this stage.
Describes at least one LLM-backed feature that reached real users.
Shows hands-on production software engineering experience, in any language or stack.

Green flags

Mentions how a feature was measured: an evaluation set, accuracy before and after, user feedback loops.
Describes a failure and what they changed because of it.
Links to code, a write-up or a demo you can check.

Red flags

Names retrieval, agent or orchestration libraries by brand, with no feature, metric or outcome attached to any of them.
Every example is a weekend build or course exercise — nothing that ran in front of a real customer with real traffic (fine at junior level, not at this one).
Claims about AI work that can't be tied back to a specific, checkable project.

Watch out for

Treating a polished, AI-written CV as evidence either way — judge the specific claims about shipped, measured features, not how well the prose reads.

Compliance checkpointNew York CityEU

For a New York City employment decision, work out whether a tool is an 'automated employment decision tool' (AEDT) under Local Law 144 — one that uses covered techniques to substantially assist or replace a discretionary decision. Before using a covered tool, make sure an independent bias audit was completed no more than a year earlier, publish the required summary, and give covered candidates or employees the required notice at least ten business days before use. In the EU, the AI Act (Annex III) lists AI used to recruit, filter applications or evaluate candidates as high-risk, subject to the Article 6(3) exception; the related obligations apply from 2 December 2027 under the amended timetable. A manual screen does not exempt an AI tool used elsewhere in the process.

Sources: NYC DCWP — Automated employment decision tools · EU AI Act — consolidated Regulation (EU) 2024/1689 · checked 2026-09-25

Stage 6 of 13Optional

0 of 3 done

Recruiter call

Recruiter screen

Who runs it: Recruiter
Done when: A 20–30 minute call that confirms logistics, motivation and one or two must-have signals.

Use when:: You are hiring at volume, or pay and logistics need confirming early.
Skip when:: You have only a handful of applicants — fold the pay and logistics questions into application screening instead.

Key tool

Ask every candidate, in this order

1Walk me through the last model-backed feature you personally shipped — what did it do, and who used it day to day?
2How did you know it was working?
3Has your LLM feature work been customer-facing and in production, or mostly internal tools and prototypes so far?
4Day to day, are you writing this in Python, TypeScript, or something else — and does that match what we use here?
5Our range for this role is [range]. Does that work for you?

AI as an advantage4D framework

Delegate: Hand AI the first draft of the screening-call script and its evidence anchors. Running the call and taking the notes stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter write a 20–30 minute structured screening call for an AI Engineer role. Context: I will paste the must-have competencies, the logistics we need to confirm (location, working pattern, notice period) and the pay range. Produce: an opening that explains the process, the timeline and the pay range; 4–6 questions in a fixed order, each with one line on what a strong answer contains; and a closing that asks whether the published range works. Use British English and inclusive wording. Do not include questions about salary history, age, family, health or any other protected characteristic. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that every question is job-related and lawful, that the pay question asks whether the range works rather than for history, and that the anchors describe evidence, not impressions.
Diligence: The recruiter asks the same questions of everyone and writes the notes. No candidate details go into the prompt; AI drafts the script, people run the call and decide.

Don't miss: Expecting the recruiter to evaluate deep technical answers. Their job here is motivation, logistics and clarity of self-description — the technical bar comes later.

Framework

Structured screening call

The same short set of questions for every candidate, with notes taken against what a strong answer contains.

Open by explaining the process, timeline and pay range. No surprises later.
Confirm logistics: location, working pattern, notice period and whether the range works for them.
Ask the same 4–6 questions in the same order, and note evidence, not impressions.
Leave time for the candidate's questions. Don't score their questions unless they test a written, job-related competency.
Decide within 48 hours and tell the candidate either way.

Why it works: Using the same questions and written evidence anchors makes answers easier to compare, and a short, predictable call respects the candidate's time.

Screening call basics

Set the decision timetable before sourcing and tell candidates when to expect each update.
Ask every candidate what draws them to this particular role, and listen for interest in the work described rather than the title.
Confirm notice period and whether they are in other processes, so you can plan the timetable.
End every call by asking whether the published pay range works — never ask for salary history.

Recruiter screen questions

1.Walk me through the last model-backed feature you personally shipped — what did it do, and who used it day to day?
Listen for: A named feature, real users, and a clear line between what the candidate did and what their team did.
Red flag: The candidate cannot separate their own decisions and work from the team's.
2.How did you know it was working?
Listen for: Some form of structured check, even a homemade spreadsheet of test cases — not just a feeling that it seemed fine.
Red flag: 'It looked good in testing' or 'users didn't complain'.
3.Has your LLM feature work been customer-facing and in production, or mostly internal tools and prototypes so far?
Listen for: A clear, honest answer — production, customer-facing work is more directly relevant experience for this role.
4.Day to day, are you writing this in Python, TypeScript, or something else — and does that match what we use here?
Listen for: A straightforward answer on language and stack fit, surfaced early rather than as a surprise later.
5.Our range for this role is [range]. Does that work for you?
Listen for: A clear answer on whether the published range works. Don't ask for salary history — this process sets pay from the published band and pre-set, job-related factors.

Compliance checkpointUS (state and city)EU

Don't ask for salary history. In the EU, national laws implementing Article 5(2) of Directive (EU) 2023/970 must prohibit the question. US rules vary, but jurisdictions including California and New York prohibit employers from seeking salary history. Ask whether the published range works instead.

Sources: EU Pay Transparency Directive 2023/970, Article 5 · California Labor Commissioner — salary history · New York State DOL — salary history ban · checked 2026-09-25

Stage 7 of 13Required

0 of 2 done

Work sample

Skills assessment / work sample

Who runs it: Hiring manager (design) + recruiter (logistics)
Done when: Every shortlisted candidate completes the same job-shaped task, marked against criteria written in advance.

Key tool

Fix a support assistant that answers confidently and wrongly

You're given a small question-answering service built on a language model, with a set of help-centre articles and 20 real-looking customer questions. Some answers are wrong, some are made up, and one question tries to trick the assistant into ignoring its instructions. Build a small evaluation that shows how often it goes wrong, make one or two changes that improve it, and show the before-and-after results.

Planned for up to three hours (adjustments available on request). At the agreed stopping time, write a short note on what you would examine next, and what you deliberately left out.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the work-sample brief and its marking sheet. Choosing the task, setting the timebox and marking stay with the hiring manager.

Show the prompt and the checks

Describe: You are helping a hiring manager design a work sample for an AI Engineer role. Context: I will paste the competency model, a real problem the team solved recently with anything confidential removed, the timebox, and our AI-tool policy for candidates. Produce: a candidate-facing brief (what to do, what to hand in, the time cap, how to request an adjustment, which AI tools are allowed and how to report their use); and a marking sheet with 4–6 criteria tied to named competencies, each with what a 1, a 3 and a 5 look like. Use British English and plain, inclusive wording. Do not design anything that needs more than the stated time. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the task fits the timebox, that every criterion maps to a competency, and that the brief gives all candidates the same starting materials.
Diligence: The hiring manager owns the task and the criteria, and people mark every submission. Do not paste candidate submissions into an AI tool for scoring; AI drafts the brief, people mark and decide.

H-EvaluateH-Evaluate can generate a job-specific assessment from your job description, so you don't have to build the task by hand.How H-Evaluate helps →

Don't miss: Skipping the seeded help-centre articles and questions and asking candidates to 'build something from scratch' instead — that measures spare time and setup speed, not evaluation discipline.

Framework

Work-sample test

A small, realistic slice of the job, marked against criteria written in advance. Give candidates equivalent materials and arrange reasonable adjustments where required.

Take a real problem the team solved recently and shrink it to fit the timebox.
Write the marking criteria first, tied to the competency model.
Timebox honestly — for take-home work, a few hours at most — and say so in the brief.
State the AI-tool policy in writing. Where the job permits these tools, use equivalent conditions in the assessment and evaluate how candidates verify their output.
Mark without CV details where you can, and have two people mark borderline submissions.

Why it works: A well-designed work sample gives direct evidence of job-relevant skill rather than a proxy for it, and gives candidates a realistic preview of the work.

Assessment rules for every candidate

State the planned time cap and how to request an adjustment. At the agreed stopping time, ask candidates to note what they would examine next and what they left out — never reward unpaid overtime.
Give every candidate the same starting materials, and use a second marker for borderline submissions.
Say in writing which AI tools are allowed and ask candidates to tell you how they used them.
Invite every candidate to ask for adjustments, and answer each request promptly.
Pay for anything the task costs the candidate to run, such as model or cloud usage.

The work sample

Fix a support assistant that answers confidently and wrongly

Timebox: Planned for up to three hours (adjustments available on request). At the agreed stopping time, write a short note on what you would examine next, and what you deliberately left out.

What the candidate hands in

Your evaluation: the test cases and how you decide whether an answer is right.
Your changes to the code, with a short explanation of each.
Before-and-after results from your evaluation.

Marking criteria (write these first)

The evaluation exists and measures something meaningful (not only 'did it return text').
The changes are aimed at the failures the evaluation found, not random tuning.
Reports gains and regressions across the evaluation cases.
The prompt-injection question is noticed and handled.
The code is clean enough that it would survive a real code review, not just work once.
The half-page note names a real trade-off — cost, coverage or time — rather than just listing what wasn't finished.

Adjustments: If a candidate needs more time or would rather do a live, pair-programmed version, that's fine — ask. We cover the model usage; the starter repository ships with a spending-capped key.

Watch out for

Banning AI tools outright. Where company policy allows them, engineers on this team use them for parts of the work — assess whether the candidate checks what the tools produce, not whether they used one.

Compliance checkpointUSGreat BritainEU

In the US, covered employers must provide a reasonable accommodation to a qualified applicant with a disability unless it would cause undue hardship (ADA). In Great Britain, the Equality Act 2010 requires reasonable adjustments in recruitment. In the EU, the duty comes from national equality law. Don't ask for more medical information than the law allows.

Sources: EEOC — prohibited practices · Equality Act 2010 · checked 2026-09-25

Stage 8 of 13Required

0 of 3 done

Structured interviews

Structured interview loop

Who runs it: Interview panel (named in advance)
Done when: Every competency has a lead interviewer and at least one second source of evidence, with the same questions for every candidate.

Key tool

Interview questions by pillar

Free in this kit: Domain questions for every round. The other three pillars and the AI-fluency check are measured by H-Evaluate.

Free in this kit

Senior engineer who marked the work sample · Evaluation discipline · LLM application engineering

1.Walk me through your evaluation. Why those test cases, and how does each one get marked right or wrong?
Evaluation discipline
Good answerChooses test cases that mirror failure types actually seen in the brief — made-up answers, missed instructions — and states a clear, repeatable rule for judging each answer right or wrong.
Weak answerTest cases chosen at random, or can't say how an individual answer is judged right or wrong.
2.Your change improves the target cases but regresses a different case in the supplied set. How would you quantify the trade-off and decide whether to keep, revise or roll it back?
Evaluation discipline
Good answerChecks how often the regressed case type shows up in real traffic, weighs it against the gain, and makes an explicit keep/revise/rollback call rather than just reporting both numbers.
Weak answerReports the regression without weighing it, or reruns the change until the overall score looks acceptable again.
3.Why did you handle retrieval the way you did in your work sample? What would you try next, and why?
LLM application engineering
Good answerExplains concretely how the documents were split and retrieved, and names one specific next experiment with the effect they'd expect it to have.
Weak answerDescribes default settings with no reasoning for the choice, or has no next step in mind.

Optional follow-ups (1)

4.A retrieval feature gives correct answers in English but poor results in Japanese, using the same help-centre articles and the same code. What parts of the system would you look at first, and in what order?
LLM application engineering
Good answerSeparates the possible causes — document coverage per language, text splitting, retrieval matching, answer generation — and names a way to test each one rather than guessing at a single fix.
Weak answerJumps straight to 'use a bigger model', or blames the model provider without isolating where in the pipeline it breaks.

AI as an advantageAI boost4D framework

Delegate: Hand AI the drafting of probe questions and strong / weak anchors for one competency. Asking, listening and scoring stay with the interviewer.

Show the prompt and the checks

Describe: You are helping an interviewer prepare a structured technical interview for an AI Engineer role. Context: I will paste one competency, its evidence description and the core question we ask every candidate. Produce: four probes in this fixed order — what would you do, why that choice, what else did you consider, how would you know it worked — tailored to the question; a one-line description of a strong answer and of a weak answer; and two follow-ups to use only if time allows. Use British English and inclusive wording. Keep everything about the work, nothing about the person. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the probes test the named competency rather than trivia, that the anchors describe observable evidence, and that no question touches a protected characteristic.
Diligence: Interviewers ask the same core questions of every candidate and score alone against the rubric. Interview notes and candidate names never go into an AI tool; AI drafts the questions, people interview and decide.

H-EvaluateH-Evaluate can add evidence on Cognitive, Situational judgement, Behavioural and AI fluency for your panel to review alongside these interviews.How H-Evaluate helps →

Don't miss: Over-crediting a confident description of a retrieval or tool-calling architecture that isn't backed by the work sample or a concrete detail in their answer to the system-design questions.

Framework

Structured technical interview

Every candidate gets the same job-specific questions, each interviewer leads on specific competencies, and every answer is probed with the same four follow-ups: what would you do, why, what else did you consider, and how would you know it worked.

Give each competency a lead interviewer, plus a second source of evidence — the work sample, another round or an independent marker. Avoid asking the same thing twice, but never let one person's view be the only evidence for a critical competency.
Ask the three core questions for your round, in order, for every candidate. Use an optional follow-up only if there is time.
Probe each answer: why that choice, what else they considered, and how they would know it worked — until you hear the candidate's own technical decision.
Take notes on evidence during the interview. Don't rate until it's over.
Fill in your interviewer sheet and submit it before you talk to any other interviewer.

Why it works: Structured interviews make answers comparable: every candidate faces the same job-related questions and is scored against the same anchors.

Interview rules

Use the same core questions in the same order for every candidate.
In the work-sample review, understand the candidate's decisions against the rubric — don't turn it into a debate about your preferred solution.
Avoid trivia disconnected from a real job decision. Use a realistic debugging, design or evaluation problem instead.

Watch out for

Letting the system-design round drift into pure infrastructure talk with no mention of how the candidate would check whether the design actually works.

Compliance checkpointUSUKEU

Don't ask about protected characteristics — such as age, disability, pregnancy or family plans, religion, race or nationality — unless applicable law permits the question for a specific, necessary, job-related purpose, and HR or counsel has confirmed that basis beforehand. Ask only whether the candidate can meet the role's work-authorisation requirements. If a candidate volunteers protected information, don't probe; record only what is lawfully needed, keep it separate and restrict access.

Sources: EEOC — prohibited practices · Equality Act 2010 · checked 2026-09-25

Where to find more interview questions

Use web question banks for inspiration only: ask every candidate the same job-related questions, check each one against this kit's competencies, and never ask about protected characteristics. Don't reuse this kit's work sample from a public bank — candidates can find questions that circulate online, so a widely posted question tests preparation more than skill.

Show 7 sites

US OPM — Structured interviews(opens in a new tab)
US federal guidance on building structured interviews, with a guide, training material and sample questions with rating scales.
FreeFor interviewers
Acas — Interviewing job applicants(opens in a new tab)
UK public guidance on preparing fair interviews, asking applicants the same questions and avoiding discriminatory ones.
FreeFor interviewers
Workable — Interview questions library(opens in a new tab)
Sample interview questions for employers, listed by job role, including machine learning, data and engineering roles.
FreeFor interviewers
Indeed for Employers — Interview questions(opens in a new tab)
Sample interview questions for employers by job title and skill, with notes on what to look for in answers.
FreeFor interviewers
Tech Interview Handbook(opens in a new tab)
Open-source software-engineering interview preparation: coding, system design and behavioural questions.
FreeFor candidates and interviewers
Chip Huyen — Machine Learning Interviews Book(opens in a new tab)
Free online book with knowledge questions on maths, computer science and machine learning, grouped by difficulty.
FreeFor candidates and interviewers
Blind — Interview questions channel(opens in a new tab)
Anonymous posts by tech workers about questions they report being asked. Unverified; some content needs sign-in.
Free, some content needs sign-inCandidate reports, unverified

Links checked 2026-10-05. External sites; we don't control their content.

Stage 9 of 13Required

0 of 2 done

Compare evidence

Scoring, calibration and debrief

Who runs it: Hiring manager (chairs) + panel
Done when: A documented hire / no-hire decision traceable to scored evidence against the competency model.

Key tool

Interactive scorecard

Score one candidate against the rubric. Everything stays in your browser. Download it as a spreadsheet for your records.

Evaluation discipline Must-have

Weight

What do 1, 3 and 5 look like?

1: No evaluation, or only 'it looked right'. Can't say how they would know a change helped.
3: Puts together a test set once prompted; catches the headline failure mode but misses the edge cases around it.
5: Has the evaluation running before anyone asks for it, aims it at real failure types, and reports a regression as readily as a win.

LLM application engineering Must-have

Weight

What do 1, 3 and 5 look like?

1: Only tutorial-level use of a model API.
3: Has shipped retrieval or tool calling; explains what they did but not always why.
5: Explains design trade-offs from experience and names the next experiment and its expected effect.

Production software engineering Must-have

Weight

What do 1, 3 and 5 look like?

1: Code works on the happy path only; no tests or error handling.
3: Tested, readable code; handles common failures when prompted.
5: Designs for timeouts, retries, fallbacks and monitoring from the start.

Cost, latency and reliability judgement Can learn

Weight

What do 1, 3 and 5 look like?

1: No idea what a request costs or how long it takes.
3: Can state cost and response time for their own features and names the usual ways to bring each down.
5: Picks the model size and caching strategy deliberately for each feature, and confirms the change didn't quietly cost quality.

Product sense and communication Must-have

Weight

What do 1, 3 and 5 look like?

1: Can't explain model behaviour without jargon.
3: Clear explanations; connects work to users when asked.
5: Can explain a model's limits honestly to any audience in the room, and frames every technical choice in terms of what the user actually experiences.

Responsible AI and safety awareness Can learn

Weight

What do 1, 3 and 5 look like?

1: Doesn't recognise prompt injection or data-leak risk.
3: Names the main risk once an interviewer points at the scenario, but wouldn't have flagged it unprompted.
5: Spots risks unprompted and designs guardrails and incident responses.

Weighted score– / 5

AI as an advantage4D framework

Delegate: Hand AI the drafting of a blank debrief worksheet and agenda from the competency rubric alone. People add the evidence locally; the scores and the decision stay with the panel.

Show the prompt and the checks

Describe: You are helping a hiring manager prepare a debrief for an AI Engineer role. Context: I will paste the competency rubric only — the competencies, their weights and their 1, 3 and 5 anchors. Produce: (1) a blank debrief worksheet with one row per competency and empty columns for each interviewer's independent score, the evidence they cite, the spread of scores and an 'evidence gap?' box; (2) a 45-minute agenda: confirm every scorecard is in and reveal all scores at once, discuss the widest spreads using evidence only, review any must-have scored 1, then look at the weighted total and decide (hire, no hire or one targeted follow-up), and record the reasons; and (3) neutral questions the chair can ask to draw out evidence rather than adjectives. Leave every evidence and score cell empty. Do not suggest a score, a rating or a recommendation. Use British English. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the worksheet follows your rubric exactly — same competencies, weights and anchors — and that the agenda keeps independent scores hidden until the reveal.
Diligence: Interviewers fill in the evidence and scores locally, alone, before the debrief, and the panel decides together. Candidate notes and personal data never go into an AI tool; AI never scores, ranks or compares candidates.

H-EvaluateH-Evaluate gives you a report on each candidate's assessment evidence to bring into this debrief.How H-Evaluate helps →

Don't miss: Scoring 'LLM application engineering' on ambition rather than what shipped — a smaller, well-evaluated retrieval feature should outscore a large, unproven one.

Stage 10 of 13Optional

0 of 3 done

Check references

References and background checks

Who runs it: Recruiter (references) + HR (checks)
Done when: At least two references that test specific competencies, plus only the checks that are lawful and relevant to the role.

Use when:: It is lawful where you hire, and specific evidence from a referee would change the decision.
Skip when:: Local law or company policy rules it out.

Key tool

Ask each referee

1What's something [candidate] built that your team still depends on today?
2What told them a change to a model-backed feature was actually an improvement, rather than just different?
3When a model feature didn't behave as expected, how did they explain that to product or support colleagues?

AI as an advantage4D framework

Delegate: Hand AI the drafting of reference questions aimed at the competency the panel was least sure about. Choosing referees, making the calls and interpreting the answers stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter write competency-based reference questions for an AI Engineer role. Context: I will paste the competency model and the one or two competencies where our evidence is thinnest — no candidate details. Produce: 3–4 questions per competency that ask for specific examples ('tell me about a time…') rather than ratings; an opening question about how the referee worked with the person and for how long; and a closing question about what their next manager could do to help them do their best work. Use British English and inclusive wording. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the questions ask for examples rather than opinions, that they stay on job-related competencies, and that none asks about health, family or other protected matters.
Diligence: References are taken with the candidate's consent and interpreted by people. Referee answers and candidate details never go into an AI tool; AI drafts the questions, people decide.

Don't miss: Asking a generic 'would you hire them again?' and treating a quick 'yes' as if it were evidence of anything specific.

Framework

Competency-based referencing

Treat references as a last structured conversation about the candidate's work, aimed at the competencies where your evidence is thinnest.

With the candidate's consent, ask for two people who saw their relevant work directly — ideally a former manager and a peer.
Accept suitable alternatives if a manager or peer isn't available, and never penalise a candidate for a former employer's no-reference policy.
Aim your questions at the one or two competencies the debrief was least sure about.
Ask for examples, not ratings. 'Tell me about a time they…' works here too.
Set the timing of each background check by the law and policy that apply.

Why it works: Generic reference questions tend to produce little job-related evidence. Asking for specific examples tied to the role's competencies turns referencing into something you can use.

Core reference questions and checks

Open with: 'How did you work together, and for how long?' — it tells you how much weight the rest can carry.
Ask: 'Tell me about a time something they owned went wrong. What did they do?'
Close every reference with: 'What should their next manager do to help them do their best work?'
Verify only claims that were material to the decision — don't verify a degree the role never required.
Run criminal-record checks only where the role justifies them and local law allows; many roles don't need one.
Schedule formal work-authorisation checks at the legally required time: before employment starts in the UK; for US Form I-9, Section 1 by the first day of paid work and Section 2 within three business days.

Reference questions specific to this role

1.What's something [candidate] built that your team still depends on today?
Listen for: Concrete, lasting work — not general praise.
2.What told them a change to a model-backed feature was actually an improvement, rather than just different?
Listen for: Confirmation, from someone else's perspective, that their evaluation habit was real and not just interview talk.
3.When a model feature didn't behave as expected, how did they explain that to product or support colleagues?
Listen for: A specific example of translating a model limitation into plain language for a non-engineer, not a general character reference.

Compliance checkpointUS

If a consumer reporting agency provides a background report, the Fair Credit Reporting Act requires a clear, stand-alone written disclosure and the candidate's written authorisation before you order it. Before rejecting someone because of the report, give them a copy and the FCRA summary of rights and a reasonable opportunity to review and dispute it before any adverse action; if you still reject, send a final adverse-action notice. State and local 'fair chance' laws may delay criminal-history questions or checks until after a conditional offer.

Sources: FTC — Background checks: what employers need to know · FTC — Using consumer reports: what employers need to know · checked 2026-09-25

Stage 11 of 13Required

0 of 2 done

Agree the offer

Offer, negotiation and close

Who runs it: Hiring manager + recruiter
Done when: An accepted written offer within the approved band, with a start date.

Key tool

Build the offer around

•The problem: they'll own evaluation for features real customers use.
•Ownership of a monthly model-experimentation budget, theirs to allocate without asking permission each time.
•Support for growth: a conference budget and paid time to write publicly about what they've shipped.
•A transparent levelling conversation: why they're slotted where they are, and exactly what the next level would take.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the offer message built around what the candidate said matters to them. The figure, the approval and the conversation stay with the hiring manager.

Show the prompt and the checks

Describe: You are helping a hiring manager draft an offer message for an AI Engineer role. Context: I will paste three general priority categories (for example: learning, flexibility, scope) — not the candidate's words, notes or personal details; the parts of the offer; and the start-date options — without the salary figure. Produce: a warm, plain-English message that links each part of the offer to one of their priorities, says what happens next and when the written offer follows, and invites questions. Use British English and inclusive wording, and leave [brackets] for figures and names. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that nothing promised is outside policy or the approved band, that every claim about the role is true, and that the message reads as personal rather than templated.
Diligence: The hiring manager places the offer in the band using the same pre-set factors for everyone and makes the verbal offer personally. Figures and personal details are filled in by people, never pasted into AI.

Don't miss: Leaving the evaluation-ownership scope and experimentation budget for the candidate to ask about, rather than making them visible in the offer conversation.

Framework

Close plan (motivators → offer → objections)

Build the offer around what the candidate told you they care about, anticipate their objections, and make sure the verbal and written offers match.

List the candidate's top three motivators from your notes at every stage.
Make the verbal offer personally, and link each part of it to a motivator.
Ask what else they are weighing and when they have to decide.
Place the offer in the band using the same pre-set, job-related factors for everyone, and record why.
Send the written offer promptly after the verbal one.

Why it works: A close plan ties the offer to the candidate's stated priorities and makes approvals explicit before the final interview.

Offer checklist and negotiation

Stay inside the approved band. If the candidate is worth more, re-level them with evidence rather than stretching the band.
Trade in things that cost less than salary where policy allows: start date, learning budget, working pattern.
Before the written offer, ask: 'What would make this an easy yes?'
Agree approval and the pay band before the final interview, so the offer isn't held up.
The hiring manager makes the verbal offer; the written offer follows within a working day.
Record which pre-set factors placed the offer where it is in the band.
Tell unsuccessful finalists personally, without avoidable delay, with brief feedback.

What will close this candidate

The problem: they'll own evaluation for features real customers use.
Ownership of a monthly model-experimentation budget, theirs to allocate without asking permission each time.
Support for growth: a conference budget and paid time to write publicly about what they've shipped.
A transparent levelling conversation: why they're slotted where they are, and exactly what the next level would take.

Compliance checkpointEUUS

Use objective, gender-neutral, pre-agreed factors to place candidates within the band, apply them consistently and record the decision as an internal pay-equity control. Check the national law implementing Directive (EU) 2023/970 and any local record-keeping rules; this recommendation is good practice, not a universal statutory documentation rule.

Sources: EU Pay Transparency Directive 2023/970 · checked 2026-09-25

Stage 12 of 13Required

0 of 2 done

Help them settle in

Pre-boarding and 30-60-90 onboarding

Who runs it: Hiring manager + buddy + HR
Done when: A new hire who has worked through a written 30-60-90 plan tied to the 12-month outcomes, with a check-in at each milestone.

Key tool

Before day one
Get the laptop, account access and repository permissions sorted before their first day, not on it.
Days 1–30: learn
Sits down individually with the product, support and data people whose work depends on the AI features — not just a group introduction.
Days 31–60: contribute
Builds or hardens the evaluation set for the feature the intake conversation flagged as highest-risk.
Days 61–90: own
Owns quality for one feature end to end, with evaluation running before each release.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the 30-60-90 plan from the 12-month outcomes. Agreeing the plan with the new hire stays with the manager.

Show the prompt and the checks

Describe: You are helping a hiring manager draft a 30-60-90 day onboarding plan for a new AI Engineer. Context: I will paste the 12-month outcomes from the intake meeting, the systems and people the hire will work with, and anything they will need before day one. Produce: a pre-boarding checklist (equipment, access, a named buddy, a first-week schedule); then three sections — days 1–30 learn, days 31–60 contribute, days 61–90 own — each with 3–4 concrete, checkable expectations and one milestone check-in question. Use British English and plain wording. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each expectation is realistic for a new starter, ties back to a 12-month outcome, and offers the hire support rather than a test.
Diligence: The manager agrees the plan with the new hire in week one and owns the check-ins. The new hire's personal details stay out of the prompt.

Don't miss: Handing a newly started AI engineer a brand-new feature to design before they've read how the existing ones are evaluated and where those evaluation sets fall short.

Framework

30-60-90 day plan

Three phases — learn, contribute, own — with written expectations for each, agreed with the new hire in week one.

Before day one: equipment, access, a named buddy and a first-week schedule.
Days 1–30, learn: systems, people and the current state of the work.
Days 31–60, contribute: a first scoped piece of work shipped with support.
Days 61–90, own: an area of responsibility with clear outcomes.
Hold a check-in at each milestone against the written plan.

Why it works: Hiring doesn't end at the signature. A written onboarding plan makes expectations and support visible to the new hire and their manager.

Before day one
- Get the laptop, account access and repository permissions sorted before their first day, not on it.
- A model-provider API key with a spending cap set before they need it.
- A named buddy from the engineering team.
- A reading list: architecture overview, the list of model-backed features, known incidents.
Days 1–30: learn
- Sits down individually with the product, support and data people whose work depends on the AI features — not just a group introduction.
- Maps every model-backed feature and how each is evaluated today (if at all).
- Ships one small, real fix to production in the first month, purely to learn the deployment path end to end.
Days 31–60: contribute
- Builds or hardens the evaluation set for the feature the intake conversation flagged as highest-risk.
- Catches and writes up at least one quality regression that nobody had measured before, using their own evaluation work rather than a user's bug report.
- Adds cost and latency tracking per feature.
Days 61–90: own
- Owns quality for one feature end to end, with evaluation running before each release.
- Ships one improvement with a measured before-and-after result attached to it, not just a claim it's better.
- Presents what they found to the product team, in language a non-engineer can actually follow.

Compliance checkpointUSUK

US: the employee completes Form I-9 Section 1 no later than their first day of paid work, and the employer completes Section 2 within three business days after that day (on the first day if the job lasts less than three business days). UK: complete the prescribed right-to-work check before employment starts, and apply it the same way to everyone.

Sources: USCIS — Form I-9 instructions · GOV.UK — Check a job applicant's right to work · checked 2026-09-25

Stage 13 of 13Required

0 of 3 done

Measure time to hire

Measure the process: mean time to hire

Who runs it: Recruiter / talent lead + hiring manager
Done when: A stage-by-stage timing record for this hire, the slowest step named, and one change agreed before the next requisition opens.

Key tool

Record when the hire accepts

•Time to hire: days from the successful candidate entering the pipeline (application or first contact) to the accepted offer. Average it across hires for mean time to hire.
•Time to fill, recorded separately: days from requisition approval to the accepted offer.
•Days per stage, from the stage's first action to its last — and whether the time was spent waiting or working.
•Offer-acceptance rate: offers accepted divided by offers made.
•Where candidates dropped out, and the reason you were given.
•Days the completed work sample sat waiting for the senior engineer's review, separate from the days spent actually marking it.

Check in at 90 days whether the hire is meeting the outcomes in the job description — that tells you whether the process picked well.

Read: mean time to hire Read: time to hire vs time to fill

AI as an advantageAI boost4D framework

Delegate: Hand AI the job of building a blank timing worksheet and the instructions for reading it. Entering dates, working out the timings and deciding what to change stay with the recruiter and the hiring manager.

Show the prompt and the checks

Describe: You are helping a recruiter and a hiring manager set up a timing review for an AI Engineer hire. Context: I will list the stage names we used, in order (for example requisition approved, sourcing, screening, recruiter call, work sample, interviews, debrief, references, offer accepted). Produce: (1) a blank worksheet with one row per stage and columns for start date, end date, days in stage, days spent waiting (scheduling, approvals, no reply) and number of candidates who entered and left; (2) step-by-step instructions for us to work out, from our own records, time to hire (the successful candidate's entry to the pipeline, by application or first contact, to the accepted offer; average it across hires for mean time to hire) and, separately, time to fill (requisition approval to the accepted offer); (3) five questions we should ask about the slowest stage, separating waiting from working; (4) a short template for recording one change to make before the next hire. Use British English. Do not include benchmarks or example figures. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the worksheet covers every stage you actually ran, that the two definitions are not mixed up, and that the questions point at the process rather than at any person.
Diligence: People fill in the dates and do the arithmetic from their own records, choose the one thing to fix and write it down. No dates, names or details about any candidate go into the prompt.

Don't miss: Treating every slow step as equally fixable — a marker's backlog and a pay-band dispute need different owners and different fixes.

Framework

Mean time to hire, by stage

Time to hire is the number of days from the successful candidate entering your pipeline (their application or your first contact) to their accepted offer; mean time to hire is that figure averaged across hires. Time to fill — requisition approval to acceptance — is a separate measure. Record both per stage, so you can see where the time went rather than only the total.

When the hire accepts, write down the date each stage started and ended, and the day the successful candidate entered the pipeline.
Work out the days per stage, the time to hire and the time to fill, and mark the slowest stage.
Note the offer-acceptance rate and the stage where most candidates dropped out.
Separate waiting time (scheduling, approvals, silence) from working time, and decide which one to shorten.
Fix one thing before the next hire, and write down what you changed.

Why it works: A single total hides where the delay sits. Per-stage timing points at the one step to fix, and the acceptance rate and drop-out points show whether candidates kept pace with the process.

Record for every hire

Time to hire: days from the successful candidate entering the pipeline (application or first contact) to the accepted offer. Average it across hires for mean time to hire.
Time to fill, recorded separately: days from requisition approval to the accepted offer.
Days per stage, from the stage's first action to its last — and whether the time was spent waiting or working.
Offer-acceptance rate: offers accepted divided by offers made.
Where candidates dropped out, and the reason you were given.
Check in at 90 days whether the hire is meeting the outcomes in the job description — that tells you whether the process picked well.

What to log for this role

Days the completed work sample sat waiting for the senior engineer's review, separate from the days spent actually marking it.
Whether the offered pay band matched what intake approved, or needed a late re-approval before the offer went out.
How many candidates the evaluation-focused work sample removed, and whether most left before the brief was even sent or after they'd read it.

Where this role's hiring slows down

Watch for the work-sample deep-dive stalling behind one marker's other commitments, with nobody able to step in as a backup.
Watch for the system-design round repeating ground the work-sample round already covered instead of testing something new.
Watch for candidates withdrawing soon after the AI-tool policy is explained. Ask whether the AI-tool policy or its explanation contributed; record volunteered reasons without assuming the cause.

Compliance checkpointUKEU

Under the EU and UK GDPR, tell candidates how long you will keep their data, or the criteria used to decide. Give that notice when you collect data from them directly, or within the Article 14 time limit when it comes from another source (such as a referral or a sourcing tool). Delete or anonymise data when it is no longer needed, keeping only what a documented lawful purpose still requires.

Sources: GDPR — Regulation (EU) 2016/679, Articles 5, 13 and 14 · ICO — UK GDPR guidance · checked 2026-09-25

General information, not legal advice. Rules depend on the employer, the role and where the work is done — confirm what applies with HR or counsel.

Frequently asked questions

Where does this role stop and a machine learning engineer's begin?

The titles overlap in practice. An AI engineer role centres on product features built with existing or adapted models; a machine learning engineer role centres on data, training and production model systems. Fine-tuning, serving and evaluation work can sit in either role, so define the boundary from what your team actually needs, not the job title alone.

How long does it take to hire an AI engineer?

With this process, plan for four to six weeks from an approved requisition to an accepted offer. Scheduling and offer approval are the two places delay creeps in most easily — settle the pay band and hold panel time before the first candidate arrives.

Should candidates be allowed to use AI tools in the work sample?

Yes, where your company's AI-tool policy allows it. Say so in the brief, ask candidates to tell you which tools they used and how, and assess whether they checked what the tools produced rather than accepting it as-is.

What is the most important skill to test?

Evaluation discipline — the habit of measuring whether a feature works before trusting a change. It's the clearest difference between someone who has shipped reliable AI features and someone who has followed tutorials.

How many interview rounds does an AI engineer need?

Start with the work sample and three focused interviews. Add another stage only when it tests a distinct, job-related criterion not covered elsewhere.

Do we need an AI engineer or a strong software engineer?

Map your model-backed features, how much evaluation and fixing each one needs, and who does that work now. When that list adds up to ongoing quality work that doesn't fit alongside existing jobs, a dedicated AI engineer makes sense.

Go deeper

Related hiring kits

Guides

Skills tests

See your first AI-generated assessment in 5 minutes

No credit card. No question library to learn. Paste a job description and watch H-Evaluate build.

Request early access