Machine Learning Engineer hiring kit

Every step to hire the engineer who trains, evaluates and ships models that hold up once real traffic hits them.

About this kit — the 5Ws

A machine learning engineer builds and maintains the models behind a product feature — not just the pipeline that trains one, but the evaluation and monitoring that keep it honest after launch. The job sits closer to software engineering than to research: reading a teammate's training code, catching leakage (when test data accidentally reveals the answer, making a model look better than it really is), and deciding when a simple model beats a fancy one. This version is calibrated for a production machine learning engineer working on supervised predictive models — churn, fraud, pricing and similar problems. Adapt the work sample and technical questions if you're hiring for ranking, recommendation, computer vision, speech or foundation-model work instead.

Calibrated for:: Mid-level through senior, individual-contributor track.
Timeline:: Set an approval-to-offer target at intake, allowing for panel availability, candidate adjustments, references and any lawful checks.

Who: Hiring managers and recruiters at companies training or fine-tuning their own models — including teams hiring their first machine learning engineer.
What: Use this kit to approve the role, define production-ML evidence, test evaluation and deployment judgement, interview consistently, and review whether the hire improved model reliability.
When: Start using it as soon as a model needs to be trained, retrained or kept accurate over time — before the requisition is approved — and run it right through to logging how long each stage of the hire actually took.
Where: Teams training or fine-tuning production models in-house.
Why: The work sample, structured questions and rubric let you compare job-related evidence consistently. For this role, that means checking whether a candidate verified an evaluation before trusting it, not only whether they can name the right techniques.

The competency model · 7

Use these competencies to compare job-related evidence. Review any apparent gap in a must-have before deciding.

Data and evaluation rigourMust-have · 22
Builds an honest evaluation before trusting a model change; catches a leaked feature, an unfair baseline or a mismatched metric; reports where a model fails, not just its headline score.
Model development judgementMust-have · 18
Chooses an approach — including 'a simple baseline is enough' — based on the data and the problem rather than habit; can explain the trade-offs behind the choice.
Production ML engineeringMust-have · 20
Ships pipelines that are tested, versioned and reproducible; handles retraining, monitoring and rollback without a manual scramble.
Software engineering craftMust-have · 10
Writes readable, tested code that a teammate can maintain and review; treats a training script with the same discipline as any other production code.
Cost, latency and infrastructure judgementCan learn · 10
Knows what training and serving a model costs in time and compute; picks a model size and serving approach on purpose rather than by default.
Communication and stakeholder translationCan learn · 10
Explains what a model can and cannot do to people who don't work in ML; ties a metric back to the business outcome it's meant to move.
Responsible AI and model risk awarenessCan learn · 10
Notices bias, drift and misuse risk in a model's data or outputs and raises it without being asked.

What's required and what's optional

Your hiring journey for Machine Learning Engineer. Select a stage to jump to it.

Required for every hire
Optional — use when it adds evidence
AI boost — a big drafting win
Colour = phase

Phase 1
Plan
1. 01Plan the roleRequired
2. 02Agree successRequiredAI boost
Phase 2
Attract
1. 03Write the jobRequiredAI boost
2. 04Find candidatesOptionalAI boost
Phase 3
Select
Phase 4
Decide
Phase 5
Onboard & learn
1. 12Help them settle inRequired
2. 13Measure time to hireRequiredAI boost

AI drafts, people decide: Use AI to draft, rewrite and summarise your own material — never to rank, score, screen out or decide on candidates, and never with a candidate's personal data.

Plan

Attract

Select

Decide

Onboard & learn

Solid = required · dashed = optional · arrow keys move between steps

Stage 1 of 13Required

0 of 3 done

Plan the role

Workforce planning and requisition approval

Who runs it: Hiring manager + finance / HR business partner
Done when: An approved requisition with a written business case, budget and a named decision-maker.

Key tool

Answer these before the role is approved

1Which models will this person own in their first year — training, retraining or both?
2How do we know today whether our existing models are still accurate?
3Could an existing engineer or data scientist grow into this within two quarters?

AI as an advantage4D framework

Delegate: Hand AI the first draft of the business case and the build / borrow / buy comparison. The decision, the budget and the sign-off stay with people.

Show the prompt and the checks

Describe: You are helping a hiring manager write a business case for a Machine Learning Engineer role. Context: I will paste the problem the hire solves, the work it involves, the capacity we already have and the time we have. Produce: (1) a one-sentence problem statement in business terms; (2) a build / borrow / buy comparison with the strongest argument for each; (3) the questions a finance partner would ask; and (4) a short recommendation with the evidence it rests on. Use British English and plain language. Do not invent figures — leave [brackets] where I need to supply numbers. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check every number and claim against your own notes and strike anything the draft assumed. Make sure 'buy' is argued from ongoing work, not from a job title.
Diligence: The hiring manager owns the final case and the budget request. No candidate information belongs here; AI drafts the case, people decide whether to open the role.

Don't miss: Use the work to define the boundary: model development and production ML may sit with an MLE; model-backed application engineering may sit with an AI engineer; product decisions remain jointly owned by product and engineering. State the division explicitly in the vacancy.

Framework

Build · Borrow · Buy

Before opening a role, decide whether to build the capability in your current team, borrow it (contractor, agency, partner), or buy it (a permanent hire).

Write the problem the hire solves in one sentence, in business terms — not the job title.
Test Build: could an existing person grow into this within the time you have?
Test Borrow: is the need time-boxed, or does it end once a system is shipped?
Choose Buy only when the work is ongoing, core to the product and nobody inside can absorb it.
Record the decision, the budget and the one person who signs off the final hire.

Why it works: Writing the build, borrow or buy decision down exposes a vague or temporary need before it becomes a permanent requisition — and gives every later stage an anchor to check against.

• Confirm who has final sign-off on the hire — typically whichever engineering or data-science leader will own the models this person joins to run.

Is this a dedicated role?

Decide whether the work is sustained and substantial enough for a dedicated owner by writing down the workload, the risk, the capacity you already have and the outcomes you expect.

Answer these before the role is approved

1.Which models will this person own in their first year — training, retraining or both?
Listen for: Named models with real usage today, or committed on the roadmap.
Red flag: 'We want to do more with machine learning' with nothing specific.
2.How do we know today whether our existing models are still accurate?
Listen for: A candid answer, even an uncomfortable one — most teams don't check this on any schedule, and that gap is the job.
3.Could an existing engineer or data scientist grow into this within two quarters?
Listen for: A named engineer plus a realistic path for them to pick up evaluation and monitoring skills, or a clear reason that's not workable.
4.Is the need ongoing, or does it end once one model ships?
Listen for: Ongoing retraining, monitoring and new models justify a permanent hire; a single one-off project may suit a contractor.
5.What's the monthly cost of leaving this seat unfilled?
Listen for: A concrete consequence: a model quietly drifting, a launch delayed, nobody able to explain a prediction.

Build, borrow or buy?

Build: pair an existing data scientist or backend engineer with a senior mentor for two quarters, with dedicated time set aside for evaluation and deployment — not squeezed between other work.
Borrow: bring in a contractor to build the first retraining pipeline and write it up, with a fixed handover date to whoever inherits it.
Buy: hire permanently once more than one model needs continuous retraining and monitoring — a single one-off model rarely justifies it.

Watch out for

Assuming one hire will also cover the work of a data scientist, a platform engineer and a researcher. Write down which of those this role actually covers.

Stage 2 of 13Required

0 of 2 done

Agree success

Intake meeting and competency model

Who runs it: Recruiter + hiring manager
Done when: A signed-off competency model: 5–7 competencies, each marked must-have or trainable, with weights.

Key tool

Ask the hiring manager

1Which of the systems or areas this person will own worries you most, and why?
2When something there went wrong recently, how did you find out?
3In 12 months, what will be shipped, fixed or running for you to call this hire a success? Agree two or three measurable outcomes.
4Who will they work with most closely?
5What could a strong candidate learn in their first three months? Those capabilities are 'trainable' — never use them to filter applications.

AI as an advantageAI boost4D framework

Delegate: Hand AI your raw intake notes to turn into a draft competency model. Choosing the must-haves and the weights stays with the hiring manager and recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter and a hiring manager build a competency model for a Machine Learning Engineer role. Context: I will paste the notes from our intake meeting, including the 12-month outcomes and what the hiring manager worries about most. Produce a table of 5–7 observable competencies with, for each: a one-line definition; whether it is a must-have or trainable, with the reason; what evidence of it looks like in real work; and a suggested weight so the weights total 100. Use British English and inclusive wording, avoid years-of-experience or named employers as evidence, and keep every competency job-related. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each competency is observable and job-related, that nothing describes a type of person rather than the work, and that trainable items haven't been marked must-have.
Diligence: The hiring manager signs off the final model and weights before anyone meets a candidate. No candidate data goes into the prompt; AI drafts, people decide.

Don't miss: Listing a specific framework or library as a must-have. Tools change every year or two; the underlying judgement doesn't.

Framework

Competency model (must-have vs trainable)

A short list of the observable capabilities the job needs, split into what the hire must bring on day one and what you will teach.

Start from the 12-month outcomes, then ask which capabilities produce them.
Keep 5–7 competencies. With more, interviewers struggle to score each one separately.
Mark each must-have or trainable. Only must-haves can count against a candidate.
Give each a weight so they sum to 100, agreed before anyone meets a candidate.
Write what evidence of each looks like, so interviewers know what they are listening for.

Why it works: Agreeing the criteria before meeting candidates stops the bar moving from one candidate to the next, and makes it harder for 'I'll know it when I see it' to decide the hire.

Core intake questions (ask for every role)

Book 45 minutes with the hiring manager before anything is advertised, and write the answers down.
Which of the systems or areas this person will own worries you most, and why?
When something there went wrong recently, how did you find out?
In 12 months, what will be shipped, fixed or running for you to call this hire a success? Agree two or three measurable outcomes.
Who will they work with most closely?
What could a strong candidate learn in their first three months? Those capabilities are 'trainable' — never use them to filter applications.
Who is on the interview panel, and is their time held for the next four to six weeks?

Questions specific to this role

1.How many models are in production today, and is there a single place anyone can see the full list?
Listen for: A real number and a source of truth — or an honest 'we don't know', which shapes how much of the role is inventory and cleanup versus new work.
2.What's our current approach to catching a model that's quietly gone stale — automated monitoring, a scheduled manual check, or nothing yet?
Listen for: An honest description of what exists today; it sets how much of the first few months should go on monitoring rather than new modelling work.
3.What's the budget for training and experimentation compute, separate from salary?
Listen for: A number, or an honest 'we haven't set one' — it shapes what you can credibly offer and what the hire can actually run in their first few months.

What success looks like in 12 months

Every production model has a retraining and monitoring schedule with a named owner.
A specific accuracy or drift issue that existed before the hire started is smaller now, and you can show the before-and-after numbers.
Training and serving cost per model is tracked and understood.
The product team can explain, in plain language, what each model predicts and where it tends to be wrong.

Watch out for

Letting 'must have published research' creep in when the job is actually keeping an existing model healthy in production.

Stage 3 of 13Required

0 of 3 done

Write the job

Job description and pay band

Who runs it: Hiring manager (content) + recruiter (publishing)
Done when: A published job description that states outcomes, the must-haves, how AI tools are used and the pay information the law requires.

Key tool

Machine Learning Engineer

We're looking for a machine learning engineer to own the models behind [product]. You'll build and evaluate training pipelines, keep models accurate after they ship, and work closely with product and data teams to decide what's worth modelling at all.

What you'll do

•Build and maintain training pipelines, with tests, version control and a repeatable way to reproduce a result.
•Design evaluation that catches leakage, unfair baselines and mismatched metrics before a model ships.
•Monitor live models for drift and quietly worsening predictions, and own the fix when one appears.
•Track training and serving cost, and choose model size and infrastructure deliberately.
•Explain what a model can and cannot predict to product, data and non-technical stakeholders.
•Flag bias, misuse or safety risk in a model's data or outputs before it ships.

What you need

•You have trained and shipped at least one model to production, not only to a notebook or a competition leaderboard.
•You can show how you evaluated it — a held-out test (checked against data the model never trained on), a baseline comparison, or a check for leakage in the data.
•You write production-quality code in [Python/other], with tests.
•You can explain a technical trade-off clearly to someone who isn't a machine learning engineer.

Nice to have

•Experience retraining or monitoring a model after it shipped, not only building the first version.
•Experience with feature pipelines or a feature store.
•Experience reducing training or inference cost in production.
•Some familiarity with AI regulation, such as the EU AI Act, as it applies to model risk.

How we use AI tools

•Where company policy permits, use approved AI tools and verify their output — including on training code and evaluation scripts you didn't write yourself.
•You'll know when a modelling problem needs a person's judgement and when a tool can safely help.
•You're welcome to use AI tools on the work sample. Note in your submission which ones you used and where they helped.

Pay

Salary range: [currency, minimum–maximum], plus [equity/bonus/benefits]. The range reflects [location/working pattern]. We place every offer within it using the same pre-set, job-related, gender-neutral factors — Level — the seniority and scope of the models they'd own, Production experience — has shipped and maintained a model after launch, not only trained one, Depth across the pipeline — data, training, evaluation and deployment, Relevant infrastructure or domain experience for this team's stack — never previous salary or negotiating style.

AI as an advantageAI boost4D framework

Delegate: Hand AI the first draft of the job description from your signed-off competency model. The must-haves, the pay range and the final wording stay with people.

Show the prompt and the checks

Describe: You are helping write an outcome-based job description for a Machine Learning Engineer role. Context: I will paste the competency model (must-have and trainable, with evidence), the 12-month outcomes, how the team uses AI tools, the location and working pattern, and the pay range. Produce: a summary paragraph that leads with the problem and what the hire will own; 'What you'll do' as outcomes, not tasks; 'What you need' containing only the must-haves; 'Nice to have' for the trainable items; 'How we use AI tools'; and a pay section that states the range. Use British English and inclusive, gender-neutral wording, avoid hype, and replace any 'X years' with the capability it stands for. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that 'What you need' contains only the must-haves, that no degree, employer or years requirement crept in, and that the pay range and location are exactly what was approved.
Diligence: The hiring manager owns the content and the recruiter owns publishing, including the pay information the law requires for each location. AI drafts the advert; people decide what it says.

Don't miss: Bundling data-engineering or platform/MLOps ownership into the same posting. It's different day-to-day work — advertise it as a separate role if you also need one.

Stage 4 of 13Optional

0 of 3 done

Find candidates

Sourcing strategy

Who runs it: Recruiter / sourcer
Done when: A shortlist-ready pipeline from at least three channels, with outreach tracked.

Use when:: Inbound applications won't give you enough qualified candidates.
Skip when:: Your shortlist already comes from inbound applications or referrals.

Key tool

Search query

("machine learning engineer" OR "ML engineer" OR "applied machine learning") AND ("evaluation" OR "model monitoring" OR "feature engineering") AND ("production" OR "shipped" OR "deployed") AND (Python OR PyTorch OR scikit-learn) NOT ("intern" OR "student")

First outreach message

AI as an advantageAI boost4D framework

Delegate: Hand AI the search queries and the first draft of the outreach template. Choosing channels, personalising to a candidate's public work and sending stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter source candidates for a Machine Learning Engineer role. Context: I will paste the must-have competencies, the kind of work we want evidence of, the channels we are using and the pay range. Produce: (1) one search query per channel, using OR for alternatives, AND for things that must all appear and NOT to exclude, with a one-line note on what each query looks for; and (2) a first-outreach template of under 120 words that opens with a [placeholder for something the candidate built], states the problem we are hiring to solve and the pay range, and ends with a low-pressure question. Use British English and inclusive wording. Do not refer to any specific person. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the queries search for evidence of the work rather than job titles alone, and that the template claims nothing about the role or the pay you can't back up.
Diligence: The recruiter personalises every message to something the candidate actually built, and sends it. No names or profile data go into the prompt; AI drafts, people decide whom to contact.

Don't miss: Search for training, evaluation and monitoring work across machine-learning engineer, data scientist, applied scientist and backend-engineer titles.

Framework

Channel mix + evidence-first outreach

Spread sourcing across channels where the work is visible, and open outreach with the problem, not the perks.

Pick channels where candidates show their work, not only where they list job titles.
Write one search query per channel. OR joins alternatives, AND joins things that must all appear, NOT excludes. Keep the version that works best.
Personalise the first line to something the candidate actually built or wrote.
Lead with the problem and the pay range. Save the perks for later.
Track reply rates by channel each week and drop the channel that isn't working.

Why it works: Searching for evidence of the work can surface candidates whose current title differs from the vacancy, or whose CV undersells what they have done.

Sourcing basics

Adapt the search query for each platform — the same words behave differently on each site.
Candidates with adjacent titles may have relevant evidence. Search for the work and outcomes the role needs, not the current title.
Channel activity is a lead, not evidence. Review linked work and examples against the written competencies.

Where to look

Public code repositories

A training pipeline with tests and a written evaluation is reviewable evidence of the discipline this role needs.

Technical write-ups and conference talks

A write-up on a model that failed, and how the failure was found, is reviewable evidence of evaluation rigour.

Your own data and engineering team's referrals

Someone who ran a production model with the candidate can point you to monitoring or retraining work worth asking about.

LinkedIn and similar networks, filtered on what people have shipped rather than their current job title

Search by the training, evaluation or monitoring work described, since the title alone will not show it.

Communities around open-source ML tooling

Answers to technical questions in these spaces can be checked directly against what the person actually built.

Boolean search string

("machine learning engineer" OR "ML engineer" OR "applied machine learning") AND ("evaluation" OR "model monitoring" OR "feature engineering") AND ("production" OR "shipped" OR "deployed") AND (Python OR PyTorch OR scikit-learn) NOT ("intern" OR "student")

First outreach message

Hi [name] — [specific detail from their repo or write-up] caught my eye. At [company], our churn model has been running for a year and nobody's checked lately whether it still holds up — that's the gap we're hiring a machine learning engineer to close, paying [range]. Worth a 20-minute conversation about what 'still holds up' would actually mean here?

Stage 5 of 13Required

0 of 3 done

Check applications

Application screening

Who runs it: Recruiter
Done when: Every application reviewed against the same written knockouts within five working days.

Key tool

Reject only if one of these is missing

•Can meet the role's work-authorisation requirements by the start date, with sponsorship where the employer offers it. Don't ask for documents at this stage.
•Describes at least one model that reached production, not only a notebook or a competition entry.
•Shows software engineering experience in any language, not only notebook scripts.

AI as an advantage4D framework

Delegate: Hand AI the job of turning your written knockouts into a consistent checklist with reason codes. Reviewing each application stays with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter prepare an application-screening checklist for a Machine Learning Engineer role. Context: I will paste the must-have competencies and our written knockout criteria. Produce: (1) each knockout as a yes/no question answerable from an application alone; (2) a short list of reason codes for a 'no'; and (3) a note of any criterion that looks like a proxy — a degree, employer brand or employment gap — rather than a job-related requirement, with a suggested rewrite. Use British English. Do not evaluate any applicant and do not ask for applicant details. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each question is binary and job-related, and that every flagged proxy has been removed or justified in writing.
Diligence: A person reviews every application against the checklist and records the reason. Never paste CVs or applicant details into an AI tool, and never let a tool screen anyone out — in New York City and the EU that can bring automated-decision rules into play.

Don't miss: Rejecting a candidate for an unconventional path into the field before checking whether their portfolio shows a model shipped and monitored in production.

Framework

Knockout criteria

A small set of yes/no requirements, written in advance, applied to every applicant the same way.

Derive knockouts only from must-have competencies and genuine legal requirements.
Keep them binary and answerable from the application itself.
Apply them in the same order to every application and log the reason for each rejection.
Do not use proxies such as university, employer brand or employment gaps as knockouts.
Anything ambiguous gets a second review or moves forward to the skills stage.

Why it works: Early screening rejects people before richer evidence exists, so its criteria need the most care. Written criteria make decisions reviewable — but each one must also be job-related, applied consistently and checked for unjustified adverse impact.

Screening rules

Do not use degrees, employment gaps or certifications as filters unless they are demonstrably job-related and justified for this role.
Judge the claims in a CV, not how polished it is — a well-written CV may or may not have had AI help, and either way it is the evidence that matters.

Knockout criteria

Can meet the role's work-authorisation requirements by the start date, with sponsorship where the employer offers it. Don't ask for documents at this stage.
Describes at least one model that reached production, not only a notebook or a competition entry.
Shows software engineering experience in any language, not only notebook scripts.

Green flags

Mentions how a model was evaluated: a held-out set, a baseline comparison, a check for a leaked feature.
Describes a model that went wrong after launch and what they changed because of it.
Links to code, a write-up or a result you can check.

Red flags

Lists many algorithms and libraries with no outcome attached to any of them.
A portfolio that's all Kaggle leaderboards and course exercises, nothing that ever ran against real traffic — a fair pattern at entry level, a mismatch for this seniority.
Skills claimed on the CV with no project the candidate can point to.

Watch out for

Treating a long list of competition results as evidence of production skill. They test a different thing.

Compliance checkpointNew York CityEU

For a New York City employment decision, work out whether a tool is an 'automated employment decision tool' (AEDT) under Local Law 144 — one that uses covered techniques to substantially assist or replace a discretionary decision. Before using a covered tool, make sure an independent bias audit was completed no more than a year earlier, publish the required summary, and give covered candidates or employees the required notice at least ten business days before use. In the EU, the AI Act (Annex III) lists AI used to recruit, filter applications or evaluate candidates as high-risk, subject to the Article 6(3) exception; the related obligations apply from 2 December 2027 under the amended timetable. A manual screen does not exempt an AI tool used elsewhere in the process.

Sources: NYC DCWP — Automated employment decision tools · EU AI Act — consolidated Regulation (EU) 2024/1689 · checked 2026-09-25

Stage 6 of 13Optional

0 of 3 done

Recruiter call

Recruiter screen

Who runs it: Recruiter
Done when: A 20–30 minute call that confirms logistics, motivation and one or two must-have signals.

Use when:: You are hiring at volume, or pay and logistics need confirming early.
Skip when:: You have only a handful of applicants — fold the pay and logistics questions into application screening instead.

Key tool

Ask every candidate, in this order

1Walk me through the last model that went from your laptop to actually running in production. What changed along the way?
2How did you know it was working before you shipped it?
3In plain terms, what does that model predict, and who acts on the prediction?
4Is that model retrained on a schedule, or only when someone notices a problem?
5Our range for this role is [range]. Does that work for you?

AI as an advantage4D framework

Delegate: Hand AI the first draft of the screening-call script and its evidence anchors. Running the call and taking the notes stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter write a 20–30 minute structured screening call for a Machine Learning Engineer role. Context: I will paste the must-have competencies, the logistics we need to confirm (location, working pattern, notice period) and the pay range. Produce: an opening that explains the process, the timeline and the pay range; 4–6 questions in a fixed order, each with one line on what a strong answer contains; and a closing that asks whether the published range works. Use British English and inclusive wording. Do not include questions about salary history, age, family, health or any other protected characteristic. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that every question is job-related and lawful, that the pay question asks whether the range works rather than for history, and that the anchors describe evidence, not impressions.
Diligence: The recruiter asks the same questions of everyone and writes the notes. No candidate details go into the prompt; AI drafts the script, people run the call and decide.

Don't miss: Expecting the recruiter to weigh in on evaluation methodology. Their job here is motivation, logistics and clarity of explanation — the technical panel judges depth.

Framework

Structured screening call

The same short set of questions for every candidate, with notes taken against what a strong answer contains.

Open by explaining the process, timeline and pay range. No surprises later.
Confirm logistics: location, working pattern, notice period and whether the range works for them.
Ask the same 4–6 questions in the same order, and note evidence, not impressions.
Leave time for the candidate's questions. Don't score their questions unless they test a written, job-related competency.
Decide within 48 hours and tell the candidate either way.

Why it works: Using the same questions and written evidence anchors makes answers easier to compare, and a short, predictable call respects the candidate's time.

Screening call basics

Set the decision timetable before sourcing and tell candidates when to expect each update.
Ask every candidate what draws them to this particular role, and listen for interest in the work described rather than the title.
Confirm notice period and whether they are in other processes, so you can plan the timetable.
End every call by asking whether the published pay range works — never ask for salary history.

Recruiter screen questions

1.Walk me through the last model that went from your laptop to actually running in production. What changed along the way?
Listen for: A specific model, real usage and the candidate's own part in it.
Red flag: Can't separate their own decisions from what the rest of the data or ML team did.
2.How did you know it was working before you shipped it?
Listen for: Any structured evaluation — even a simple held-out test counts.
Red flag: 'It scored well in training' with no separate check.
3.In plain terms, what does that model predict, and who acts on the prediction?
Listen for: A clear, jargon-free explanation of what the model does and who relies on it.
4.Is that model retrained on a schedule, or only when someone notices a problem?
Listen for: Whether the candidate can describe how retraining is triggered — scheduled, monitored or ad hoc — and their own role in it.
Red flag: No idea whether or how the model gets updated after it ships.
5.Our range for this role is [range]. Does that work for you?
Listen for: A clear answer on whether the published range works. Don't ask for salary history — this process sets pay from the published band and pre-set, job-related factors.

Compliance checkpointUS (state and city)EU

Don't ask for salary history. In the EU, national laws implementing Article 5(2) of Directive (EU) 2023/970 must prohibit the question. US rules vary, but jurisdictions including California and New York prohibit employers from seeking salary history. Ask whether the published range works instead.

Sources: EU Pay Transparency Directive 2023/970, Article 5 · California Labor Commissioner — salary history · New York State DOL — salary history ban · checked 2026-09-25

Stage 7 of 13Required

0 of 2 done

Work sample

Skills assessment / work sample

Who runs it: Hiring manager (design) + recruiter (logistics)
Done when: Every shortlisted candidate completes the same job-shaped task, marked against criteria written in advance.

Key tool

Find out why a churn model looks too good to be true

You're handed a small pipeline that trains a customer-churn model: a training script, a saved model, and an evaluation notebook reporting an accuracy score that looks too good to be true. Work out what's wrong with the evaluation, fix it, and retrain a baseline you'd actually trust — showing honest before-and-after results, including anything that gets worse.

Planned for up to three hours (adjustments available on request). At the agreed stopping time, write a short note on what you would examine next, and what you deliberately left out.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the work-sample brief and its marking sheet. Choosing the task, setting the timebox and marking stay with the hiring manager.

Show the prompt and the checks

Describe: You are helping a hiring manager design a work sample for a Machine Learning Engineer role. Context: I will paste the competency model, a real problem the team solved recently with anything confidential removed, the timebox, and our AI-tool policy for candidates. Produce: a candidate-facing brief (what to do, what to hand in, the time cap, how to request an adjustment, which AI tools are allowed and how to report their use); and a marking sheet with 4–6 criteria tied to named competencies, each with what a 1, a 3 and a 5 look like. Use British English and plain, inclusive wording. Do not design anything that needs more than the stated time. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the task fits the timebox, that every criterion maps to a competency, and that the brief gives all candidates the same starting materials.
Diligence: The hiring manager owns the task and the criteria, and people mark every submission. Do not paste candidate submissions into an AI tool for scoring; AI drafts the brief, people mark and decide.

H-EvaluateH-Evaluate can generate a job-specific assessment from your job description, so you don't have to build the task by hand.How H-Evaluate helps →

Don't miss: Handing out a task with no known-wrong baseline to diagnose against — without a seeded problem to find, you're measuring luck, not rigour.

Framework

Work-sample test

A small, realistic slice of the job, marked against criteria written in advance. Give candidates equivalent materials and arrange reasonable adjustments where required.

Take a real problem the team solved recently and shrink it to fit the timebox.
Write the marking criteria first, tied to the competency model.
Timebox honestly — for take-home work, a few hours at most — and say so in the brief.
State the AI-tool policy in writing. Where the job permits these tools, use equivalent conditions in the assessment and evaluate how candidates verify their output.
Mark without CV details where you can, and have two people mark borderline submissions.

Why it works: A well-designed work sample gives direct evidence of job-relevant skill rather than a proxy for it, and gives candidates a realistic preview of the work.

Assessment rules for every candidate

State the planned time cap and how to request an adjustment. At the agreed stopping time, ask candidates to note what they would examine next and what they left out — never reward unpaid overtime.
Give every candidate the same starting materials, and use a second marker for borderline submissions.
Say in writing which AI tools are allowed and ask candidates to tell you how they used them.
Invite every candidate to ask for adjustments, and answer each request promptly.
Pay for anything the task costs the candidate to run, such as model or cloud usage.

The work sample

Find out why a churn model looks too good to be true

Timebox: Planned for up to three hours (adjustments available on request). At the agreed stopping time, write a short note on what you would examine next, and what you deliberately left out.

What the candidate hands in

Your diagnosis: what was wrong with the original evaluation, and how you found it.
What you changed and why — enough detail that someone else could follow the reasoning, not just read the diff.
Before-and-after results from your corrected evaluation.

Marking criteria (write these first)

The candidate finds the actual problem with the evaluation, not a cosmetic issue.
The fix is aimed at the diagnosed problem, not random retuning.
Reports metric changes, segment effects and any weaker results after correcting the evaluation.
The candidate proposes a baseline or sanity check, not only a more complex model.
A colleague could pick this code up cold and understand what it does without asking for a walkthrough.
The closing note shows sound judgement about what mattered most and what was reasonably left for later.

Adjustments: Candidates can ask for more time or a live, pair-programmed version instead. We'll cover the cost of any compute the task needs — the starter repository runs on a laptop.

Watch out for

Banning AI tools outright, when your own engineers would use them for this work. Assess whether the candidate checks what the tool produces, not whether they used one at all.

Compliance checkpointUSGreat BritainEU

In the US, covered employers must provide a reasonable accommodation to a qualified applicant with a disability unless it would cause undue hardship (ADA). In Great Britain, the Equality Act 2010 requires reasonable adjustments in recruitment. In the EU, the duty comes from national equality law. Don't ask for more medical information than the law allows.

Sources: EEOC — prohibited practices · Equality Act 2010 · checked 2026-09-25

Stage 8 of 13Required

0 of 3 done

Structured interviews

Structured interview loop

Who runs it: Interview panel (named in advance)
Done when: Every competency has a lead interviewer and at least one second source of evidence, with the same questions for every candidate.

Key tool

Interview questions by pillar

Free in this kit: Domain questions for every round. The other three pillars and the AI-fluency check are measured by H-Evaluate.

Free in this kit

Senior machine learning engineer who marked the work sample · Data and evaluation rigour · Software engineering craft

1.In the work sample, what told you the evaluation was wrong rather than the model? Walk me through what you checked, and in what order.
Data and evaluation rigour
Good answerNames the specific failure found — for example leakage (test data quietly revealing the answer), an unfair baseline, or a mismatched metric — and describes a logical checking order: the data split first, then the features, then the metric definition.
Weak answerA vague sense that something was off, with no reproducible method or order of checks.
2.A model's offline score improves, but the gain disappears once you look only at the most recent month of data. What would you check before deciding what happened?
Data and evaluation rigour
Good answerChecks for temporal drift (the data changing shape over time), a leaking feature, a changed split, or a shift in which records show up in that recent month — and treats the gap as a signal to investigate rather than noise.
Weak answerConcludes the model is simply worse now from the one comparison, or reruns training without changing anything.
3.A training pipeline has a data-loading step, a feature-transform step and a model-scoring step. Which of those would you cover with an automated test, which would you check by hand, and why?
Software engineering craft
Good answerAutomates the deterministic logic — data transforms, feature calculations, the serving path — with tests, and explains clearly what can't be fully automated (for example, whether a new feature genuinely helps) and how they check that instead.
Weak answerNo tests at all, or tests that only confirm the pipeline runs without checking any output values.

Optional follow-ups (1)

4.A feature pipeline and a training pipeline produce different values for the same record. How would you locate the mismatch, and what would you put in place so it doesn't happen again?
Software engineering craft
Good answerCompares the two code paths systematically — transformations, library versions, timing of inputs — and proposes a parity check or a shared code path plus a regression test, not just a one-off fix.
Weak answerRetrains or tweaks the model until the numbers look acceptable, without finding the root cause.

AI as an advantageAI boost4D framework

Delegate: Hand AI the drafting of probe questions and strong / weak anchors for one competency. Asking, listening and scoring stay with the interviewer.

Show the prompt and the checks

Describe: You are helping an interviewer prepare a structured technical interview for a Machine Learning Engineer role. Context: I will paste one competency, its evidence description and the core question we ask every candidate. Produce: four probes in this fixed order — what would you do, why that choice, what else did you consider, how would you know it worked — tailored to the question; a one-line description of a strong answer and of a weak answer; and two follow-ups to use only if time allows. Use British English and inclusive wording. Keep everything about the work, nothing about the person. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the probes test the named competency rather than trivia, that the anchors describe observable evidence, and that no question touches a protected characteristic.
Diligence: Interviewers ask the same core questions of every candidate and score alone against the rubric. Interview notes and candidate names never go into an AI tool; AI drafts the questions, people interview and decide.

H-EvaluateH-Evaluate can add evidence on Cognitive, Situational judgement, Behavioural and AI fluency for your panel to review alongside these interviews.How H-Evaluate helps →

Don't miss: Letting the hiring-manager round turn into a second technical deep-dive, instead of testing how clearly the candidate explains a model's limits to someone outside ML.

Framework

Structured technical interview

Every candidate gets the same job-specific questions, each interviewer leads on specific competencies, and every answer is probed with the same four follow-ups: what would you do, why, what else did you consider, and how would you know it worked.

Give each competency a lead interviewer, plus a second source of evidence — the work sample, another round or an independent marker. Avoid asking the same thing twice, but never let one person's view be the only evidence for a critical competency.
Ask the three core questions for your round, in order, for every candidate. Use an optional follow-up only if there is time.
Probe each answer: why that choice, what else they considered, and how they would know it worked — until you hear the candidate's own technical decision.
Take notes on evidence during the interview. Don't rate until it's over.
Fill in your interviewer sheet and submit it before you talk to any other interviewer.

Why it works: Structured interviews make answers comparable: every candidate faces the same job-related questions and is scored against the same anchors.

Interview rules

Use the same core questions in the same order for every candidate.
In the work-sample review, understand the candidate's decisions against the rubric — don't turn it into a debate about your preferred solution.
Avoid trivia disconnected from a real job decision. Use a realistic debugging, design or evaluation problem instead.

Watch out for

Scoring 'production ML engineering' purely from the work sample, when the sample never touches deployment or monitoring — save that judgement for the production-systems round.

Compliance checkpointUSUKEU

Don't ask about protected characteristics — such as age, disability, pregnancy or family plans, religion, race or nationality — unless applicable law permits the question for a specific, necessary, job-related purpose, and HR or counsel has confirmed that basis beforehand. Ask only whether the candidate can meet the role's work-authorisation requirements. If a candidate volunteers protected information, don't probe; record only what is lawfully needed, keep it separate and restrict access.

Sources: EEOC — prohibited practices · Equality Act 2010 · checked 2026-09-25

Where to find more interview questions

Use web question banks for inspiration only: ask every candidate the same job-related questions, check each one against this kit's competencies, and never ask about protected characteristics. Don't reuse this kit's work sample from a public bank — candidates can find questions that circulate online, so a widely posted question tests preparation more than skill.

Show 7 sites

US OPM — Structured interviews(opens in a new tab)
US federal guidance on building structured interviews, with a guide, training material and sample questions with rating scales.
FreeFor interviewers
Acas — Interviewing job applicants(opens in a new tab)
UK public guidance on preparing fair interviews, asking applicants the same questions and avoiding discriminatory ones.
FreeFor interviewers
Workable — Interview questions library(opens in a new tab)
Sample interview questions for employers, listed by job role, including machine learning, data and engineering roles.
FreeFor interviewers
Indeed for Employers — Interview questions(opens in a new tab)
Sample interview questions for employers by job title and skill, with notes on what to look for in answers.
FreeFor interviewers
Tech Interview Handbook(opens in a new tab)
Open-source software-engineering interview preparation: coding, system design and behavioural questions.
FreeFor candidates and interviewers
Chip Huyen — Machine Learning Interviews Book(opens in a new tab)
Free online book with knowledge questions on maths, computer science and machine learning, grouped by difficulty.
FreeFor candidates and interviewers
Blind — Interview questions channel(opens in a new tab)
Anonymous posts by tech workers about questions they report being asked. Unverified; some content needs sign-in.
Free, some content needs sign-inCandidate reports, unverified

Links checked 2026-10-05. External sites; we don't control their content.

Stage 9 of 13Required

0 of 2 done

Compare evidence

Scoring, calibration and debrief

Who runs it: Hiring manager (chairs) + panel
Done when: A documented hire / no-hire decision traceable to scored evidence against the competency model.

Key tool

Interactive scorecard

Score one candidate against the rubric. Everything stays in your browser. Download it as a spreadsheet for your records.

Data and evaluation rigour Must-have

Weight

What do 1, 3 and 5 look like?

1: No real evaluation, or only 'the metric looked good'. Can't say how they'd know a change actually helped.
3: Builds a basic held-out evaluation when asked; catches the main issue but misses subtler ones.
5: Builds the evaluation first without being asked, actively hunts for leakage and unfair baselines, and reports regressions honestly.

Model development judgement Must-have

Weight

What do 1, 3 and 5 look like?

1: Reaches for the most complex model available with no reasoning.
3: Chooses a sensible approach and can explain it when asked.
5: Reasons from the data and the problem, takes a simple baseline seriously, and can defend the trade-off either way.

Production ML engineering Must-have

Weight

What do 1, 3 and 5 look like?

1: Pipeline works once, by hand; no versioning, tests or monitoring.
3: Tested, reproducible pipeline; handles retraining when prompted.
5: Designs for automated retraining, monitoring and rollback from the start.

Software engineering craft Must-have

Weight

What do 1, 3 and 5 look like?

1: Code is difficult to read or maintain; no tests.
3: Readable, tested code; responds well to review feedback.
5: Writes code a teammate could extend without asking, and gives review feedback that improves the codebase.

Cost, latency and infrastructure judgement Can learn

Weight

What do 1, 3 and 5 look like?

1: No idea what training or serving costs.
3: Tracks cost and latency and knows the usual ways to bring them down.
5: Chooses model size and infrastructure deliberately, with a clear view of what each choice costs and whether it's still worth it once quality is checked.

Communication and stakeholder translation Can learn

Weight

What do 1, 3 and 5 look like?

1: Can't explain a model's behaviour without jargon.
3: Clear explanations; connects work to business outcomes when asked.
5: Can walk a non-technical stakeholder through exactly where the model breaks down, and connects a modelling choice back to the business number it's meant to move.

Responsible AI and model risk awareness Can learn

Weight

What do 1, 3 and 5 look like?

1: Doesn't notice bias, drift or misuse risk even when pointed at it.
3: Recognises the main risks when they're raised.
5: Spots risks unprompted and proposes concrete mitigations.

Weighted score– / 5

AI as an advantage4D framework

Delegate: Hand AI the drafting of a blank debrief worksheet and agenda from the competency rubric alone. People add the evidence locally; the scores and the decision stay with the panel.

Show the prompt and the checks

Describe: You are helping a hiring manager prepare a debrief for a Machine Learning Engineer role. Context: I will paste the competency rubric only — the competencies, their weights and their 1, 3 and 5 anchors. Produce: (1) a blank debrief worksheet with one row per competency and empty columns for each interviewer's independent score, the evidence they cite, the spread of scores and an 'evidence gap?' box; (2) a 45-minute agenda: confirm every scorecard is in and reveal all scores at once, discuss the widest spreads using evidence only, review any must-have scored 1, then look at the weighted total and decide (hire, no hire or one targeted follow-up), and record the reasons; and (3) neutral questions the chair can ask to draw out evidence rather than adjectives. Leave every evidence and score cell empty. Do not suggest a score, a rating or a recommendation. Use British English. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the worksheet follows your rubric exactly — same competencies, weights and anchors — and that the agenda keeps independent scores hidden until the reveal.
Diligence: Interviewers fill in the evidence and scores locally, alone, before the debrief, and the panel decides together. Candidate notes and personal data never go into an AI tool; AI never scores, ranks or compares candidates.

H-EvaluateH-Evaluate gives you a report on each candidate's assessment evidence to bring into this debrief.How H-Evaluate helps →

Don't miss: Scoring 'model development judgement' high just because the candidate named advanced techniques, without checking whether the reasoning behind the choice actually held up.

Stage 10 of 13Optional

0 of 2 done

Check references

References and background checks

Who runs it: Recruiter (references) + HR (checks)
Done when: At least two references that test specific competencies, plus only the checks that are lawful and relevant to the role.

Use when:: It is lawful where you hire, and specific evidence from a referee would change the decision.
Skip when:: Local law or company policy rules it out.

Key tool

Ask each referee

1What did [candidate] build or fix that the team still depends on today?
2What did they check before calling a change to a model an improvement — and can you recall a time that check caught a problem?
3What was it like for a non-technical colleague to work with them?

AI as an advantage4D framework

Delegate: Hand AI the drafting of reference questions aimed at the competency the panel was least sure about. Choosing referees, making the calls and interpreting the answers stay with the recruiter.

Show the prompt and the checks

Describe: You are helping a recruiter write competency-based reference questions for a Machine Learning Engineer role. Context: I will paste the competency model and the one or two competencies where our evidence is thinnest — no candidate details. Produce: 3–4 questions per competency that ask for specific examples ('tell me about a time…') rather than ratings; an opening question about how the referee worked with the person and for how long; and a closing question about what their next manager could do to help them do their best work. Use British English and inclusive wording. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the questions ask for examples rather than opinions, that they stay on job-related competencies, and that none asks about health, family or other protected matters.
Diligence: References are taken with the candidate's consent and interpreted by people. Referee answers and candidate details never go into an AI tool; AI drafts the questions, people decide.

Don't miss: Accepting 'the model performed well' from a reference without asking how they knew — hold the reference to the same evaluation rigour you'd expect from the candidate.

Framework

Competency-based referencing

Treat references as a last structured conversation about the candidate's work, aimed at the competencies where your evidence is thinnest.

With the candidate's consent, ask for two people who saw their relevant work directly — ideally a former manager and a peer.
Accept suitable alternatives if a manager or peer isn't available, and never penalise a candidate for a former employer's no-reference policy.
Aim your questions at the one or two competencies the debrief was least sure about.
Ask for examples, not ratings. 'Tell me about a time they…' works here too.
Set the timing of each background check by the law and policy that apply.

Why it works: Generic reference questions tend to produce little job-related evidence. Asking for specific examples tied to the role's competencies turns referencing into something you can use.

Core reference questions and checks

Open with: 'How did you work together, and for how long?' — it tells you how much weight the rest can carry.
Ask: 'Tell me about a time something they owned went wrong. What did they do?'
Close every reference with: 'What should their next manager do to help them do their best work?'
Verify only claims that were material to the decision — don't verify a degree the role never required.
Run criminal-record checks only where the role justifies them and local law allows; many roles don't need one.
Schedule formal work-authorisation checks at the legally required time: before employment starts in the UK; for US Form I-9, Section 1 by the first day of paid work and Section 2 within three business days.

Reference questions specific to this role

1.What did [candidate] build or fix that the team still depends on today?
Listen for: Concrete, lasting work — not general praise.
2.What did they check before calling a change to a model an improvement — and can you recall a time that check caught a problem?
Listen for: Independent confirmation of evaluation rigour, plus honesty about a call that didn't pan out.
3.What was it like for a non-technical colleague to work with them?
Listen for: Specific moments of plain-language explanation or a disagreement handled well.

Compliance checkpointUS

If a consumer reporting agency provides a background report, the Fair Credit Reporting Act requires a clear, stand-alone written disclosure and the candidate's written authorisation before you order it. Before rejecting someone because of the report, give them a copy and the FCRA summary of rights and a reasonable opportunity to review and dispute it before any adverse action; if you still reject, send a final adverse-action notice. State and local 'fair chance' laws may delay criminal-history questions or checks until after a conditional offer.

Sources: FTC — Background checks: what employers need to know · FTC — Using consumer reports: what employers need to know · checked 2026-09-25

Stage 11 of 13Required

0 of 2 done

Agree the offer

Offer, negotiation and close

Who runs it: Hiring manager + recruiter
Done when: An accepted written offer within the approved band, with a start date.

Key tool

Build the offer around

•The problem: they'll own the models real customers or decisions depend on.
•Budget for experimentation: compute and time they control for their own projects.
•Learning: budget to attend an ML conference and time to write up what they build.
•Clear levelling: how you arrived at this level and what it takes to reach the next one.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the offer message built around what the candidate said matters to them. The figure, the approval and the conversation stay with the hiring manager.

Show the prompt and the checks

Describe: You are helping a hiring manager draft an offer message for a Machine Learning Engineer role. Context: I will paste three general priority categories (for example: learning, flexibility, scope) — not the candidate's words, notes or personal details; the parts of the offer; and the start-date options — without the salary figure. Produce: a warm, plain-English message that links each part of the offer to one of their priorities, says what happens next and when the written offer follows, and invites questions. Use British English and inclusive wording, and leave [brackets] for figures and names. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that nothing promised is outside policy or the approved band, that every claim about the role is true, and that the message reads as personal rather than templated.
Diligence: The hiring manager places the offer in the band using the same pre-set factors for everyone and makes the verbal offer personally. Figures and personal details are filled in by people, never pasted into AI.

Framework

Close plan (motivators → offer → objections)

Build the offer around what the candidate told you they care about, anticipate their objections, and make sure the verbal and written offers match.

List the candidate's top three motivators from your notes at every stage.
Make the verbal offer personally, and link each part of it to a motivator.
Ask what else they are weighing and when they have to decide.
Place the offer in the band using the same pre-set, job-related factors for everyone, and record why.
Send the written offer promptly after the verbal one.

Why it works: A close plan ties the offer to the candidate's stated priorities and makes approvals explicit before the final interview.

Offer checklist and negotiation

Stay inside the approved band. If the candidate is worth more, re-level them with evidence rather than stretching the band.
Trade in things that cost less than salary where policy allows: start date, learning budget, working pattern.
Before the written offer, ask: 'What would make this an easy yes?'
Agree approval and the pay band before the final interview, so the offer isn't held up.
The hiring manager makes the verbal offer; the written offer follows within a working day.
Record which pre-set factors placed the offer where it is in the band.
Tell unsuccessful finalists personally, without avoidable delay, with brief feedback.

What will close this candidate

The problem: they'll own the models real customers or decisions depend on.
Budget for experimentation: compute and time they control for their own projects.
Learning: budget to attend an ML conference and time to write up what they build.
Clear levelling: how you arrived at this level and what it takes to reach the next one.

Compliance checkpointEUUS

Use objective, gender-neutral, pre-agreed factors to place candidates within the band, apply them consistently and record the decision as an internal pay-equity control. Check the national law implementing Directive (EU) 2023/970 and any local record-keeping rules; this recommendation is good practice, not a universal statutory documentation rule.

Sources: EU Pay Transparency Directive 2023/970 · checked 2026-09-25

Stage 12 of 13Required

0 of 2 done

Help them settle in

Pre-boarding and 30-60-90 onboarding

Who runs it: Hiring manager + buddy + HR
Done when: A new hire who has worked through a written 30-60-90 plan tied to the 12-month outcomes, with a check-in at each milestone.

Key tool

Before day one
Laptop, accounts, repository and compute access ready before day one.
Days 1–30: learn
Sits down individually with whoever's downstream of each model — product, data, platform — to hear how they actually use the predictions.
Days 31–60: contribute
Puts a proper evaluation in place for whichever model was flagged as highest-risk at intake, building one from scratch if none exists.
Days 61–90: own
Owns one model end to end, with evaluation and monitoring running on a schedule.

AI as an advantage4D framework

Delegate: Hand AI the first draft of the 30-60-90 plan from the 12-month outcomes. Agreeing the plan with the new hire stays with the manager.

Show the prompt and the checks

Describe: You are helping a hiring manager draft a 30-60-90 day onboarding plan for a new Machine Learning Engineer. Context: I will paste the 12-month outcomes from the intake meeting, the systems and people the hire will work with, and anything they will need before day one. Produce: a pre-boarding checklist (equipment, access, a named buddy, a first-week schedule); then three sections — days 1–30 learn, days 31–60 contribute, days 61–90 own — each with 3–4 concrete, checkable expectations and one milestone check-in question. Use British English and plain wording. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that each expectation is realistic for a new starter, ties back to a 12-month outcome, and offers the hire support rather than a test.
Diligence: The manager agrees the plan with the new hire in week one and owns the check-ins. The new hire's personal details stay out of the prompt.

Don't miss: Handing a new machine learning engineer a brand-new model to build before they understand how the existing ones are evaluated and retrained.

Framework

30-60-90 day plan

Three phases — learn, contribute, own — with written expectations for each, agreed with the new hire in week one.

Before day one: equipment, access, a named buddy and a first-week schedule.
Days 1–30, learn: systems, people and the current state of the work.
Days 31–60, contribute: a first scoped piece of work shipped with support.
Days 61–90, own: an area of responsibility with clear outcomes.
Hold a check-in at each milestone against the written plan.

Why it works: Hiring doesn't end at the signature. A written onboarding plan makes expectations and support visible to the new hire and their manager.

Before day one
- Laptop, accounts, repository and compute access ready before day one.
- A sensible spending limit on any cloud training resources.
- A named buddy from the engineering or data team.
- Point them to the architecture docs, a current inventory of what's live, and a short history of what's gone wrong before — reading for week one, not meetings.
Days 1–30: learn
- Sits down individually with whoever's downstream of each model — product, data, platform — to hear how they actually use the predictions.
- Maps every live model and how each is currently evaluated and monitored, if at all.
- Ships one small fix to a production pipeline, to learn the deployment path.
Days 31–60: contribute
- Puts a proper evaluation in place for whichever model was flagged as highest-risk at intake, building one from scratch if none exists.
- Surfaces one accuracy or drift issue nobody had a number for before, and gets it onto someone's radar.
- Adds cost and latency tracking for at least one model.
Days 61–90: own
- Owns one model end to end, with evaluation and monitoring running on a schedule.
- Ships one improvement they can back up with before-and-after numbers.
- Walks the product team through what changed, in language a non-ML colleague can follow.

Compliance checkpointUSUK

US: the employee completes Form I-9 Section 1 no later than their first day of paid work, and the employer completes Section 2 within three business days after that day (on the first day if the job lasts less than three business days). UK: complete the prescribed right-to-work check before employment starts, and apply it the same way to everyone.

Sources: USCIS — Form I-9 instructions · GOV.UK — Check a job applicant's right to work · checked 2026-09-25

Stage 13 of 13Required

0 of 3 done

Measure time to hire

Measure the process: mean time to hire

Who runs it: Recruiter / talent lead + hiring manager
Done when: A stage-by-stage timing record for this hire, the slowest step named, and one change agreed before the next requisition opens.

Key tool

Record when the hire accepts

•Time to hire: days from the successful candidate entering the pipeline (application or first contact) to the accepted offer. Average it across hires for mean time to hire.
•Time to fill, recorded separately: days from requisition approval to the accepted offer.
•Days per stage, from the stage's first action to its last — and whether the time was spent waiting or working.
•Offer-acceptance rate: offers accepted divided by offers made.
•Where candidates dropped out, and the reason you were given.
•Days lost to compute or data-access setup for the work sample, kept separate from the time candidates spent actually diagnosing the evaluation.

Check in at 90 days whether the hire is meeting the outcomes in the job description — that tells you whether the process picked well.

Read: mean time to hire Read: time to hire vs time to fill

AI as an advantageAI boost4D framework

Delegate: Hand AI the job of building a blank timing worksheet and the instructions for reading it. Entering dates, working out the timings and deciding what to change stay with the recruiter and the hiring manager.

Show the prompt and the checks

Describe: You are helping a recruiter and a hiring manager set up a timing review for a Machine Learning Engineer hire. Context: I will list the stage names we used, in order (for example requisition approved, sourcing, screening, recruiter call, work sample, interviews, debrief, references, offer accepted). Produce: (1) a blank worksheet with one row per stage and columns for start date, end date, days in stage, days spent waiting (scheduling, approvals, no reply) and number of candidates who entered and left; (2) step-by-step instructions for us to work out, from our own records, time to hire (the successful candidate's entry to the pipeline, by application or first contact, to the accepted offer; average it across hires for mean time to hire) and, separately, time to fill (requisition approval to the accepted offer); (3) five questions we should ask about the slowest stage, separating waiting from working; (4) a short template for recording one change to make before the next hire. Use British English. Do not include benchmarks or example figures. Ask me questions if anything is missing. Rule: do not ask for, and I will not paste, candidate names, interview notes, CVs, submissions or any other personal data. If any appears, stop and tell me to remove it.
Discern: Check that the worksheet covers every stage you actually ran, that the two definitions are not mixed up, and that the questions point at the process rather than at any person.
Diligence: People fill in the dates and do the arithmetic from their own records, choose the one thing to fix and write it down. No dates, names or details about any candidate go into the prompt.

Don't miss: Counting only the time candidates spent on the task itself, and missing the hours lost waiting for data or compute access to actually work.

Framework

Mean time to hire, by stage

Time to hire is the number of days from the successful candidate entering your pipeline (their application or your first contact) to their accepted offer; mean time to hire is that figure averaged across hires. Time to fill — requisition approval to acceptance — is a separate measure. Record both per stage, so you can see where the time went rather than only the total.

When the hire accepts, write down the date each stage started and ended, and the day the successful candidate entered the pipeline.
Work out the days per stage, the time to hire and the time to fill, and mark the slowest stage.
Note the offer-acceptance rate and the stage where most candidates dropped out.
Separate waiting time (scheduling, approvals, silence) from working time, and decide which one to shorten.
Fix one thing before the next hire, and write down what you changed.

Why it works: A single total hides where the delay sits. Per-stage timing points at the one step to fix, and the acceptance rate and drop-out points show whether candidates kept pace with the process.

Record for every hire

Time to hire: days from the successful candidate entering the pipeline (application or first contact) to the accepted offer. Average it across hires for mean time to hire.
Time to fill, recorded separately: days from requisition approval to the accepted offer.
Days per stage, from the stage's first action to its last — and whether the time was spent waiting or working.
Offer-acceptance rate: offers accepted divided by offers made.
Where candidates dropped out, and the reason you were given.
Check in at 90 days whether the hire is meeting the outcomes in the job description — that tells you whether the process picked well.

What to log for this role

Days lost to compute or data-access setup for the work sample, kept separate from the time candidates spent actually diagnosing the evaluation.
How many days it took to find a slot where both the staff engineer and a data scientist were free for the production-systems round.
How many candidates the leaky-pipeline work sample removed, and whether most left before or after the setup actually worked.

Where this role's hiring slows down

Watch for compute or data-access provisioning taking longer than the work sample itself.
Watch for the production-systems round slipping every time because it needs a data scientist's calendar as well as an engineer's.
Watch for the known-wrong baseline in the starter repository going stale between searches so it no longer teaches what it's meant to.

Compliance checkpointUKEU

Under the EU and UK GDPR, tell candidates how long you will keep their data, or the criteria used to decide. Give that notice when you collect data from them directly, or within the Article 14 time limit when it comes from another source (such as a referral or a sourcing tool). Delete or anonymise data when it is no longer needed, keeping only what a documented lawful purpose still requires.

Sources: GDPR — Regulation (EU) 2016/679, Articles 5, 13 and 14 · ICO — UK GDPR guidance · checked 2026-09-25

General information, not legal advice. Rules depend on the employer, the role and where the work is done — confirm what applies with HR or counsel.

Frequently asked questions

How do you interview a machine learning engineer?

Run a work sample first — a small pipeline with a real evaluation problem to diagnose — then three structured rounds covering data and evaluation rigour, production engineering, and how they explain a model to a non-technical stakeholder. Score every candidate against the same rubric, and in each round probe answers with the same follow-ups — why that choice, what else they considered, and how they would know it worked.

Machine learning engineer vs data scientist — how are the roles different?

Titles overlap. A data scientist role centres on analysis, experimentation and modelling; a machine learning engineer role centres on reliable training, deployment and monitoring once a model is a real system in production. Define the role from the work your team actually needs, not from the title on a CV.

Do you need a PhD to hire a good machine learning engineer?

A PhD may be relevant where the role requires research depth, but it should not be a default filter for production engineering work. Select for the software, data and evaluation capabilities the role actually requires.

What is the best work sample test for a machine learning engineer?

Give the candidate a small pipeline with a model that looks better than it should be, and ask them to find out why before changing anything. It tests the exact skill that separates a reliable hire from a risky one: whether they check a number before trusting it. Keep it to two or three hours, mark against written criteria, and let candidates use whatever AI tools they'd use on the job.

How long does it take to hire a machine learning engineer?

Plan for four to six weeks. Before sourcing, reserve panel time and agree the offer-approval process so those steps do not create avoidable delay.

Should a machine learning engineer's work sample allow AI tools?

Where company policy permits AI tools on the job, allow them in the assessment under the same conditions. Assess whether the candidate verifies the output.

Go deeper

Related hiring kits

Guides

Skills tests

See your first AI-generated assessment in 5 minutes

No credit card. No question library to learn. Paste a job description and watch H-Evaluate build.

Request early access