27 Interview Questions for ML/AI Engineers (With What Good Answers Sound Like)

The best interview questions for ML/AI engineers test what the candidate did with models in production, and only then how well they understand the models themselves. Most question banks online do the opposite: they list definitions (what is overfitting, explain gradient descent) that any candidate can rehearse and that say nothing about whether the person can ship. This list, drawn from the screening Vintti AI runs before placing dedicated AI/ML engineers with US companies, is organized around the six things that actually fail in real hires, with guidance on what a good answer sounds like at each seniority level.

‍

How to Use This List

Pick 8-10 questions per interview, never all 27, and match them to the seniority of the role.

A useful rule from Vintti AI's process: every interview needs at least one question from the "Production" block and one from the "Ownership" block, whatever the level. Those two blocks predict retention.

‍

Block 1: Fundamentals (Questions 1-5)

Fundamentals questions should test whether the candidate can reason, and rehearsed definitions do not count.

1. Explain the bias-variance trade-off using a model you actually built. Good: a specific model, what overfit, what they changed, how they measured it. Weak: the textbook definition with no project attached.

2. Your model gets 96% accuracy and the product team is unhappy. What might be going on? Good: class imbalance, wrong metric for the business problem, leakage, offline-online gap. Strong candidates ask what the 4% costs.

3. When would you choose a simple logistic regression over a deep model? Good: small data, interpretability requirements, latency budgets, speed of iteration. Seniors have a story where the simple model won.

4. How do you choose an evaluation metric for a new problem? Good: start from the business decision the model informs, then pick the metric that tracks it, then define the offline threshold for shipping.

5. What is data leakage, and describe a time you found it. Good: a concrete case (a future-derived feature, a duplicated record across splits) and how it was caught. Everyone knows the definition; few have caught it.

‍

Block 2: Production and MLOps (Questions 6-11)

Production questions carry the most weight because production is where most ML hires fail.

6. Walk me through the path from a model on your laptop to a model serving customers. Good: packaging, serving layer, versioning, rollout strategy, monitoring, rollback plan. Weak: "we deployed it" with no steps.

7. Tell me about a production incident involving a model. What broke, how did you find out, what did you change? Good: specific symptom, specific root cause, a permanent fix (alerting, tests, guardrails). Candidates who have never had an incident have never owned production.

8. How do you detect model drift, and what do you do when you see it? Good: input distribution monitoring, prediction distribution monitoring, delayed ground-truth checks, retraining triggers with human review.

9. How would you reduce inference cost by half without hurting quality? Good: caching, batching, smaller or distilled models, quantization, cheaper routing for easy cases, measuring quality before and after.

10. What does your ideal CI/CD pipeline for a model look like? Good: tests on data schema and model behavior, evaluation gates, reproducible training, artifact registry, staged rollout.

11. How do you decide a model is good enough to ship? Good: a pre-agreed threshold tied to the business metric, an offline evaluation set that mirrors production, and a plan for what happens in the first week live.

‍

Block 3: LLM and Applied AI (Questions 12-16)

LLM questions separate engineers who have shipped AI features from engineers who have used ChatGPT.

12. Explain retrieval-augmented generation to a product manager, then tell me where it breaks. Good: a clear plain-language explanation, then chunking issues, retrieval misses, stale indexes, context limits, evaluation difficulty.

13. How do you evaluate an LLM feature when there is no single correct answer? Good: golden sets with rubrics, LLM-as-judge with calibration against humans, pairwise comparisons, regression suites run on every prompt change. This is the core skill of an evals engineer, and any applied AI engineer should know the basics.

14. Fine-tune, prompt, or retrieve: how do you pick? Good: prompt first, retrieval when the knowledge is external or changing, fine-tuning when style or format matters at scale or when latency and cost demand a smaller model. Bonus for mentioning that the decision should be reversible.

15. How do you handle hallucinations in a customer-facing feature? Good: grounding with citations, constrained outputs, confidence signals, human fallback for high-stakes cases, and measuring the rate rather than hoping.

16. What is in your prompt versioning and testing setup? Good: prompts in version control, an evaluation run before every change, tracking of model version alongside prompt version. Candidates who edit prompts in production by hand have never been burned yet.

For roles centered on this block, the better-fitting title may be LLM integration developer or prompt engineer; the explainer What Does an AI/ML Engineer Do? covers the boundaries.

‍

Block 4: Data (Questions 17-20)

Data questions reveal whether the candidate respects the part of the job that consumes most of the time.

17. Describe the worst dataset you ever worked with and what you did about it. Good: specific problems (labels, duplicates, missing values, drift over time), specific fixes, and an opinion on what should have been done upstream.

18. How do you know your training data represents production? Good: sampling strategy, comparing distributions, monitoring the gap after launch, and knowing when to refresh.

19. How would you set up labeling for a new classification problem? Good: written guidelines, a pilot batch, inter-annotator agreement, a gold set, iteration on the guidelines. Engineers who have worked with a dedicated data annotation specialist usually answer this well.

20. What do you do when the data you need does not exist? Good: proxies, weak supervision, synthetic data with its risks named, or arguing that the project should wait. The last answer is the senior one.

‍

Block 5: Ownership and Collaboration (Questions 21-24)

Ownership questions predict whether the engineer will still be on your team, and still shipping, in eighteen months.

21. Tell me about a time you argued against using machine learning for a problem. Good: an instant, specific answer. Senior applied engineers spend real time talking teams out of unnecessary ML.

22. Describe a project that failed and your share of the reason. Good: a real failure with personal ownership of at least part of it. Universal blame on data, product or infrastructure is a pattern to note.

23. How do you keep non-technical stakeholders informed when a project is going badly? Good: early, specific, with options. Weak: "I would explain the technical challenges", which usually means late.

24. What did the last system you built look like six months after you stopped touching it? Good: they know, because they kept an eye on it or handed it over properly. Not knowing is an honest and informative answer.

‍

Block 6: Live Exercise (Questions 25-27)

One live exercise close to your real work beats ten conceptual questions. Pick one, 60-90 minutes, screen shared, thinking out loud:

25. Here is a small API and a model artifact. Wire the model into the endpoint, add one test, and tell me what is missing before this could go to production. Tests engineering hygiene and production instinct.

26. Here is an LLM feature and twenty examples of its output, some bad. Design an evaluation that would catch the bad ones automatically. Tests applied LLM judgment and the ability to turn vague quality into a measurable thing.

27. This inference pipeline is slow and occasionally wrong. Find out why. A deliberately broken pipeline tests debugging process, not just the fix: what they check first, what they measure, what they ask.

Vintti AI runs a version of this block in every screening; watching the working process for one hour tells you more than any resume.

‍

Scoring Rubric

Score each answer on evidence, not eloquence.

Hiring bar Vintti AI suggests to clients: an average of 2.3 or higher across the chosen questions, no zeros in the Production or Ownership blocks, and a live exercise the technical interviewer would trust in production with light supervision.

‍

What a Wrong Hire Costs

The cost of a mis-hire depends on where you hire, and that changes how much evaluation you can afford. Yearly compensation from the Vintti AI Roles Benchmark:

At nearshore rates, a paid two-week trial project becomes an affordable final interview step, and it is the most reliable one. The full role-by-role dataset is at vintti.com/ai/salary-benchmark.

(Note: Vintti AI is a nearshore staffing and recruiting firm. We are not an Employer of Record (EOR), a managed service provider, or a BPO. We source, screen and place dedicated AI/ML engineers, as long-term embedded roles or project-based placements, and your company employs or contracts them directly. While Latin America is our core talent pool, we also recruit US-based and global candidates when the role calls for it.)

‍

Frequently Asked Questions

How many interview questions should an ML engineer interview have?

Eight to ten, across at least three of the six blocks above, plus one live exercise. Longer lists trade depth for coverage, and depth is where the signal lives.

‍

Should ML engineer interviews include coding algorithm puzzles?

Rarely. Algorithm puzzles predict algorithm puzzle skill. A 60-90 minute exercise close to your real work (Block 6) tests what the job requires.

‍

What is the most revealing question on this list?

Question 21: "Tell me about a time you argued against using machine learning." It tests judgment, business literacy and ownership at once, and senior engineers answer it instantly.

‍

How do I interview for LLM skills if we have never shipped an LLM feature?

Use Block 3 and let the candidate teach you. A strong applied engineer explains RAG, evaluation and the fine-tune-versus-prompt decision clearly to a non-expert; that clarity is itself the skill you are hiring.

‍

How does Vintti AI screen before the client interview?

Production stories told in depth, a live exercise from Block 6, an English evaluation for engineering conversations and reference checks. Clients receive a shortlist in about 7 days, run first interviews within 1-2 weeks, and complete the hire in 2-4 weeks.

‍

Do these questions work for project-based hires?

Yes. Vintti AI places both long-term embedded engineers and project-based placements; for project work, weight Block 2 and Block 6 more heavily, since the engineer has less time to ramp.

‍

Good interviews for ML/AI engineers are built on evidence: what shipped, what broke, what the candidate chose not to build, and how they explain it. Choose eight to ten questions from the blocks above, run one live exercise, score on specifics, and decide within two weeks of the first interview.

‍

Hire a pre-screened AI/ML engineer - start with a conversation