The definitive, practical 16-level curriculum for engineers. Every level builds sequentially on the ones before it, balancing real software fundamentals with cutting-edge agentic workflows, evals, and capstone portfolio builds.
YOUR ROADMAP PROGRESS0 / 16 (0%)
ACTIVE IN TRAINING
THE WHOLE PATH REGISTER // CLICK TO SCROLL OR INSPECT16 LEVELS · 286 SESSIONS
RULE 01Start Where You Are
Each level lists prerequisite assumptions. If you already deploy Python APIs, skip level 0. Don't waste time re-learning what you already ship.
RULE 02One Level at a Time
The path is strictly cumulative. RAG (L4) relies on structured outputs (L2). Agents (L5) rely on RAG. Do not jump to agents before mastering APIs.
RULE 03Finish by Building
A level is only completed when you build its capstone deliverable with verifiable code. Watching videos without shipping yields zero retention.
PHASE 1 // FoundationsLevels 0–3
67 SESSIONS TOTAL
0
LEVEL 022 sessions · 8 modulesACCELERATED CORE
Software Engineering Fundamentals
Become comfortable building, debugging, and deploying real software before adding LLMs.
INTERVIEW WEIGHT9 / 10
Many AI engineer loops screen candidates as software engineers first with coding rounds, backend architecture, and API design. If you fail here, you never reach the AI rounds.
ON-THE-JOB WEIGHT10 / 10
Most of the job is regular software engineering: APIs, databases, testing, telemetry, debugging, and deployments. Most production incidents are software bugs, not model problems.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Resilient Async Backend API & Telemetry Pipeline
Build a production-ready asynchronous Python API with Postgres, connection pooling, background worker queues, structured logging, and Docker containerization.
Build a strong mental model of how LLMs work, why they behave the way they do, and how to choose the right model.
INTERVIEW WEIGHT8 / 10
Interviewers probe whether you understand underlying mechanics (e.g., token-to-word ratios, KV cache growth, temperature vs top-p) rather than just calling an API endpoint.
ON-THE-JOB WEIGHT8 / 10
A solid mental model prevents rookie mistakes: using high temperature for JSON output, sending unbounded prompts, or picking a massive model when a fast 8B model performs identically.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Interactive Token & Latency Inspector
Build a diagnostic tool that accepts text inputs, visualizes Byte-Pair Encoding (BPE) token boundaries, calculates token-to-word expansion, and benchmarks TTFT (Time To First Token) across multiple providers.
Stack:Python / TypeScripttiktokenFastAPI / Next.jsLMSYS API / Groq
Turn individual model calls into resilient software workflows with streaming, structured outputs, and tools.
INTERVIEW WEIGHT9 / 10
Standard coding round for AI engineers: given an unstructured request, use function calling and schema validation to extract normalized data with robust fallback handling.
ON-THE-JOB WEIGHT10 / 10
This is the core plumbing you will touch every single day. If your API integration lacks schema enforcement, streaming, and retries, your application will feel sluggish and fragile.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Resilient Multi-Provider LLM Gateway
Build an API gateway that routes requests across multiple LLM providers (e.g. Gemini, Groq, Anthropic), with automatic fallback on 429/500 errors, strict Pydantic JSON validation, SSE streaming, and token usage accounting.
Key Tools:Vercel AI SDKInstructorPydanticZodLiteLLMLangfuse
3
LEVEL 311 sessions · 9 modules
Prompt Engineering & Evaluation Fixtures
Design deterministic instructions that make LLM behavior reliable and testable across versions.
INTERVIEW WEIGHT6 / 10
Interviewers test whether you rely on brittle adjectives or know how to structure prompts with delimiters, negative constraints, and few-shot calibration.
ON-THE-JOB WEIGHT7 / 10
A well-structured prompt solves 70% of problems without needing complex agent loops or expensive fine-tuning.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Automated Prompt Regression Testing Suite
Build a continuous testing suite that runs versioned prompt templates against 30 edge-case test fixtures, asserting schema validity, tone compliance, and constraint satisfaction.
Ground models in external knowledge with high-precision ingestion, hybrid search, and re-ranking.
INTERVIEW WEIGHT10 / 10
Guaranteed interview topic. Expect questions on chunking strategies, vector index trade-offs (HNSW vs IVF), hybrid search fusion, re-ranking, and mitigating 'lost in the middle'.
ON-THE-JOB WEIGHT10 / 10
Nearly every enterprise AI feature is a form of RAG. Mastering retrieval quality separates real AI engineers from API wrappers.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Production Hybrid RAG Engine with Re-Ranking
Build a complete document retrieval and answering pipeline that ingests complex PDFs/markdown, generates dense embeddings, performs hybrid search (pgvector + BM25), re-ranks top candidates with Cohere/BGE, and outputs answers with verifiable line citations.
Build systems where models decide what to do next, invoke tools, maintain state, and recover from failures.
INTERVIEW WEIGHT9 / 10
Senior candidate rounds focus on agent architecture: how do you prevent infinite loops, manage state across turns, handle tool execution failures, and implement human approvals?
ON-THE-JOB WEIGHT9 / 10
Agents unlock the most lucrative AI products (e.g., coding assistants, market research engines, customer support bots). Building them reliably requires strict graph discipline.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Autonomous Market Research Agent with MCP & LangGraph
Build a multi-step research agent that takes a company or product idea, plans search tasks, calls Model Context Protocol (MCP) tools for live web queries, synthesizes competitor data, pauses for human approval before final drafting, and compiles a clean markdown intelligence dossier.
Manage limited context windows deliberately — decide what enters, what stays, and what gets moved outside.
INTERVIEW WEIGHT8 / 10
Tests understanding of practical production constraints: how do you run an agent for 50 steps without overflowing a 128k context window or paying $10 per query?
ON-THE-JOB WEIGHT10 / 10
Context engineering is the difference between an agent that costs a fortune and degrades into confusion, versus an agent that runs smoothly for hours.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Context-Engineered Long-Horizon Agent Harness
Build an agent harness that executes multi-step coding or research tasks with a hard 16k token context budget, using dynamic scratchpad files on disk, sliding history summarization, and prompt caching to reduce token spend by >80%.
Design, execute, and automate evaluations so you can ship changes without fear of regressions.
INTERVIEW WEIGHT9 / 10
Top-tier companies ask: 'How do you know this prompt change or model migration actually improved the product instead of breaking edge cases?' If you don't know evals, you're not hired.
ON-THE-JOB WEIGHT10 / 10
Evals turn AI engineering from guessing and vibes into real engineering. Every prompt, model, or RAG tweak must be verified against an eval suite.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Automated CI/CD Eval Pipeline with Ragas & Synthetic Benchmarks
Build a full evaluation suite for a RAG system: generate 50 synthetic test question-answer pairs, run evaluations across Faithfulness and Answer Relevance metrics using an LLM judge, and block PR merges in GitHub Actions if scores drop below 0.85.
Secure LLM, RAG, and agent systems against model-specific attacks, unsafe actions, and data leakage.
INTERVIEW WEIGHT8 / 10
Enterprise security questions: 'How do you prevent an agent with email tools from being hijacked by a malicious website it visits?'
ON-THE-JOB WEIGHT9 / 10
Critical for production compliance (SOC2, HIPAA) and protecting your company from unauthorized actions or severe API billing drains.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Automated AI Security & Red-Teaming Fuzzer
Build a security testing harness that attacks an AI application with 40+ prompt injection payloads, indirect RAG document poisonings, and tool SSRF attempts, outputting a vulnerability scorecard with remediation guidance.
Key Tools:NeMo GuardrailsLlama GuardGarakPresidio (PII)OWASP LLM Top 10
PHASE 5 // ProductionLevels 9–11
57 SESSIONS TOTAL
9
LEVEL 916 sessions · 10 modulesACCELERATED CORE
AI System Design & Production Architecture
Design a complete production AI application and defend every architectural choice across latency, cost, and reliability.
INTERVIEW WEIGHT10 / 10
The dedicated AI System Design round for staff/senior positions. You will be asked to design systems like ChatGPT, Copilot, or Perplexity from scratch on a whiteboard.
ON-THE-JOB WEIGHT10 / 10
Determines whether your company spends $100k/month or $5k/month on AI infrastructure, and whether your latency is 200ms or 15 seconds.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
End-to-End Enterprise AI Search Architecture
Design and prototype a high-throughput AI document search engine with semantic caching, asynchronous background indexing worker queues, multi-tenant database isolation, and detailed cost/latency telemetry.
Deploy your RAG and agents to production — versioned, monitored, guarded, and continuously improving.
INTERVIEW WEIGHT8 / 10
Interviewers look for real deployment battle scars: handling upstream provider outages, tracking model version deprecations, and debugging production regressions.
ON-THE-JOB WEIGHT10 / 10
Your code is useless if it cannot be deployed, monitored, and maintained reliably by your team without breaking at 3 AM.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Production Docker-Compose & Observability Stack
Build a production-grade infrastructure blueprint deploying an AI service with Prometheus metrics, Langfuse distributed tracing, automated healthcheck endpoints, and alert webhooks for 5xx spikes and cost runaway.
Ship applications on the enterprise platforms teams actually buy and deploy.
INTERVIEW WEIGHT7 / 10
Enterprise roles specifically test experience with AWS Bedrock / Vertex AI IAM roles, VPC private endpoints, and compliance boundaries.
ON-THE-JOB WEIGHT8 / 10
Knowing when to pick managed cloud vs low-latency inference routers cuts infrastructure integration times from months to days.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Intelligent Dynamic Multi-Cloud Model Router
Build a multi-cloud router that dynamically evaluates query complexity: sends simple, latency-critical tasks to Groq LPU (400 tok/sec), complex reasoning to Gemini 1.5 Pro / Claude 3.5, and sensitive data through an enterprise AWS Bedrock private endpoint.
Stack:TypeScript / PythonGroq SDKAWS Bedrock SDKGoogle Generative AI SDKFastAPI
Build systems that see, hear, and converse in realtime beyond text-only interfaces.
INTERVIEW WEIGHT8 / 10
Differentiating skill: explaining how vision tokens consume context windows, how to handle voice interruptions, and how to parse complex charts accurately.
ON-THE-JOB WEIGHT8 / 10
Multimodal features (voice assistants, document scanning, chart analysis) provide massive UX upgrades that standard text chatbots cannot touch.
Build a web application where users can drag in complex financial charts/receipts, converse with the assistant via low-latency streaming voice, and receive extracted structured line items with visual bounding box highlights.
Fine-tune models when prompting and RAG have run out, proving the investment with eval benchmarks.
INTERVIEW WEIGHT8 / 10
Interviewers test whether you understand the trade-offs: 'Why would you fine-tune instead of using few-shot RAG?' Know the real mathematical and architectural reasons.
ON-THE-JOB WEIGHT7 / 10
Enables deploying specialized, ultra-fast 3B or 8B models that outperform huge frontier models on specific structured tasks at 1/20th the cost.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Domain-Specialized 8B Model with LoRA & Eval Benchmarks
Curate a synthetic training dataset of 1,000 domain examples, fine-tune an open-weight Llama 3 8B model using QLoRA and Unsloth, quantize to GGUF/AWQ, and run evaluation scripts demonstrating higher JSON adherence and lower latency than base models.
Proof, not concepts. Ship 3 public production AI systems that compose every earlier level.
INTERVIEW WEIGHT10 / 10
The #1 factor in getting hired or funded: a link to a working product you built with live users, clean code, and demonstrated problem-solving.
ON-THE-JOB WEIGHT10 / 10
Shipping products end-to-end forces you to solve real frontend, backend, database, and model integration edge cases that tutorials gloss over.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Public Production AI Product Launch on DevCastle Launchpad
Build, deploy, and publicly ship a complete AI application solving a real customer pain point. Package it with a live public URL, interactive demo video, GitHub repository with clean docs, and launch it to the DevCastle builder community.
Convert everything you learned into interview performance and executive architectural defense.
INTERVIEW WEIGHT10 / 10
This entire level is focused on winning the offer: articulating trade-offs clearly, showing deep engineering rigor, and standing out from prompt-only applicants.
ON-THE-JOB WEIGHT8 / 10
Defending architectural decisions to engineering leaders, VPs, and founders is how senior engineers get projects approved and funded.
LEVEL CAPSTONE BUILDVERIFIABLE DELIVERABLE
Comprehensive AI Engineer System Defense Dossier
Assemble a 20-scenario interview dossier covering complete architectures, failure mitigations, latency calculations, and cost models for real-world interview prompts (e.g. 'Design Copilot for Codebases', 'Build a Multimodal Customer Support Agent').