LLM Engineering Platform

Overview

From raw web data to a fine-tuned LLM serving live traffic — one pipeline, zero duct tape.

LLM Engineering Platform is a full-stack, production-style system for building and shipping large language model applications. It follows Domain-Driven Design and covers the entire LLM lifecycle: crawling and cleaning data, building a RAG retrieval layer, generating instruct/preference datasets, fine-tuning on AWS SageMaker, and serving inference behind a REST API.

Problem

A model that answers well in a notebook and a model that survives production are two different problems. Real deployments need versioned data, reproducible pipelines, experiment tracking, a vector store for retrieval, and a deployment target that can actually take traffic — getting those dozen moving pieces to work together correctly is the actual engineering challenge.

What I Built

• A Domain-Driven package (llm_engineering/) cleanly layered into domain → application → model → infrastructure, keeping business logic decoupled from AWS, Qdrant, and MongoDB integrations
• ZenML-orchestrated pipelines for the full ML lifecycle: digital data ETL, feature engineering, instruct/preference dataset generation, training, and evaluation
• A RAG retrieval module backed by Qdrant vector search, testable standalone via tools/rag.py
• SFT and DPO fine-tuning pipelines deployed to AWS SageMaker, with results tracked in Comet ML and Hugging Face
• A FastAPI inference microservice (ml_service.py) with prompt monitoring wired through Opik
• Local dev environment fully containerized with Docker Compose (MongoDB, Qdrant, ZenML server) so the entire stack runs without touching the cloud
• CI/CD via GitHub Actions and a full linting/testing suite (Ruff, pre-commit, pytest)

Key Outcomes

• Assembled a genuinely production-shaped LLM stack — data → RAG → fine-tuning → deployment — rather than a single-notebook demo
• Hands-on experience with the tools real ML platform teams use daily: ZenML for orchestration, Qdrant for vector search, SageMaker for training/serving, Comet/Opik for observability
• A reusable reference implementation for anyone standing up an LLM system from scratch, following DDD principles for long-term maintainability