Is this for you?
- Your team shipped LLM features in the last 12–18 months and is now dealing with the consequences.
- Costs are unpredictable and nobody wants to own the bill.
- There's no eval strategy — you find out things broke when users complain.
- Prompts are strings scattered through the codebase, and changing one feels dangerous.
- A provider change is on the table and you're not sure what it implies.
- Someone on the team wants to add RAG, agents, or fine-tuning and you're not sure the foundation is ready.
What you get
- Written review (15–20 pages) with findings grouped by severity.
- Architecture map: every LLM call, every model, every pattern.
- Cost analysis with concrete optimization recommendations and projected savings.
- Quality and eval strategy recommendation — what to measure, how, and where to put it in CI.
- Prompt management review — versioning, testing, and safer iteration patterns.
- Security review: prompt injection surface, PII handling, output validation, logging hygiene.
- Prioritized remediation roadmap with effort estimates.
- 60-minute walkthrough with your engineering team.
Scope
What's in
- Application-layer LLM integration
- Prompts, evals, and cost
- Security posture around LLM calls
- Provider choice and migration cost
What's out
- Model training or fine-tuning
- Infrastructure for self-hosted models (separate engagement)
- UX or product strategy
- Implementing the recommendations (available separately)
Process
Intake
Day 1Kickoff call, access to repo and dashboards, intro to the team.
Discovery
Days 2–5Codebase read, call-graph of LLM usage, cost and telemetry review, stakeholder conversations.
Analysis
Days 6–9Findings synthesis, remediation design, cost modeling.
Report
Days 10–12Draft, review, finalize.
Walkthrough
Day 1460-minute call with your team to walk the report and answer questions.
Pricing
50% to start, 50% on report delivery. Includes one 30-minute follow-up call within 30 days of delivery.
One fixed price. No surprises, no “starting at” language. If we agree on scope and you pay the deposit, the engagement is locked in.
Questions
We haven't shipped RAG yet. Is this still useful?
Yes — if you're using LLMs in production at all, there's something to review.
Can you implement the fixes?
Separately, yes. This engagement is advisory; implementation is a separate scope.
We use OpenAI / Anthropic / Google / multiple. Does that matter?
No — the review is provider-agnostic.
Will you sign an NDA?
Yes. Send yours or use mine.
What access do you need?
Read access to the repo, read access to your LLM provider dashboard(s), and 2–3 hours of engineering team time across the two weeks.
About
Cornell NLP certified, building with LLMs since the GPT-3 beta. I've shipped RAG pipelines, production prompt systems, and content-understanding features at scale — so the review is grounded in what actually breaks, not what the docs warn about.
More about the studioReady to start?
Book an intro call. If we're not a fit, I'll tell you on the call.