Available for new projects
Nikita Kalachev QA & LLM Evaluation Engineer
I find where software quietly fails — now in LLM outputs and the products built on them.
I find where software quietly fails — now in LLM outputs and the products built on them.
I spent seven years finding bugs in software companies could not afford to break: checkout systems, payment flows, medical records. Four and a half of those years were at EPAM Systems, on platforms used by more than a million people.
For the past year I have been doing the same work on AI. I built CrossCheck, a tool that sends one question to several AI models and shows where they disagree. Most tools give you a single answer. This one shows you which parts of it to distrust.
I now take contract work in LLM evaluation and QA for AI products.
I check model outputs for accuracy, invented facts, whether the model followed the instruction, and whether the reasoning holds. I write the criteria down so a second person grading the same output gets a similar result.
Teams usually know one answer is worse but cannot say why. I turn that into a written scale with weights, so the score can be checked and argued with.
I build test sets meant to break the model: prompt injection, edge cases, and the failures that only show up under pressure.
Release testing for teams shipping AI features. Regression on what the model says, not just on the interface around it.
Most engagements start with a small paid trial task — the low-risk way to see the work before committing. Start a trial task →
CrossCheck sends the same question to several AI models, has them argue with each other, and reports where they disagree. Most tools hand you one confident answer. This one shows you the parts the models could not agree on, so you know what to check.
It runs in three stages. Each model answers on its own. Then they critique each other, two rounds in the standard setting. Then everything is combined into one report that keeps the disagreements visible.
Every disagreement is weighted by how serious it is: 20 points off for a critical conflict, 10 for a moderate one, 5 for a minor one. It is plain arithmetic, and you can read the function and check the math. No model grades another model.
There is no automated benchmark for the verification itself, and I do not pretend otherwise. I tested the behaviour by running cases by hand. The scoring math is covered by unit tests. One part of the system, the PII scrubber, has a real measured benchmark against a synthetic test set.
Four providers available to users: Anthropic, OpenAI, Google, Mistral. Up to eleven models depending on which keys you add. A fifth adapter, AWS Bedrock, is wired up but internal. Roughly 1,300 automated tests, and about as much test code as production code. Next.js, TypeScript, Supabase. Public beta since May 2026, no revenue yet.
I write about verification and where AI reliability claims break down.
Built a multi-model LLM verification system on my own, from the first problem sketch to public beta.
Four and a half years on enterprise projects for large retail, healthcare and fintech clients.
Led a team of four on a dental clinic CRM: booking, billing, inventory.
Full-cycle QA on 8+ projects in retail, telecom, finance and healthcare, all shipped with no critical defects after launch. Cross-browser and security testing.
LLM output evaluation, evaluation methodology and rubric design, adversarial test sets and red-teaming, and QA for teams shipping AI features. I am strongest where the judgment is about correctness, reasoning, instruction-following, and safety of outputs. I do not take work that depends on line-by-line code judgment in languages I do not work in, or on native-level English style judgment.
I directed AI coding agents instead of writing the code by hand. I designed the method, the scoring model and the architecture, set the rules the agents had to follow, and checked what came back. That is why the project has around 1,300 tests: generated code needs more checking, not less. It is also how I ended up in AI evaluation — reviewing AI output is what I do every day.
Yes. I take contract work through my company, Platilus LLC, with a contract and invoices. CrossCheck is a public beta with no revenue, so it is not a company I am hiring for. A small paid trial task is usually the easiest way to start. I work in English (B2) and Russian (native). Write to [email protected].
A small paid trial task is usually the easiest way to start — contract and invoices through Platilus LLC.