Requirements
Define the task, use case, risk, labels or scoring dimensions, and acceptance criteria.
Quality
The score is only useful when you can inspect how it was made.
Marka’s quality process starts before the first item is reviewed. The instructions, reviewer requirements, checks, and handling of uncertainty are part of the project scope.
Scope a pilotInstruction design
Write examples, edge cases, evidence rules, and a clear path for uncertainty.
Reviewer calibration
Test reviewers on representative cases before production work begins.
Production review
Use project-specific checks, hidden quality items, and confidence signals.
Disagreement handling
Route uncertain or conflicting decisions to a more qualified review step.
Delivery approval
Check output format, quality findings, corrections, and documented limitations.
Qualification
Projects can require language proficiency, domain experience, prior role experience, or formal credentials. The actual requirement is agreed with the customer.
Calibration
Reviewers work through representative examples and receive feedback before production access is granted.
Ongoing measurement
Agreement, correction rate, hidden quality cases, confidence, and pattern-specific performance can be monitored during delivery.
Limited access
Reviewers should see only the data needed for their assigned task and only for the required period.
Evidence
What one reviewed item can record
The final schema is project-specific. These fields show the type of provenance Marka is designed to preserve.
| Input and output | The original item, model response, trace, or source record. |
| Instruction version | The exact rubric or annotation guidance used for the decision. |
| Reviewer decision | Label, score, correction, confidence, and written explanation. |
| Supporting evidence | Policy passage, tool result, source text, or expected outcome. |
| Quality history | Second review, correction, disagreement, and final approval status. |
| Dataset version | The delivery batch and change history in which the item belongs. |
Quality starts with a task we can define clearly.
Send a sample and the standard you currently use. Marka will identify what needs to be clarified before a pilot begins.
Start a pilot