Method / From submission to report
Follow the evidence.
A score is the end of a process. Here is what happens to the work along the way, and where a teacher’s judgement belongs.
- 01Student work
- 02Evidence
- 03Skill-tagged feedback
- 04Rubric score
- 05Teacher review
- 06Report
Student work
The task gives the assessment its context.
A writing submission includes text or OCR output, the prompt and the selected scale. Speaking contributes captured audio and transcripts. Reading and listening contribute answers to a prepared test.
InputEvidence
Keep the material behind the judgement.
Writing corrections retain the original text and suggested replacement. Speaking records retain the raw transcript and can anchor a finding to an audio span. Receptive tasks retain the question, accepted answers and the pupil’s response.
Evidence recordSkill-tagged feedback
Give the finding a stable name.
Models propose writing and speaking feedback. The taxonomy service resolves labels to known domain, category and skill codes. Unknown labels have an explicit fallback; a valid code does not make the judgement infallible.
Model judgement + code rulesRubric score
Apply the chosen rubric, then calculate.
Writing and speaking use model judgements for relevant criteria. Standard writing totals, rounding and scale projections are calculated in code. Pronunciation can use acoustic measurements. Reading and listening use answer-key matching.
Judgement / measurement / arithmeticTeacher review
A person decides what to publish.
The teacher can inspect and correct feedback before publication. The speaking review design distinguishes report approval, transcript review and listening to the audio. Bulk approval must not be represented as audio verification.
Human reviewReport
Make the result usable.
The report brings the score, evidence and next steps together. Background workers prepare report artefacts separately from assessment calls. Publication checks report readiness before a result is released to the learner and family.
PublicationWho does what
Judgement is not
the same as counting.
| Mechanism | What it does | Where judgement remains |
|---|---|---|
| Model assessment | Proposes feedback, rubric judgements and task-compliance findings. | Interpreting language, relevance and the teacher’s instructions. |
| Deterministic rules | Resolve taxonomy labels, match accepted answers, assemble totals and apply scale boundaries. | Rules operate on the evidence and configuration they receive. |
| Acoustic measurement | Supplies pronunciation evidence where the measurement is available. | A transcript alone cannot establish how a word sounded. |
| Teacher review | Accepts, rejects or edits findings and controls publication. | Knowing the learner, checking the evidence and deciding what is fair. |
Task context
The teacher’s instructions
are part of the task.
Writing includes a separate check against mandatory teacher requirements. Configured failures can affect the score. If an instruction explicitly permits a format or language feature, that permission is passed into the assessment prompts.
The compliance verdict still involves model judgement. Applying the resulting penalty or score boundary is a separate calculation.
The Fix-It loop
Their own error.
Their next practice.
RALE can turn a stored writing correction into a “choose the fix” exercise. It compares the original and corrected sentence, finds a clean single-word replacement, and uses the pupil’s own wrong word as an answer option.
This step makes no new AI call. It is a deterministic transformation of feedback already in the report. Teachers can prescribe skill codes to draw practice from a learner’s past errors.
It skips changes that cannot form a clean word-choice gap, including punctuation-only and capitalisation-only edits. Extra answer options are added only when local rules establish that they are wrong.
modules/services/fixit_service.pydocs/architecture/03_RALE_BACKGROUND_ENGINES.mddocs/architecture/04_PROMPT_ENGINEERING_AND_TAXONOMY.mddocs/architecture/domain_writing_unified.mddocs/architecture/speaking_engine_spec.mddocs/architecture/domain_speaking_*.mddocs/architecture/domain_reading_*.mddocs/architecture/domain_listening_*.mdmodules/common/scoring_math.pymodules/student_processing/publishing.py