Research

Evaluating practice: learning evidence, research questions and limitations

Evaluate a practice programme against a defined learning question. Agree the task, baseline, feedback and review method before collecting results. Improvement on the same prompt can reflect familiarity; use a fresh task and appropriate follow-up to explore whether learning transfers.

See pricing

Define the question before the pilot

Specify the behaviour you want to observe and the decision the evaluation will inform. Record the learner group, task, conditions, review criteria, and relevant accessibility needs.

Compare more than repeated-task scores

Review the first attempt and a later attempt using the same criteria, then add a fresh task. Separate task familiarity, self-reported confidence, observed performance, and longer-term outcomes in the report.

Keep assessment interpretation accountable

Use qualified human reviewers for consequential judgments. Check scoring consistency and record disagreements, missing data, and accommodations rather than treating an automated score as a direct measure of ability.

Ask for the study before citing a result

A learning-gain claim needs a report describing the sample, comparison, assessment instrument, timing, missing data, and limitations. Ask for those details and publication permission before quoting a percentage or treating a company study as independent research.

Report what the evidence supports

Describe the programme, method, findings, and uncertainty together. A small internal pilot should not be generalized to all subjects, learners, employment outcomes, or regulatory decisions.

Sample exchange

Reading an attempt-to-attempt result

FacultyAttempt one scored 52 and attempt three scored 68.

ResearcherWhich criterion moved most?

FacultyQuestioning, from 10 to 16.

ResearcherA higher score is a starting point. Review the scoring consistency and try a fresh task before attributing the change to the programme.

Frequently asked questions

Does this page provide a validated learning-gain percentage?+

No numerical result is asserted here. Request the study report and methodology before citing a learning-gain claim.

Can a higher score establish that learning transferred?+

A higher score on a familiar task is one observation. Use fresh tasks, review the criteria and conditions, and report alternative explanations before drawing a broader conclusion.

What should an institution agree before a pilot?+

Agree the evaluation question, task, learner group, baseline, reviewer responsibilities, data handling, success criteria, and how limitations will be reported.

Teachrity supports practice and evidence. Consequential decisions, such as grades, certification or hiring, stay with accountable people at the institution or employer.