Test comparisons
Analysis
Purpose
Comparing two tests of proficiency to identify whether they are interchangeable.
Approach
I used a range of methods from quantitative educational metrics analysis to identify whether the outcomes on two measures of English proficiency could be mapped to each other to obtain a concordance.
They key outcome was a report, profiling whether the tests appeared to be:
- Measuring the same construct
- Able to conform to the principle of equity, i.e. once two test forms were equated, it should not matter to the test-taker which form of the test is administered.
- Symmetrical, i.e. equating of X to Y must be the inverse of equating Y to X.
- Invariant by subpopulation, i.e. there would be no differences when equating by subpopulations of takers
- Providing the same inferences, i.e. designed for use to reach the same kinds of conclusions
- Applied to the same target population
Outcome
- I provided advice to the client as to why “traditional” statistical tests, as suggested in a previous consultation by someone else, were not appropriate for this task, and practical actionable steps forward based on the analysis.
Key tools
- R: Rmarkdown, tidyverse; Git + GitHub.