Types of reliability
- Test-retest: correlate scores from two administrations; memory and practice effects are drawbacks.
- Parallel forms: compare two equivalent versions.
- Split-half: correlate two halves and adjust with the Spearman-Brown formula.
- Internal consistency: Cronbach's alpha, with about 0.7 or above often treated as acceptable.
- Inter-rater: agreement among observers, such as Cohen's kappa.
Types of validity
- Content validity: experts judge whether items cover the concept's domain; face validity is a surface judgement.
- Criterion validity: correlation with an external standard, either concurrent or predictive.
- Construct validity: fit with the theoretical concept, shown by convergent and discriminant evidence and factor analysis.
How they relate
In the target analogy, reliability is how tightly shots cluster and validity is whether they hit the centre. Random error lowers reliability, systematic error lowers validity. Low reliability caps validity, but high reliability does not guarantee it.
Improving measures
Add items, revise ambiguous wording, standardise conditions, train interviewers and pilot test. Clear concept definitions and established scales support validity. These differ from the internal and external validity of a research design.
References
- Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
- Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.
- Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302.
- Rubin, A., & Babbie, E. R. (2016). Research Methods for Social Work (9th ed.). Cengage.