APPLIED LINGUISTICS for BA Students in English
EVALUATION
Criteria of good tests
Validity
Validity always refers to the degree to which the gathered empirical evidence supports the adequacy and appropriateness of the inferences that are made from the scores (Bachman 1990) It means that the interpretations and uses that we make of test scores are to be valid, in other words, a test is said to be valid to the extent that is measures what it is supposed to measure, (Alderson, 1995) If the test is not valid for the purpose for which it was designed, the scores do not mean what they are supposed to mean. There are different types of validity, which in reality are different ’methods’ of assessing validity (Bachman 1990) There are three main types/aspects of validity that can be distinguished: internal, external and construct validity.
Internal validity relates to the perceived content of the test and its perceived effect. There are three aspects: face validity, content and response validity
- Face validity refers to the surface credibility or public acceptability of the test. It involves intuitive judgment about the test content made by so called ’lay’ people, who are involved in the testing process but who are not experts in testing. Such people include non-expert users, students, their teachers, administrators. If test takers accept the test as a face valid test, they are more likely to perform to the best of their ability on that test. Data on face validity can be collected by interviewing test takers, students or asking them to complete questionnaires about their feelings about or reactions to the test that they have taken.
- Content validity shows whether the test contains a representative sample of the relevant language skills. It can be proved by a systematic analysis of the test content carried out by experts, who make their judgments by comparing it with e.g. the test specification, which states what the content should be or with a formal syllabus or curriculum, or by rating test items and texts following a precise list of criteria. A further alternative can be interviewing teachers of a range of academic subjects, or administering a questionnaire survey in which respondents are asked to make judgments about the texts and tasks.
- Response validity can be checked by gathering information on how test takers respond to the test items. It can be collected by asking learners and test takers to tell how they responded to the test item, what their test taking behavior was, because the reasoning and the processes they follow when they are solving the items give important indications of what the test is testing.
External validity relates to procedures which compare students’ test scores with measures of their ability taken from outside the test. It has two types: concurrent and predictive validity.
- Concurrent validation involves comparing the test scores of the candidates with some other measure for the same candidates taken roughly at the same time as the test. This measure can be expressed numerically with statistical methods by correlating students’ test results with their test scores gained on other tests, with teachers’ rankings or with the students’ own ratings of their language ability in the form of self-assessment.
- Predictive validation involves comparing the students’ test scores with some other external measure taken some time after the test has been administered. This measure also can be expressed numerically with statistical methods by correlating students’ test results with their scores gained on other tests taken some time later (e.g. correlate entrance test results with scores of final tests).
Construct validity shows to what extent the test is based upon its underlying theory, that is, how well test performance can be interpreted as a meaningful measure of some characteristic or quality. The term ‘construct’ refers to a psychological construct, a theoretical concept about a kind of language behavior that the test makers want to measure. It can be regarded as a kind of attribute or ability of people which is assumed to be reflected in test performance. Construct validity refers to the extent to which performance on tests is consistent with the predictions that test makers make based on a theory of abilities, or constructs. (Bachman, 1990) It can be assessed by correlating different test components (sub tests) with each other or by complex statistics, a combination of internal and external validation.
