Introduction
Over the last 30 years or so, computers have had quite an impact on language testing. As regards Computer Adaptive Testing (CAT) in particular, it is currently on the leading edge of assessment technology.
Defining and exploring CAT
CAT is a sophisticated method of test delivery that tailors test question difficulty to each individual test taker: if the test taker gives a wrong answer, the computer follows up with an easier question, and if he/she answers correctly, the next question will be more difficult.
Text by: Paul Bouniol
Advantages of CAT
All assessment methods come with their sets of ‘advantages’ and ‘disadvantages’ and clearly CAT is no panacea. Here are some of its strengths followed by its limitations:
- Test takers are provided with a personalised assessment experience, most of the items being in their ‘learning zone’.
- CAT is challenging but not overwhelming and this lowers the test takers’ anxiety.
- CAT can provide very precise ability estimates for all sections of a test.
- CAT requires fewer items compared to ‘traditional testing’: a computer adaptive test can be shortened by 50% compared to a ‘traditional test’ and still be precise.
- No time is ‘wasted’ on items too hard or too easy.
- Immediate feedback can be issued immediately after the test.
- Test security is improved (e.g., all tests being tailored to each test taker).
- CAT can be on-demand, available any time and offered online.
Limitations of CAT
- Test takers cannot go back in the test to change an answer.
- Test takers must respond to all items before they are allowed to go on to the next one.
- If questions start getting easier, test takers may become nervous believing they are not performing well.
- CAT can disadvantage test takers with lower PC skills or with no access to device that can ensure optimal performance (e.g., broadband Internet connection).
How the computer algorithm works
The computer follows distinct steps:
- The algorithm initially selects an item of medium difficulty.
- The test taker answers correctly or incorrectly.
- If he/she succeeds, the ability estimate is raised; if not, the ability estimate is lowered.
- The ability estimate is updated so that the next item can be even more appropriate.
- The same process follows till the test is stopped.
The item bank
The item bank is a collection of items constructed to measure specific abilities at various levels. It comprises a great amount of items and normally holds 30 to 40 times the items needed to administer a test (e.g., if a Listening and Reading test consists of 60 items, then the item bank will normally consist of 1,800 - 2,400 items).
All items are also ‘calibrated’. For each single item, the bank contains information such as its grade level, the content area it belongs to, its difficulty estimate, its use so far etc.
How scores are reported
It is important to note that the score is not based on the number of questions answered correctly, but on the ‘level’ of difficulty of questions reached. In the case of a language CAT, scores are usually expressed on a scale (e.g., from 0 to 100), linked to a corresponding CEFR level (e.g., ‘B2’) or translated into a grade (e.g., ‘Very good’).
Test security
Computer adaptive tests ensure security at a number of
levels:
- No two test takers see the same test items.
- During their test, the test takers’ desktop is frozen so that they cannot have access to the Internet/any other desktop tools (e.g., Google).
- Access to the system is secure and password protected, encryption algorithms are used etc.
The LanguageCert LTE Listening & Reading Test
The LanguageCert LTE Listening and Reading Test, a test gaining more and more popularity recently, is a computer-adaptive test of maximum duration (90 mins) assessing the test taker’s English language proficiency. The content of each individual test is responsive to the language proficiency the examinee demonstrates as the test progresses.
Who the LanguageCert LTE Listening and Reading Test is intended for
The LanguageCert LTE Listening and Reading Test is intended for non-native speakers of English who wish to:
- acquire an internationally recognised certification of their English language competence,
- provide employers with information as to their English language level,
- improve their target language and monitor their own progress,
- ndertake a course of study at an English-speaking university/college and require evidence of their English proficiency.
The Listening component
The test taker is required to listen to a range of real-life/work-related material, including spoken utterances, dialogues, longer conversations, interviews, or talks.
The Reading component
The test taker is asked to read short and long authentic texts on general/work-related topics. It assesses the test taker’s ability to understand the main idea of a text or look for specific information or line of argument. The test taker’s language is also assessed with various items testing vocabulary and lexico-grammatical awareness.
How the test taker is assessed
With the completion of the test, the test-taker receives a score within a scale from 0 - 100 that also comes with an indication of how this score translates into an English language level. This LTE test scale (0-100) is aligned to the six CEFR levels (A1-C2).
Benefits for the test taker
The LanguageCert LTE Listening and Reading Test:
- is computer-based and can easily suit some candidates’ needs and schedule
- is mapped to the CEFR
- assesses real life skills
- assesses listening and reading skills as well as awareness of lexico-grammatical structures
- is available all year round
- can be taken online from an appropriate location, any time
- provides results within two business days and hard copy certificates within five business days
- does not require any entry requirements
- is Ofqual regulated and recognised by ASEP and internationally
Conclusion
Despite a few limitations, CAT is becoming widely popular nowadays and promises to further develop very soon. As regards the LanguageCert LTE Listening and Reading Test, feedback from the thousands of recent test takers has so far been more than positive, most probably due to its many benefits for candidates. Might the LanguageCert LTE Listening and Reading Test be an attractive option for YOUR students too?
