Every new test at ETS is created through a careful design and development process. The newest addition to the TOEFL® Family of Assessments, the TOEFL® Essentials™ test, is no exception. As detailed in the Design Framework for the TOEFL® Essentials™ Test 2021, the TOEFL Essentials test is a digitally delivered English-proficiency assessment that seeks to maximize efficiency while emphasizing test takers’ ability to communicate in English. Utilizing a multistage adaptive test design, it also enables the measurement of the full proficiency range of the Common European Framework of Reference. Here, we highlight the important roles played by rigorous task design and expert test development practices in creating a new test.
Text by Pablo Garcia Gomez* & Shoko Sasayama*
ETS’s approach to assessment task design
The tasks that make up the new TOEFL Essentials test were designed based on task-based principles to elicit what test takers can do in English rather than their mere knowledge of the language. This is particularly important when we are using a test like this to predict how successful test takers will be in the real world where they need to communicate effectively in English.
Let’s take writing tasks in the TOEFL Essentials test to illustrate our approach to assessment task design. The writing tasks are designed to give test takers a purpose and audience for writing, while also reflecting what test takers encounter in the real world. For example, writing a sentence or two to describe a picture is a fairly typical test task that has no purpose or audience. The same picture description task becomes much more engaging and meaningful when test takers imagine themselves posting the picture to their own social media account and writing an extended description about their related experiences. Such tasks can tell us a lot about what the test takers can do in English.
These writing tasks are also designed to be maximally effective at gauging test takers’ writing abilities at different levels of proficiency. For example, the social media post task is shown by research to effectively distinguish among learners at beginning levels of proficiency. However, its effectiveness is limited when it comes to test takers at higher proficiency levels, as they all are likely able to perform well. For these higher proficiency learners, the academic discussion task is much more effective in that it challenges test takers by requiring more sophisticated language use (e.g., vocabulary, grammar, discourse markers) to successfully complete the task, and as a result, it distinguishes among their abilities to do so. All in all, writing tasks (and other tasks) on the TOEFL Essentials test are designed to elicit what test takers can do in English, provide clear context and purpose to communicate in English, and allow maximal distinction among learners at different proficiency levels. All of these features optimize the effectiveness of the test and support accurate interpretations of test takers’ English abilities.
Leveraging leading assessment development expertise
Utilizing these kinds of design principles, assessment specialists at ETS have developed new tasks on the four skills (listening, reading, writing, and speaking) evaluated by the TOEFL Essentials test. Just as the design process was multistaged (i.e., a prototype stage, a pilot test and a field test), the development of the new test content also involves a complex, multistage process. The overall goal of this development process is to ensure that the content generated meets ETS’s quality and fairness standards and results in test-taking experiences that are similar in content, difficulty and level of engagement.
Assessment specialists are at the center of this process, and they lead it from start to finish. Assessment specialists, key members of the TOEFL Essentials test team, are experts in their field who have been trained in language learning or related subjects at the university level, and the majority of whom have taught at K–12 schools, colleges, or universities internationally. Many assessment specialists are themselves English-language learners who have achieved graduate-level degrees from universities where English is the language of instruction.
Throughout the content development process, assessment specialists follow detailed specifications. These specifications are generated during the test design stages and they are used for the creation and review of every single task or item in the test. For example, the writing tasks in the TOEFL Essentials test have clear specifications about the type and length of texts, instructions to the test taker, audio, graphics and photos used. After the test tasks are authored, they receive multiple, thorough reviews from different assessment specialists as well as separate editorial and fairness reviews.
Integrating technology for assessment development
A wide range of state-of-the-art technological tools are used to aid the test development process to make it more efficient. For example, a comprehensive platform is used by assessment specialists to generate, review and store items. For some item types, assessment specialists also use a proprietary technological capability, called Technology Assisted Item Creation (TAIC), to facilitate the item development process. TAIC integrates task content specifications and difficulty parameters specifically developed for the TOEFL Essentials test. It is important to note, however, that after the content for these items is generated through TAIC, it undergoes the rigorous, multistage review process performed by the assessment specialists to ensure that the items developed meet ETS’s standards and expectations before they ever get presented to a test taker.
These careful and rigorous design and development practices are what makes the new TOEFL Essentials test a valid and reliable test that provides trustworthy information about test takers’ ability to use English for communication purposes.
To learn more about the new TOEFL Essentials test, visit https://www.ets.org/s/toefl-essentials/score-users/
*Pablo Garcia Gomez is an assessment designer at ETS.
*Shoko Sasayama is an associate research scientist at ETS.
