Abstract: English language learners (ELLs) now account for more than 10 percent of the US preK-12 population, and the number is still growing. How to help ELLs to achieve academic success while they are also acquiring English as a second language has become an increasingly important issue on the U.S. K-12 educational agenda. Meanwhile, the concept of literacy is changing in the 21st century with advances of technology, so is the way of learning and working. These changes pose challenges to traditional paper-and-pencil testing practices. They, however, also open up an array of opportunities for CTB to demonstrate leadership in providing new generations of high-quality online language assessment that are positioned to prepare K-12 ELLs for academic success in this digital age. Our proposed 2011 R&D project intends to contribute to CTB's capacity building in this aspect by introducing innovative, web-based task types that will be essential of those assessments. Different from a regular CTB assessment development operation, this project emphasizes groundbreaking yet cost-efficient applications of computer technology to design of language performance tasks. Furthermore, implementation issues regarding automated scoring and IRT scaling are tackled in order to expedite future process of converting those lab prototypes into industry-scale products. In this project we will focus on the initial design phase, which includes the following key aspects:
- 1) review of existing CTB LAS Links item types for its correspondence to language proficiency that may be required for ELLs to achieve Common Core State Standards,
- 2) proposal of new task types to address current LAS Links limitations,
- 3) development of algorithms for artificial intelligence (AI) scoring (AI scoring of extended oral or written discourse is not considered),
- 4) documentation of task specifications for future item writing, scoring, and technology implementation,
- 5) judgment of psychometric challenges and proposal of solutions,
- 6) live online demo of proposed task types, and
- 7) usability study with indepth feedback from a small sample of students, teachers, and assessment professionals.
In consideration of the one-year project scope restriction, no large-scale field test will be conducted for empirical evaluation regarding scoring rubrics and psychometric performance of the proposed task types. Empirical data collection and analysis, however, will be performed in the usability study to inform development and to provide validity evidence. Further restriction of the project scope includes investigating LAS Links-Common Core correspondence for just two grades: Grades 6 and 10. As LAS Links is targeted to the whole K-12 ELL population, we plan to address this population as a whole in the test design and not to focus on any subgroup. In the proposal below we describe the problem and context, planned research activities, deliverables and expected delivery dates, and staff roles, responsibilities, and estimated project cost.
