Resources

Presentations

Title Conference Author Date
The I-SMART Project: Creating Multi-Dimensional Science Assessments Using Principles of Universal Design for Learning and Fine-Grained Learning Maps National Conference on Student Assessment, Orlando, FL Shipman, S., Tiemann, G.C., Dolan, R, & Swinburne Romine, R. June, 2019
Design Features Supporting Teachers' Use of a Dashboard for Diagnostic Assessment Results AERA 2019 Annual Meeting, Toronto, ON Starr, E., Dolan, R., Wojcik, C., Ducharme, K., & Blackorby, J. April, 2019
Teacher-Centric Design Process for a Dashboard to Support Formative Assessment National Council on Measurement in Education, Toronto, ON Dolan, R., Starr, E., Wojcik, C., Ducharme, K., & Blackorby, J. April, 2019
Blending Evidence-Centered Design and Univeral Design for Learning in Next-Generation Science Assessment National Council on Measurement in Education, Toronto, ON Karvonen, M., Andersen, L., Swinburne Romine, R., & Tiemann, G. April, 2019
Using Learning Map Models to Design Accessible Assessments NARST 2019 Conference, Baltimore, MD Andersen, L., Bechard, S., Swinburne Romine, R., Ruhter, L., & Shipman, M. April, 2019
Design and Use of Innovative Science Testlets for Struggling Learners NCME Special Conference on Classroom Assessment, Lawrence, KS Andersen, L., Shipman, M., & Tiemann, G.  September, 2018
Innovations in Science Map, Assessment, and Report Technologies (I-SMART) National Conference on Student Assessment, San Diego, CA Torchon, M. & Tiemann, G. June, 2018
The Development of Universally Designed, Fine-Grained Science Learning Map Models 2018 NARST International Conference, Atlanta, GA Andersen, L. & Swinburne Romine, R. March, 2018
Innovations in Science Map, Assessment, and Report Technologies (I-SMART) 2018 SCILLSS & I-SMART Introductory Meeting Karvonen, M. January, 2018
Innovations in Science Map, Assessment, and Report Technologies (I-SMART) Technical Issues in Large Scale Assessment (TILSA) State Collaborative on Assessment and Student Standards (SCASS), Dallas, TX Karvonen, M. October, 2017

Publications

Title Publication Type Author Date
Developing and Evaluating Learning Map Models in Science: Evidence from the I-SMART Project Technical Report Swinburne Romine, R., Andersen, L., Schuster, J., & Karvonen, M. 2018
Developing & Evaluating Science Learning Map Models (I-SMART Project Brief No. 19.1) Project Brief Center for Accessible Teaching, Learning, & Assessment Systems 2019

Essential Element Concept Maps (EECMs)

Essential Element Concept Maps (EECMs) incorporate important features of evidence-centered design (ECD), learning map models (LMs), and Universal Design for Learning (UDL) in a framework that was appropriate for educators serving as item writers. The EECMs for this project are extensions of prior work for the DLM  ELA, math, and science assessments. In the DLM assessment, the five assessment targets  (linkage levels)  derived from LMs each have a unique test design pattern. The EECM serves as guide for test developers that contains the specifications for assessment items and testlets in a format that integrates the structure of the learning map model.  

Example EECM (pdf)

Each EECM has a separate section for each linkage level that expands upon the description of the assessment targets. Here is an example EECM that describe the target linkage level test design for EE.5.LS2-1. Vocabulary words and example questions are specified in the EECMs so that all students, regardless of their communication mode, can respond to items. Misconceptions are possible student errors related to the concepts assessed by the node. Progression information  summarizes the science concepts and SEP-related knowledge, skills, and understandings within the LL. The phenomenon  section describes the general mechanism behind the science concepts being assessed, as well as an example phenomenon that could be used in an assessment that is appropriate for the LL. The science and engineering practices  section describes how the students use the SEP at this linkage level. The four nodes assessed at this LL are presented in a table with detailed descriptions of the skills and examples of how the skills can be observed or assessed. The skills and connections in the progression of learning within a LL are shown in the Linkage Level Mini-Map. 

UDL options to be used in the LL are organized in the EECM by the three principles of multiple means of representation, engagement, and action and expression, as well as the corresponding guidelines under each principle. The UDL options are specific to each LL and include options provided through testlet content, such as options for language and symbols, comprehension, recruiting interest, self-regulation, expression and communication, and executive functions. For example, at the target level, a wonder question and a “How did you do?” reflection question are self-regulation options that helps students develop self-assessment and reflection skills.

The EECM also includes an excerpt from the research narrative that was prepared during LM construction, which describes the progression of knowledge for the DCI and SEP in more detail. The narrative highlights specific connections among the nodes in the LLs.