Skip to content
Printer-friendly version

NCSC Frequently Asked Questions

National Center and State Collaborative (NCSC) Frequently Asked Questions Summative Assessment-February 15, 2013.

OVERVIEW OF QUESTIONS

  1. What is the general design of the NCSC AA-AAS?

    1. What is the format of the NCSC Alternate Assessment based on Alternate Achievement Standards (AA-AAS)?
    2. What grades and content will be assessed?
    3. How long will it take a student to complete the assessment?
    4. How will we assess students at all levels of current achievement?
    5. How will students be matched to items of appropriate challenge?
    6. Do students need to take items at a certain level of complexity to be determined as “on track” or College and Career Ready?
    7. How will students with the greatest challenges access the test?
    8. How will technology be used in the NCSC assessment?
    9. How will state partners be involved in the NCSC summative assessment development?

  1. What is the general design of the NCSC AA-AAS?
    The general design decisions made as of February 2013, are described in responses to questions 1a-1i. These decisions will be tested by iterative item and test development steps, cognitive labs, and pilot testing that are central to our work through 2015, when the first administration of the full census field/operational test is complete.

    1. What is the format of the NCSC Alternate Assessment based on Alternate Achievement Standards (AA-AAS)?

      The NCSC AA-AAS in mathematics and English language arts (ELA) will be on demand, item-based assessments of approximately 30 items that assess a set of approximately 10 prioritized content targets per grade level. A trained testing administrator familiar to the student (e.g., the student’s teacher) will administer the items to the student over the course of one or more testing sessions, based on the student’s needs. Testing sessions will be scheduled within a testing window of approximately two months.

      A variety of item types will appear in the assessments. A principled design approach, based on the evidence-centered design (ECD) literature, is being used to determine the appropriate item types for each assessed content target. The items are primarily selected response. Constructed response items are used when the principled design approach indicates they are needed to measure the intended content. Each content target will be assessed by items that have been carefully and intentionally designed to assess performance across the expected performance continuum. The NCSC principled design approach involves the development of item prototypes for each content target. Similar to ECD, the process produces Design Patterns and Task Templates that define the characteristics of items. NCSC’s Task Templates include prototypes for items assessing the content targets at each of four complexity levels, labeled 4-3-2-1. These prototypes inform the item specifications for NCSC item development work occurring in 2013.

      The process for ensuring that each student receives items at an appropriate level of challenge is likely to involve the use of classroom data, locator tests, or multi-stage adaptive testing. The administration of the items will be scripted with a structured protocol for determining the ways in which a student may interact with the items so that what is being measured is not changed. (See question 1d and 1e for more information.)

    2. What grades and content will be assessed?

      NCSC will deliver an AA-AAS in mathematics and in English language arts (both reading and writing), for grades 3-8 and 11. Prioritized content targets have been identified for all grades based on learning progressions and alignment to the Common Core State Standards (CCSS). The full census field/operational test on which achievement standards are being set, will be administered in spring 2015.

    3. How long will it take a student to complete the assessment?

      Expected testing time is approximately 1.5 – 2 hours per student per content area (mathematics and ELA), individually administered. The length of assessment will be informed by research from the item tryouts and cognitive labs being conducted in 2013. Within the expected two-month testing window, teachers can complete each student’s assessment in multiple smaller time slots that meet the student’s needs.

    4. How will the NCSC AA-AAS assess students at all levels of achievement?

      Students eligible to be assessed on an AA-AAS represent a highly diverse population of learners with unique learning needs. These needs may require a range of instructional supports as knowledge and skills are developed in the academic domains. Given consideration of the varied learning characteristics of students with significant cognitive disabilities, and their varied levels of current academic achievement, item prototypes for math and ELA are based on expected performance of students across the full ability range.

      Most of the items (about 75%) are aligned to the grade-level CCSS (near link1) and allow demonstration of skills and knowledge at a broad range of academic complexity. Some of the items (about 25%) are a farther link to the grade-level CCSS, and are designed to allow students who are just beginning to interact with the academic content to show what they know and can do on the content targets at a less complex level. The proportion of items at each complexity level that any individual student will see will be guided by classroom data/locator/stage adaptive processes, as described below.

      In addition to content complexity, the embedded scaffolds for each complexity level give all students with significant cognitive disabilities the opportunity to respond independently. The complexity range of the items provides access to the assessment for students with unique access needs and represents the range of content performance observed in research when students have had appropriate opportunity to learn conditions.

      The design work for the assessment makes the assumption that students have received effective, evidence-based instruction, as promoted by NCSC’s instructional resources, directly aligned to the CCSS. The project is dedicated to providing professional development to teachers so that they understand the assessed content and can align instruction with assessment in ways that include all students who will participate in the NCSC AA-AAS.

    5. How will students be matched to items of appropriate challenge?

      The NCSC AA-AAS is designed to help ensure that all students have an opportunity to show what they know consistent with their progress through the content. That is, a student who is new to the academic content covered on the assessment is assumed to best show what he or she knows on some of the less complex items.

      One of the design priorities for the summative assessment is to minimize student interaction with content that is at an inappropriate level of challenge for the student. The summative assessment will include an embedded locator test that, in combination with classroom data, will help determine the level of complexity and the length of the summative assessment for individual students. Moreover, current design plans involve using performance information from an initial test session to inform the level of complexity of a subsequent session – a stage adaptive design.

      Based on NCSC data on academic achievement for students participating in AA-AAS, it is likely that few students initially will take items at the highest level of complexity. In the initial years of the assessment program, it is anticipated that many, if not most, students will take items at the lower levels. Over time, with a longer history of instruction in the CCSS, and with increased teacher implementation of NCSC curriculum, instruction, and the provision of professional development resources, we anticipate most students will demonstrate success on items and tasks that are increasingly cognitively complex.

    6. Do students need to take items at a certain level of complexity to be determined as “on track” or College and Career Ready?

      The item complexity levels (4-3-2-1) in the NCSC Task Templates (see question 1a) do not correspond to eventual achievement levels that will be defined and set. They simply are a way to develop items along a broad range of performance to ensure that all students are able to show their current performance on the NCSC AA-AAS. Achievement standards will be set after administration of the NCSC AA-AAS in the spring of 2015. Just as for the general assessment consortia, it is anticipated that the shift to the NCSC AA-AAS will be very challenging for students, but that with time and opportunity to learn, performance will increase and achievement standards revisited.

    7. How will students with the greatest challenges be able to access the test?

      NCSC’s commitment is to build educator capacity in each state to intervene appropriately with students who currently do not have an effective system of communication. This is NCSC’s highest priority for ensuring all students can access the test. Without an effective mode of communication, a student cannot demonstrate his or her knowledge and skills.

      NCSC’s research shows what students can demonstrate after evidence-based instruction for a reasonable amount of time. The research data informed the range of complexity included in the assessment. Evidence from tryouts of NCSC’s curricular materials2, small scale item tryouts, and cognitive labs, as well as the NCSC pilot testing is used to systematically ensure that the range of complexity in the NCSC AA-AAS is appropriate for the targeted population of students.

      There is a small group of students with extremely low incidence needs among those participating in the NCSC AA-AAS. This group, sometimes referenced as the 1% of the 1%, will be supported by their teachers who receive professional development materials focused on providing access for these students to the CCSS and for participation in the NCSC AA-AAS.

    8. How will technology be used in the NCSC assessment?

      Students interact with the NCSC assessment in a variety of ways, depending on their needs. Some students will interact with the computer and the testing platform directly, with the test administrator monitoring their work. For other students, their test administrator may print out testing materials, administer the items, and enter student responses into the computer. The NCSC system architecture is designed to accommodate use of emerging technologies such as tablets. Training materials will introduce students and teachers to the technologies for the NCSC assessment.

      The assessment will be delivered through a technology platform that has been defined in the NCSC Technology Architecture Plan and Report [http://3f071e93aad6392d132c-25358a031817aa7f80c72ac2922ef9ef.r3.cf2.rackcdn.com/NCSC_TechArchReport_103112.pdf] External link opens in new window or tab. (PDF; 7MB). See pages 69-71 for hardware requirements for end users.

    9. How will state partners be involved in the NCSC summative assessment development?

      See the detailed work plans for the item development vendor (Measured Progress/Questar) and the summative assessment vendor (CTB/McGraw Hill) which will be posted on SharePoint.

      States will be intensively involved in discussions on performance level descriptors, high level blueprints, and results from small scale tryouts of item prototypes in the ELA task templates. In addition, a steering committee of NCSC partners and states provides direction for processes and deliverables for the vendors. Each of the work plans associated with item development and with summative assessment development involve state leads, state stakeholders, and district teachers for item reviews, cognitive labs, pilot testing, and review and revision processes. In addition, small topical expert groups of state volunteers are consulted on specific topics, to provide rapid turn-around discussion and recommendations on issues that arise, e.g., related to technology infrastructure, assessment design considerations for specific disabilities , or accessibility features. Weekly calls take place to share assessment information beyond the core groups working at any given time, and to bring assessment policy decisions to the larger group for review, discussion, and consensus. The weekly calls also continue to address curriculum, instruction, and professional development topics.

Back to top


1 Alignment judgments are based on the Links for Academic Learning model. See: Flowers, C., Wakeman, S., Browder, D., & Karvonen, M. (2009). Links for Academic Learning (LAL): A conceptual model for investigating alignment of alternate assessments based on alternate achievement standards. Educational Measurement: Issues and Practice, 28(1), 25-37.
NCSC Internal FAQ Winter 2013 Page 3

2 A video, released March 2013 of one of the Mathematics Activities for Scripted Systematic Instruction (MASSI) training webinars, features teacher and student interactions on the effective use of the curricular materials to adapt instruction for students with the most complex challenges.

Questions:   Kristen Brown | NCSC@cde.ca.gov | 916-445-1064
Download Free Readers