This is an accessible alternate version of the Transition to New Assessments Update (PPT; 2.2 MB; 45 slides). Reauthorization of the Statewide Pupil Assessment System and SMARTER Balanced Assessment Consortium (SBAC). This document provides text translation to the Transitioning to New Assessments: Reauthorization of the Statewide Pupil Assessment System and SMARTER Balanced Assessment Consortium (SBAC) Update PowerPoint presentation. The PowerPoint presentation was presented at the Assessment and Accountability Information Meetings in Ontario California on September 19, 2012, and in Sacramento California on October 1, 2012.
Transitioning to New Assessment
Reauthorization of the Statewide Pupil Assessment System and SMARTER Balanced Assessment Consortium (SBAC) Update
Ontario - September 19, 2012
Sacramento - October 1, 2012
California Department of Education
Tom Torlakson, State Superintendent of Public Instruction
California Current Assessments
- Standardized Testing and Reporting Program (STAR)
- California Standards Tests (CSTs)
- Early Assessment Program (EAP)
- California Alternate Performance Assessment (CAPA)
- California Modified Assessment (CMA)
- Standards-based Tests in Spanish (STS)
- California High School Exit Exam (CAHSEE)
- California English Language Development Test (CELDT)
- General Educational Development Test (GED)
- California High School Proficiency Exam (CHSPE)
- National Assessment of Educational Progress (NAEP)
- Physical Fitness Test (PFT)
Transitioning to New Assessments
- California Education Code (EC) Section 60604.5 requires the State Superintendent of Public Instruction (SSPI) to:
- Develop recommendations addressing 16 areas toward reauthorization of the assessment system
- Develop a plan for transitioning to new “high quality assessments”
- Consult with specific stakeholders:
- The State Board of Education
- Public Schools Accountability Act (PSAA) Committee
- Measurement experts from California’s private and public universities
- Experts in assessing students with disabilities and English learners
- Teachers, administrators, and governing boards from California’s local educational agencies
- Parents
- Recommendations and Plan due to the Legislature fall 2012
“High-quality assessment”
- Means an assessment designed to measure a pupil’s knowledge of, understanding of, and ability to apply critical concepts through the use of a variety of item types and formats, including, but not limited to, items that allow for open-ended responses and items that require the completion of performance-based tasks
16 Areas of Consideration
The recommendations shall consider including all of the following elements in the reauthorized assessment system:
- Aligning assessments to standards
- Implementing common assessments developed by state collaborative
- Conform to Elementary and Secondary Education Act (ESEA) reauthorization
- Measurement of achievement at a point in time and over time for groups and subgroups of pupils and for individual pupils.
- Allow for comparison from one year to the next as a reflection of growth over time
- Valid, reliable, and fair for all students including English learners and students with disabilities
- Assessments of English learners using primary language assessments
- Ensure no bias with respect to race, ethnicity, culture, religion, gender, or sexual orientation
- Incorporate a variety of item types including open-ended and performance-based tasks
- Generating multiple measures of pupil achievement, which, when combined with other measures, can be used to determine the effectiveness of instruction and the extent of learning
- Assess science and history–social science in all grade levels at or above grade 4
- Assess understanding and ability to use technology necessary for success in the 21st century classroom and workplace
- Formative and interim assessments that provide timely feedback for purposes of continually adjusting instruction to improve learning
- Use test administration and scoring technologies that will allow the return of test results to parents and teachers as soon as possible
- Minimizing testing time
- Including options for diagnostic assessments for pupils in grade 2
Diagnostic Assessment
- Education Code 60603 defines diagnostic assessment as: “ . . . assessment of the current level of achievement of a pupil that serves both of the following purposes:
- The identification of particular academic standards or skills a pupil has or has not yet achieved.
- The identification of possible reasons that a pupil has not yet achieved particular academic standards or skills.”
Formative Assessment
Education Code 60603 defines formative assessment as: “Assessment tools and processes that are embedded in instruction and are used by teachers and pupils to provide timely feedback for the purposes of adjusting instruction and to improve learning.”
Interim Assessments
Education Code 60603 defines interim assessments as: “ . . . an assessment that is given at regular and specified intervals throughout the school year, is designed to evaluate a pupil’s knowledge and skill relative to a specific set of academic standards, and produces results that can be aggregated by course, grade level, school, or local education agency in order to inform teachers and administrator at the pupil, classroom, school, and local education agency levels.”
Summative Assessment
Education Code 60603 defines “achievement test” as: “ . . . any standardized test that measures the level of performance that a pupil has achieved in the core curriculum areas.”
Relationship of Assessments
Graphic illustration from The Role of Interim Assessments in a Comprehensive Assessment System (2007). Authors: Perie, Marion, Gong, Wurtzel.
The right triangle graphic illustrates the distinctions between three types of assessments: Formative classroom (minute-by-minute, integrated into the lesson) (the largest portion of the triangle), Interim (instructional, evaluative, predictive) (the next largest portion of the triangle), and Summative (the smallest portion of the triangle). The y-axis of the triangle is labeled: Score and Duration of Cycle: increasing with an arrow moving up the triangle. The x-axis is labeled Frequency of administration with an arrow moving to the right.
Common Assessment Formats
- Paper and pencil testing
- Most common type of format
- Used for all assessments
- Computer based testing (CBT)
- Uses fixed form but is administered using a computer
- Computer adaptive testing (CAT)
- Presents harder or easier questions, depending on how the student performs as the test proceeds
- Can provide greater accuracy with fewer questions
Computer Adaptive Testing
The system will adjust the difficulty of items throughout the assessment. If the student answers the question correctly, the system will give a more difficult question as the next question. If the student answers the question incorrectly, the system will give an easier question as the next question.
Computer Adaptive Testing
20 Items
A whisker point range graph. The y-axis is labeled Achievement Score and has a scale from 150 to 250 with tick marks every 10 points. The x-axis is labeled Test Questions and has a scale of 0 to 20 with a tick mark at every point.
There is a line for “Basic” at 200 point of the y-axis. There is a line for “Advanced” at about 225 on the y-axis. A label for “Proficient is at 210 on the y-axis.
The data points are as follows:
- Test question 1 is 175 with a whisker score range from 0 to 200
- Test question 2 is 191 with a whisker score range from 175 to 210
- Test question 3 is 202 with a whisker score range from 187 to 216
- Test question 4 is 210 with a whisker score range from 199 to 224
- Test question 5 is 216 with a whisker score range from 205 to 228
- Test question 6 is 221 with a whisker score range from 211 to 231
- Test question 7 is 225 with a whisker score range from 216 to 235
- Test question 8 is 228 with a whisker score range from 220 to 236
- Test question 9 is 231 with a whisker score range from 224 to 240
- Test question 10 is 229 with a whisker score range from 221 to 236
- Test question 11 is 226 with a whisker score range from 219 to 232
- Test question 12 is 228 with a whisker score range from 221 to 235
- Test question 13 is 230 with a whisker score range from 224 to 236
- Test question 14 is 232 with a whisker score range from 225 to 238
- Test question 15 is 234 with a whisker score range from 228 to 240
- Test question 16 is 235 with a whisker score range from 229 to 241
- Test question 17 is 234 with a whisker score range from 228 to 240
- Test question 18 is 233 with a whisker score range from 228 to 238
- Test question 19 is 234 with a whisker score range from 229 to 239
- Test question 20 is 235 with a whisker score range from 230 to 240
Six Item Types
- Selected Response
- Short Constructed Response
- Extended Constructed Response
- Performance Tasks
- Technology-Enabled
- Technology-Enhanced
The Smarter Balanced Assessment Consortium (SBAC)
- Area of Consideration 2 (of 16): Implementing common assessments developed by state collaborative
- Assessments for grades 3 through 8 and 11 aligned to the Common Core State Standards (English language-arts and Math)
- Operational across Consortium states in the 2014–15 school year
- Meets Elementary and Secondary Education Act (ESEA) assessment requirements
SMARTER Balanced Assessment Consortium (SBAC) States
- 25 states educating approximately 20 million of the nation’s public Kindergarten to grade 12 students
- 21 governing states, 4 advisory states
Illustration: a graphic of the United States showing the SMARTER Balanced Assessment Consortium (SBAC) Member States.
A legend shows states shaded in green are Governing States and states shaded in blue are Advisory States.
The states that are Governing States are as follows:- Washington
- Oregon
- California
- Idaho
- Nevada
- Montana
- South Dakota
- Kansas
- Iowa
- Missouri
- Wisconsin
- Michigan
- West Virginia
- Maine
- New Hampshire
- Vermont
- Connecticut
- Delaware
- North Carolina
- South Carolina
- Hawaii
- North Dakota
- Wyoming
- Pennsylvania
- Alabama
SMARTER Balanced Assessment Consortium (SBAC) Procurements Managed by the State of Washington
Status current as of 8/22/12
| Request for Proposals (RFP) ID | Request for Proposal Name | Contractor | Request for Proposal Released | Work Start Date |
|---|---|---|---|---|
| RFP-00 | Project Management Partner | WestEd | 06/04/2010 | 10/01/2010 |
| RFP-01 | Communications Services Provider | GMMB | 02/25/2011 | 06/03/2011 |
| RFP-02 | Technology Readiness Tool | Pearson | 05/20/2011 | 12/07/2011 |
| RFP-03 | IT Systems Architecture | Measured Progress | 05/27/2011 | 09/15/2011 |
| RFP-04 | Item Specifications | Measured Progress / Education Testing Services | 07/27/2011 | 12/01/2011 |
| RFP-05 | Psychometric Services | Education Testing Services | 10/07/2011 | 02/06/2012 |
| RFP-06 | Accessibility and Accommodations Policy Guidelines | Measured Progress | 09/22/2011 | 01/04/2012 |
| RFP-07 | Item Authoring / Item Pool | Pacific Metrics | 12/09/2011 | 04/03/2012 |
| RFP-08 | Item/Task Materials Development | Measured Progress | 10/14/2011 | 02/06/2012 |
| RFP-09 | Test and Computer Adaptive Testing Specifications | Education Testing Services | 10/25/2011 | 02/06/2012 |
| RFP-11/ | Test Engine Development for Pilot and Field Test | 03/07/2012 | 08/20/2012 | |
| RFP-18/ | and Computer Adaptive Testing Simulations | |||
| RFP-20 | ||||
| RFP-12 | Initial Achievement Level Descriptors | CTB/McGraw-Hill | 03/14/2012 | 07/13/2012 |
| RFP-13 | Translations | 06/15/2012 | 07/15/2012 | |
| RFP-14 | Item/Task Writing/Review - Pilot | CTB/McGraw-Hill | 12/13/2011 | 03/28/2012 |
| RFP-15 | Report Development | 04/18/2012 | 09/2012 | |
| RFP-16 | Item/Task Development - Field Test | 09/2012 | 01/2013 | |
| RFP-17 | Scoring | 11/2012 | 02/2013 | |
| RFP-19 | Test Administration | 04/13/2012 | 09/2012 | |
| RFP-21 | Standard Setting | 01/2014 | 04/2014 | |
| RFP-22 | Sustainability Task Force | 04/12/2012 | 09/2012 | |
| RFP-23 | Formative Assessment Practices and Professional Learning | 08/2012 | 11/2012 |
Logical Components
- Shared Services
- Portal:
- Single sign on entry point
- Custom to specific role/user
- Widgets from other components.
- Program Management
- Program Information
- Style sheets
- Core Standards and Metadata for learning data and ontology
- Digital Library
- Monitoring and Alerting
- SSO
- User Preferences
- Permissions
- Identifier Management (i.e., Student ID map)
- Assessment Creation and Management
- Item Authoring
- Graphic edit/view items and stats
- Manage item workflow.
- Item Bank
- Create, update, get and delete item
- Item grouping
- Store item stats
- Item versioning
- Item lineage
- Item media?
- Item query capability
- Test Authoring
- Manage test creation workflow
- Test Packager
- Prepare items and blueprints for delivery system
- Test Spec Bank
- Blueprints
- Test Item Bank
- Operational and field test items
- Interim test items
- Assessment Delivery
- Adaptive Engine
- Human Scoring
- Machine Scoring
- Distributed Scoring
- AI Scoring Engine
- Test Administration
- Manage Scheduling
- Test Delivery
- Proctor Workstation
- Student Workstation
- Test Registration
- Manage Rostering
- Pull Accessibility Profile from SIS?
- Assessment Reporting
- Data Warehouse
- Time Variant Data
- Cleansed data
- Reporting
- Canned SMARTER Balanced Assessment Consortium (SBAC) created reports
- Custom Reports
- Report Delivery mechanisms
- Batch and Real-time execution
English Language Arts and Mathematics, Grades 3 through 8 and High School
Timeline graphic reviewing assessment processes for English-language arts and mathematics in grades 3 through 8 and high school. The timeline covers from the beginning to the end of the school year. Dotted boxes at the beginning and middle of the school year represent Interim Assessment using computer adaptive assessment and performance tasks. Two clipboard graphics near the end of the timeline depict first performance tasks in reading, writing, and math, and second end of year adaptive assessment with a re-take option. A note below the timeline states: Scope, sequence, number, and timing of interim assessments locally determined. A legend under the timeline shows the dotted box represents optional interim assessment system and the clipboard graphic represents summative assessment for accountability. A footnote at the bottom of the graphic states: Time windows may be adjusted based on results from the research agenda and final implementation decisions.
School-level Technology Considerations
- Technology-enhanced test items will require compatibility with latest browser technologies
- Test security will require device to run in “kiosk mode”
- Devices must support text-to-speech software
- Test items with multimedia content will require sufficient internal and Internet network bandwidth
Technology Readiness Tool
- Assesses current capacity and compares to the technology that is expected to be required for new SMARTER Balanced Assessment Consortium (SBAC) assessments
- Areas evaluated: devices, devices to tester ratio, network infrastructure, and staff and personnel
- Performs inventory, gap analysis and recommended areas where local educational agencies (LEAs) will need to upgrade technology
Technology Readiness Tool Data Collection
- First data collection window was from March 20, 2012 to June 30, 2012
- Subsequent data collections will occur each spring and fall through 2014
- Input information for computers expected to be available for online testing in 2014.
- 600,000 devices entered into system
- Operating systems inventory
- Windows XP: 52%
- Windows 7: 20%
- Mac OS X: 20%
- iOS 5.x: 3%
- Others (Vista, Linux, Windows – Other, iOS 4.x, Google Chrome, Android)
- Hardware types
- Desktops: 67%
- Laptops: 19%
- Netbooks: 6%
- Tablets: 4%
- Thin Clients: 4%
Guidelines for Purchasing New Hardware
- Inform current and future purchases
- Covering the vast majority of commercially available computers and tablets
- Full guidelines are available at the SMARTER Balanced Assessment Consortium Technology Web site.
California Work with SMARTER Balanced Assessment Consortium (SBAC): Opportunities for Teacher Involvement
- Feedback on draft content and item specifications
- Writing and reviewing of test items and tasks (2012 through 2013 pilot test; 2013 through 2014 field test)
- Range-finding and score validation
- Collaborate on designing score reports and Web tools for digital libraries (2013 through 2014)
- Scoring of performance tasks (2014 through 2015 and beyond)
- Nominations for determining Achievement Level Descriptors (ALD)
Current California Department of Education Participation
Work Groups Involved
- Group 1: Item Development + Performance Tasks
- Group 2: Technology + Reporting
- Group 3: Test Design + Validation and Psychometrics
- Group 4: Formative Assessment Practices and Professional Learning + Transition to Common Core State Standards (CCSS)
- Group 5: Accessibility and Accommodations + Test Administration
- Deb Sigman – Executive Committee
Current California Participation
- SMARTER Balanced Assessment Consortium (SBAC) Work Groups
- Technology Approach Work Group—Rodney Okamoto, California Department of Education (CDE) Education Administrator II, Technology Services Division.
- Assessment Design: Performance Tasks—Dr. Linda Hooper, Education Research and Evaluation Consultant High School and Physical Fitness Assessment.
- Assessment Design: Item Development – Jessica Valdez, National Assessment of Educational Progress (NAEP) State Coordinator
- Assessment Design: Test Design—Jamie Contreras, Education Research and Evaluation Consultant Standardized Testing and Reporting (STAR) Program and Assessment Transition Office.
- Formative Processes and Tools/Professional Development Support—Laura Watson, Education Research and Evaluation Consultant High School and Physical Fitness Assessment.
- Accessibility and Accommodations/English Learners—Noelia Ramirez, Education Research and Evaluation Consultant Standards-based Tests.
- Architecture Review Board Members
- Rodney Okamoto, Data Processing Manager III, Technology Services Division from the California Department of Education.
- John Fleischman, Assistant Superintendent of Technology Services, Sacramento County Office of Education
- Bonnie Edwards, Senior Information Systems Analyst, California Community Colleges Chancellor’s Office
- Sue Burr, Executive Director, State Board of Education (SBE); member on the Sustainability Task Force
- Jose Ortega, Administrator, Educational Data Management Division; SMARTER Balanced Assessment Consortium (SBAC) California Department of Education (CDE) Technology Readiness Coordinator
- Higher Education Leads
- Dr. Monica Lin, Associate Director, Undergraduate Admissions at University of California, Office of the President
- Dr. Linda Michalowski, Vice Chancellor for Student Services and Special Programs for the California Community Colleges Chancellor's Office
- Dr. Beverly Young, Assistant Vice Chancellor, California State University.
- Executive Committee
- Dr. Beverly L. Young, Assistant Vice Chancellor of Academic Affairs for the California State University System, was elected by the SMARTER Balanced Assessment Consortium (SBAC) higher education state leads to serve on SBAC’s Executive Committee.
- Deb Sigman, Deputy Superintendent, District, School, and Innovation Branch
- Technical Advisory Committee
- Linda Darling-Hammond, Ph.D., Professor of Education at the Stanford University School of Education
- Jamal Abedi, Ph.D. Professor in the School of Education at the University of California, Davis
- Edward Haertel, Ph.D. Jacks Family Professor of Education at the Stanford University School of Education.
- W. James Popham, Ph.D. Professor Emeritus, Graduate School of Education and Information Studies at the University of California, Los Angeles (UCLA)
- English Language Learners Advisory Committee
- Jamal Abedi, Ph.D. Professor in the School of Education at the University of California, Davis
- Richard Durán, Ph.D. professor at the Gevirtz Graduate School of Education at University of California at Santa Barbara
- Kenji Hakuta, Ph.D. professor of education at Stanford University
- Robert Linquanti, Project director for English Learner Evaluation and Accountability Support (ELEAS) and senior researcher for the California Comprehensive Center at WestEd
- Maria Santos, Deputy Superintendent of Instruction, Leadership, and Equity-in-action for the Oakland Unified School District in California
- Guadalupe Valdes, Ph.D., Professor of Education at Stanford University
Pilot Test
- Pilot Test in early 2013
- More than 10,000 items and performance tasks
- Open to all schools in the Consortium
- Two million students
- 22% of students from each Governing State
SMARTER Balanced Assessment Consortium (SBAC) Resources
- California Department of Education/SBAC presentations can be found on the California Department of Education SMARTER Balanced Assessment Consortium Web page.
- Join the SMARTER Balanced Assessment Consortium California Department of Education electronic mailing list by sending a blank e-mail to: subscribe-sbac@mlist.cde.ca.gov
Assessments To Consider (by content area and grade level)
SMARTER Balanced Assessment Consortium (SBAC) Assessments will assess grades 3 through 8 and grade 11 in English language-arts and Mathematics. Assessments for Kindergarten through grade 2, grades 9, 10, and 12 are not included in the SMARTER Balanced Assessment Consortium (SBAC) assessments. Science and History-Social Science also are not assessed under the SMARTER Balanced Assessment Consortium (SBAC).
Statewide Assessment Reauthorization Timeline
- March 2012:
- March 21 and 22 Work Group Meeting
- April 2012:
- April 17 and 18: Work Group Meeting
- Regional Public Meetings: 5 meetings between March and April
- E-mail Account opened
- May 2012:
- May 22 and 23 Work Group Meeting
- June 2012:
- June 12 and 14: Work Group Meeting
- July 2012:
- Launch of Online Survey, July 5
- July 25 and 26 Work Group Meeting
- August 2012:
- Focus Groups: July to September
- Report writing and review
- Online Survey closed September 4
- September 2012:
- September 6: Work Group Meeting
- E-mail account closed
- October 2012:
- Report Writing and Review
- November 2012
- State Board of Education Meeting
- Recommendations to the Legislature in fall 2012
Common Developing Themes
- Diagnostic assessments: English language arts and Math in grade 2 and grades 3 through 12 as needed
- Formative Assessment Tools:
- SMARTER Balanced Assessment Consortium (SBAC): English language-arts and Math
- Explore the state providing formative tools and processes for science and history/social-science
- Interim assessments:
- SMARTER Balanced Assessment Consortium (SBAC): English language arts and Math Grades 3 through 8, 11
- English language arts and Math for all students in grade 2, 9 through 10
- Science/history social science – state provided computerized interim assessments similar to summative
- Summative:
- SMARTER Balanced Assessment Consortium (SBAC) assessments in grades 3 through 8 and 11 (English language arts and Math).
- Summative assessments for science and history-social science in grades 3 through 12, for English language arts and math in grade 12, and writing in grades 3 through 12
- Matrix sampling in some grades/subjects, specifically science and history–social science (HSS)
- Reduce linguistic complexity of assessments to more accurately measure what students know and can do
- Items that evaluate critical thinking, problem solving, communication, collaboration, creativity, and innovation
- Other measures: student engagement survey, parent survey, graduation rates, attendance
- Multiple measures: within classroom, (e.g., tests, quizzes, projects, essays, journaling, class work, portfolios)
- Using results to satisfy the high school graduation requirement
SMARTER Balanced Assessment Consortium (SBAC) Timeline
- 2011–2012 School Year
- Content and Item Specifications Development
- Technology readiness tool available
- Formative Processes, Tools, and Practices Development Begins
- Writing and Review of Pilot Items/Tasks (including Cognitive Labs and Small-Scale Trials)
- 20122013 School Year
- Writing and Review Items/Tasks for Field Testing (throughout the school year)
- Teams of teachers evaluate formative assessment practices and curriculum resources
- Pilot Testing of Summative and Interim Items/Tasks Conducted
- 2013–2014 School Year
- Formative tools available to teachers
- Field testing of summative assessment, training school- and district-level staff in formative tools
- 2014–2015 School Year
- Full implementation of assessment system
Contact Information
Patrick Traynor, Director
Assessment Development and Administration Division
E-mail: ptraynor@cde.ca.gov
Jessica Barr, Reauthorization Lead Consultant
Assessment Development and Administration Division
Phone: 916-319-0364
E-mail: jbarr@cde.ca.gov
Kristen Brown, SMARTER Balanced Assessment Consortium (SBAC) Lead Consultant
Assessment Development and Administration Division
Phone: 916-319-0334
E-mail: kbrown@cde.ca.gov
reauthorization@cde.ca.gov