Skip to content
Printer-friendly version

Evaluating NAEP and CST Results

Evaluating State-Level Scores from NAEP and the CSTs: Why Both Sets of Results Should Be Considered But Not Compared.

To monitor the academic achievement of California students, many people turn to results from state and national assessments, namely the California Standards Tests (CSTs) and the National Assessment of Educational Progress (NAEP). Both assessments provide valid information about student achievement, but differences in the purpose, content, format, and scoring and reporting of these assessments make direct comparisons of results inappropriate. This document offers a glimpse at those differences.

OVERVIEW
NAEP CSTs

NAEP is the only nationally representative and continuing assessment of what America’s students know and can do in various subjects including reading, writing, mathematics, and science. NAEP is administered by the United States Department of Education, with oversight and direction from the National Assessment Governing Board.

Note: NAEP is a broad program that consists of the main NAEP assessment, the long-term trend assessment, and a number of special studies. Because main NAEP is the primary component that provides state-level results, it is the focus of comparison in this document.

 The CSTs are the primary component of California’s Standardized Testing and Reporting (STAR) Program. The CSTs are developed exclusively for California’s public schools and are designed to measure how well students in grades two through eleven are achieving the state content standards, adopted by the State Board of Education (SBE), for English-language arts, mathematics, history-social science, and science. The CSTs are administered by the California Department of Education (CDE), with policy direction from the SBE.
PURPOSE

NAEP and the CSTs serve different primary purposes:

NAEP CSTs
  • Originally established solely as a national measure of student achievement; only later offered state-level results to measure the progress of individual states.
  • No accountability stakes for states, districts, schools,   or students.
  • Developed specifically to measure how well California’s students are achieving California’s content standards.
  • Results used for state and federal accountability purposes.
CONTENT

Although there is overlap in the content tested by NAEP and the CSTs, the two assessments are developed independently. As a result, there are a number of distinct and important differences in the selection, organization, and weighting of standards and objectives for each assessment.

NAEP CSTs
  • Content is based on assessment frameworks developed and adopted by the National Assessment Governing Board with input from teachers, curriculum specialists, school administrators, parents, and members of the general public. Each framework covers a broad range of assessment objectives.
  • Assesses many different subjects, but reports state-level results only for grades four and eight reading, writing, mathematics, and science.
  • Content is based on test blueprints which outline the specific California content standards assessed by grade and subject. The test blueprints are developed through a consultative process with teachers, curriculum specialists, school administrators, and representatives of postsecondary education.
  • Report state-level results for mathematics, English-language arts (i.e., reading and writing), science, and history-social science.

Clear distinctions between the assessments can be seen in the allocation of test questions by content area. A further distinction exists in the English-language arts assessments, as the CST for English-language arts encompasses reading and writing within the same test, while NAEP assesses reading separately from writing.

FORMAT

NAEP and the CSTs differ in their overall designs, including question and test formats, as well as administration logistics and target student populations.

NAEP CSTs
  • NAEP tests a sample of students in grades four,   eight, and twelve in randomly selected schools.
  • Uses a variety of question formats including multiple-choice, short and extended response, performance tasks, and hands-on experiments.
  • Uses a matrix test design in which each student is tested in only one subject and takes only a portion of the assessment for that subject.
  • NAEP assessments are timed. Students have 50 minutes to complete the test for most subjects.
  • Administered by trained NAEP field staff not associated with the school or district.
  • The CSTs test all students in grades two through eleven, with few exceptions.
  • Use only multiple-choice question format, except for the writing tasks in grades four and seven.
  • Each student takes the entire assessment.
  • The CSTs are untimed tests, although most students complete each test for each subject within approximately 150 minutes.
  • Administered by trained school staff.

SCORING AND REPORTING

Differences between NAEP and the CSTs are evident when examining state-level scores from the same year, grade, and subject. There are a variety of reasons why NAEP and CST scores are not directly comparable.

NAEP CSTs
  • State-level scores are reported for grades four, eight, and twelve.
  • Scores are reported on a scale of 0 to 500 for reading and mathematics and a scale of 0 to 300 for writing and science.
  • Achievement levels are: Advanced, Proficient, Basic, and Below Basic.*
  • Results released annually for the nation and bi-annually for all states and selected large urban school districts; no results released for individual schools or students.
  • Scores allow for valid state-to-state and state-to-nation comparisons.
  • State-level scores are estimates based on a sample of students tested, so sampling error must be considered when interpreting results and small differences in scores may not be significantly different.
  • State-level scores are reported for grades two through eleven.
  • Scores are reported on a scale of 150 to 600 for all subjects tested.
  • Achievement levels are: Advanced, Proficient, Basic, Below Basic, and Far Below Basic.*
  • Results released annually for the state as well as every county, district, school, and student.
  • Scores allow for valid comparisons between and within students, schools, districts, counties, and the state.
  • State-level scores are based on the entire population of students tested, so sampling error is not a factor.

*  Although NAEP and the CSTs have similar achievement level labels, there are fundamental differences in how their achievement levels are defined. As a result of the differences in criteria for each achievement level, marked differences in the percentage of students scoring at each level are not uncommon.                                                                                            

This document provides an introduction to the unique perspectives of California students’ academic progress that NAEP and the CSTs each provide. Results from both assessments can be considered in drawing conclusions about student achievement, but it is important to understand how the assessments differ. More information about the NAEP assessments and special studies is available on the United States Department of Education NAEP Web page (Outside Source). More information about the CSTs and other STAR Program tests is available on the CDE STAR Web page. [http://www.cde.ca.gov/ta/tg/sr/]

Questions:   Jessica Valdez | jvaldez@cde.ca.gov | 916-319-0332
Download Free Readers