• Home »
• Testing & Accountability »
• Accountability »
• Academic Performance Index (API)

2004 API Base Technical Information

On November 9, 2004, the State Board of Education (SBE) adopted the method for calculating the 2004 Academic Performance Index (API) Base. The 2004 API Base reports are scheduled to be released in March 2005. The following changes to the API will occur, beginning with the 2004 API Base:

• Results of the California Standards Test in science (CST in Science), grade five, and in history-social science ( CST in HSS), grade eight, will be added.

• Norm referenced test (NRT) results will be included only at grades three and seven.

• Because the tests included in the API will change, content area (indicator) weights will change.

• Because the tests included in the API will no longer have results at every grade level, a new and simpler way of calculating the API will be used. The calculations will take into account that students at some grade levels are tested in more content areas and/or with different tests. The new calculation will also easily accommodate schools with different grade span configurations. This calculation change will result in 2004 API Base scores that will better reflect the combined accomplishments of all of the students at a school and, therefore, will be a more accurate measure of the schools' academic accomplishments.

The change to the new calculations will result in content area weights that may be slightly different for each school. Each school's content area weights will be determined based on (1) test weights established by the SBE and (2) the number of valid test scores in each content area and grade level at a school.  

The purpose of this document is to describe the SBE decisions regarding calculation of the 2004 API Base. The document, organized into six sections, describes:

• Issues to be resolved for the 2004 API Base

• Changes in the Standardized Testing and Reporting (STAR) Program

• API revisions to accommodate changes in the STAR Program

• Advantages of the revised API method

• 2004 API Base calculations

• School content area weights for the most common grade spans

Issues to be Resolved for the 2004 API Base

The Public Schools Accountability Act (PSAA) of 1999 requires the State Superintendent of Public Instruction (SSPI), with approval of the SBE, to develop an API to measure the performance of schools. The law also provides for an Advisory Committee to assist the SSPI and the SBE in the creation of the index. The Committee established a Technical Design Group (TDG), comprised of educational measurement specialists, to provide guidance on technical issues. The TDG met during the spring and summer of 2004 and identified issues to be resolved in determining the 2004 API Base.

The first issue in deciding the calculations for the 2004 API Base is that test results for most schools will reflect an uneven distribution across content areas and tests given (as illustrated in Table 1 below). Because different content areas are tested and/or additional tests are given at different grade levels, the biggest challenge was to establish a methodology for calculating the API that would equitably represent the contributions of all of the students at a school.  

The second issue is the fact that grade spans of California schools vary widely from school to school. For example, with the addition of the CST in Science at grade five only and the CST in HSS at grade eight only, elementary schools without grade five or middle schools without grade eight would need separate API calculation adjustments if the current API method were used. This issue has continued to be a challenge in API calculations since 2002, when the CAHSEE and the CST HSS, grades ten and eleven, were added to the 2002 API Base.

The third issue is the increasing need for adjustments in API calculations due to missing test data for a school. In order to simplify API calculations and make them less complicated, a new method was needed that would require fewer adjustments.  

Changes in the STAR Program

The issues to be resolved in determining the 2004 API Base also involved the changes that have occurred and will occur in the STAR program.

Results of CST in Science (Grade Five) and CST in HSS (Grade Eight)

The CST in Science, grade five, is the first of three universally administered science tests developed to meet the requirements of the NCLB. The CST in Science, grade five, became operational in 2004 and is a cumulative test based on state academic standards in science for grades four and five. All grade five students are expected to participate in this assessment unless otherwise exempted.  

The CST in HSS, grade eight, is a cumulative test based on state academic standards in history-social science for grades six, seven, and eight. All grade eight students are expected to participate in this assessment unless otherwise exempted.  

Consistent with The API: A Six-Year Plan for Development (2001–2006) the CST in Science, grade five, and CST in HSS, grade eight, will be added to the API beginning with the 2004 API Base.  

Only Grades Three and Seven NRT Results

California's current statewide NRT is the California Achievement Test, Sixth Edition Survey (CAT/6 Survey). In August 2004, Senate Bill 1448 was enacted, reauthorizing the STAR Program. This bill also specified a change in the administration of California's NRT under the STAR Program. Beginning with the 2005 test administration, only students in grades three and seven will be administered the NRT. (In previous years, students in grades two through eleven were administered the NRT.) The reduction in the number of grade levels with test results means that the 2004 API Base indicators, weights, and calculation need to match the 2005 API Growth that will use NRT test results in grades three and seven only. In keeping with past practice, only tests given in the growth year will be included in the base year API.  

Table 1 shows the assessments to be used in the 2004 API Base as a result of changes in the STAR Program. This table shows all assessments given at the same grades in both 2004 and 2005.

Table 1
Assessments to be Used in the 2004 API Base

STAR = Standardized Testing and Reporting Program
CAHSEE = California High School Exit Examination
ELA = English-language arts
HSS = History-social science

API Revisions to Accommodate Changes in the STAR Program

The TDG developed an issue paper about API revisions needed to accommodate the changes in the STAR Program. The paper described key issues to be resolved in determining the 2004 API Base, alternatives to address those issues, and recommendations for calculating the 2004 API Base. The PSAA Advisory Committee met in September 2004, discussed the TDG's paper, and concurred with the TDG recommendations. In November 2004, the SBE discussed the paper and concurred with the recommendations of the PSAA Advisory Committee and TDG. The SBE's decisions are described in this section.

New API Calculation Method

To meet requirements of the PSAA, new indicators are added to the API Base each year as additional test results become available. The new indicators, along with the existing API indicators, establish the baseline components for an API reporting cycle. As new indicators are incorporated into the API, changes to the API calculations occur. The SBE adopts the API Base calculations for each reporting cycle.  

In considering the 2004 API Base, the SBE recognized the importance of maintaining consistency in the API and minimizing any changes in the API calculations. However, it also recognized the need for maintaining the statistical soundness of the API given current and future changes to the state's testing system. After considering three alternatives for 2004 API Base calculations, the SBE adopted a new method of calculating the API. (The three alternatives considered are described in detail in agenda item #3 for the SBE November 9, 2004 meeting http://www.cde.ca.gov/be/ag/ag/yr04/agenda1104.asp.)

The new method adopted by the SBE calculates the API as the weighted average of student scores across content areas and tests results within the school.

Test Weights

Once the SBE adopted the new methodology, it was necessary to determine the weights that should be assigned to each tested content area used in the API. These weights, which are applied at the student level, are referred to as "test weights." The SBE recognized that the question of the appropriate test weights is a policy issue rather than a technical issue, and the SBE members adopted test weights that they believed reflected the curriculum priorities in California public education.

Test weights are different from the API indicator weights used in prior API calculations because they are applied at the individual student test level rather than at the school level. The test weights are shown as decimals rather than percentages to distinguish them as student test level weights. The test weights set for the API Base will be the same for the API Growth within an API reporting cycle. The SBE adopted separate test weights for grades two through eight and for grades nine through eleven.        

Grades Two Through Eight

For grades two through eight, the SBE adopted test weights shown in table 2.

Table 2
2004 API Base: Test Weights, 2-8 Grade Levels

Note: The test weights shown in this table do not reflect the content area weights for a
school, which will vary based upon the number of valid test scores in each content area.
Test weights do not total 1.00.

The SBE adopted relatively low test weights of .20 for the CST in Science, grade five, and .20 for the CST in HSS, grade eight. The SBE acknowledged that elementary and middle schools and their staffs will need sufficient time and resources to focus greater emphasis in the areas of science and history-social science. In addition, the instructional materials adoption for these content areas is not scheduled until 2006. Comparatively low test weights, therefore, were adopted for the 2004 API Base to provide adequate start up time for schools and districts over the next several years. The SBE will reevaluate the weights after instructional materials are adopted.

Grades Nine Through Eleven

The SBE also adopted test weights for grades nine through eleven as shown in table 3.  

Table 3
2004 API Base: Test Weights, 9-11 Grade Levels

Note: The test weights shown in this table do not reflect the content area weights for a
school, which will vary based upon the number of valid test scores in each content area.
Test weights do not total 1.00.

Content Area Weights for Each School

Content area weights at the school level will be unique to each school, based on the test weights established by the SBE, the school's grade span configuration, and the number of valid test scores in each content area for the school. A school's content area weights are not needed in calculating the 2004 API Base, but they will be provided on the API reports for information only.  

Content area weights differ from test weights because they are school level weights (rather than student test level weights), and they are not the same for all schools. In addition, although the test weights established by the SBE remain the same within an API reporting cycle, a school's unique content area weights within a reporting cycle may be slightly different for the API Base and Growth (e.g., 2004 API Base and 2005 API Growth). The amount of difference will depend on the amount of variation in the counts and grade levels of test takers in the base year (e.g., 2004) and the growth year (e.g., 2005) at the school.

Examples of content area weights are provided in the last section of this document.

Advantages of the Revised API Method

The SBE recognized several advantages of the new calculation method:

• This method takes into account the fact that students at some grade levels are tested in more content areas and/or with different tests (i.e., CST in Science at grade five, CST in HSS, CAHSEE , and NRT). Test results of these students provide more information about the school's achievement than results of students at other grade levels who take fewer tests. Under this method, more weight is attributed to a content area and/or test when more students are tested.

• This method better reflects the combined accomplishments of the students at a school.
  
  
• Schools with different grade spans will have APIs that more accurately represent the schools' performances and the contribution of all content areas and tests at each grade level.

• This method is more sensitive to the number of students taking tests in each content area and/or taking additional tests and avoids instability in the API due to low numbers of students taking certain tests.

• This method provides a more balanced way of including results of content areas/tests that are not assessed at all grade levels.

• This method easily accommodates the case where a school has valid scores on some but not all tests. (Currently, each school with missing scores needs special data adjustments.)

• This method offers the greatest flexibility in accommodating future changes to the API (such as the addition or deletion of tests or content areas).

• This method, when all students at a school take all tests used in the API, will result in the same API score as the current method.

2004 API Base Calculations

The scale calibration factors (SCFs) are used to maintain consistency in the statewide API average from one API reporting cycle to the next. As a result, the statewide average API Growth and same year API Base match exactly (e.g., 2004 API Growth and 2004 API Base). For the 2004 API Base, the SCFs will be calculated and included in the final calculations. Specification of the SCFs will be provided in the 2004 API Base Information Guide, which will be posted on the CDE Web site prior to the release of the 2004 API Base reports in March 2005.

If all students at a school take all tests, have valid scores, and there are no missing data, then a school's 2004 API Base would result in the same score using the current method or the new method of calculation . If some students at a school do not take one or more tests, the 2004 API Base score would be slightly different using each calculation method.

Inclusion/exclusion rules will be applied to a school's test results used in the API in the same way as in prior API calculations. For example, a grade ten student record showing no CST Math score would have a performance level weighting factor of 200 applied for API calculations.

The same methodology used for calculating the API for schools will be used to calculate the APIs for subgroups, school districts, and county offices of education.

API performance level weighting factors are shown in Table 4.

Table 4
API Performance Level Weighting Factors

Note: These performance level weighting factors are the same as those applied
in the current API calculation methodology.  

School Content Area Weights for the Most Common Grade Spans

Table 8 shows the school level content area weights for common grade spans, assuming there are an equal number of valid scores at each grade level and no missing data. If some students at a school do not take one or more tests, the indicator weights would be slightly different than those shown below.

Table 8
2004 API Base: School Content Area Weights for the Most Common Grade Spans

*Most common grade spans statewide.