2004 API Base Technical Information

On November 9, 2004, the State Board of Education (SBE) adopted the method for calculating the 2004 Academic Performance Index (API) Base. The 2004 API Base reports are scheduled to be released in March 2005. The following changes to the API will occur, beginning with the 2004 API Base:

• Results of the California Standards Test in science (CST in Science), grade five, and in history-social science ( CST in HSS), grade eight, will be added.

• Norm referenced test (NRT) results will be included only at grades three and seven.

• Because the tests included in the API will change, content area (indicator) weights will change.

• Because the tests included in the API will no longer have results at every grade level, a new and simpler way of calculating the API will be used. The calculations will take into account that students at some grade levels are tested in more content areas and/or with different tests. The new calculation will also easily accommodate schools with different grade span configurations. This calculation change will result in 2004 API Base scores that will better reflect the combined accomplishments of all of the students at a school and, therefore, will be a more accurate measure of the schools' academic accomplishments.

The change to the new calculations will result in content area weights that may be slightly different for each school. Each school's content area weights will be determined based on (1) test weights established by the SBE and (2) the number of valid test scores in each content area and grade level at a school.  

The purpose of this document is to describe the SBE decisions regarding calculation of the 2004 API Base. The document, organized into six sections, describes:

• Issues to be resolved for the 2004 API Base

• Changes in the Standardized Testing and Reporting (STAR) program

• API revisions to accommodate changes in the STAR program

• Advantages of the revised API method

• Examples of 2004 API Base calculations

• School content area weights for the most common grade spans

Issues to be Resolved for the 2004 API Base

The Public Schools Accountability Act (PSAA) of 1999 requires the State Superintendent of Public Instruction (SSPI), with approval of the SBE, to develop an API to measure the performance of schools. The law also provides for an Advisory Committee to assist the SSPI and the SBE in the creation of the index. The Committee established a Technical Design Group (TDG), comprised of educational measurement specialists, to provide guidance on technical issues. The TDG met during the spring and summer of 2004 and identified issues to be resolved in determining the 2004 API Base.

The first issue in deciding the calculations for the 2004 API Base is that test results for most schools will reflect an uneven distribution across content areas and tests given (as illustrated in Table 1 below). Because different content areas are tested and/or additional tests are given at different grade levels, the biggest challenge was to establish a methodology for calculating the API that would equitably represent the contributions of all of the students at a school.  

The second issue is the fact that grade spans of California schools vary widely from school to school. For example, with the addition of the CST in Science at grade five only and the CST in HSS at grade eight only, elementary schools without grade five or middle schools without grade eight would need separate API calculation adjustments if the current API method were used. This issue has continued to be a challenge in API calculations since 2002, when the CAHSEE and the CST HSS, grades ten and eleven, were added to the 2002 API Base.

The third issue is the increasing need for adjustments in API calculations due to missing test data for a school. In order to simplify API calculations and make them less complicated, a new method was needed that would require fewer adjustments.  

Changes in the STAR Program

The issues to be resolved in determining the 2004 API Base also involved the changes that have occurred and will occur in the STAR program.

Results of CST in Science (Grade Five) and CST in HSS (Grade Eight)

The CST in Science, grade five, is the first of three universally administered science tests developed to meet the requirements of the NCLB. The CST in Science, grade five, became operational in 2004 and is a cumulative test based on state academic standards in science for grades four and five. All grade five students are expected to participate in this assessment unless otherwise exempted.  

The CST in HSS, grade eight, is a cumulative test based on state academic standards in history-social science for grades six, seven, and eight. All grade eight students are expected to participate in this assessment unless otherwise exempted.  

Consistent with The API: A Six-Year Plan for Development (2001-2006) (PDF; 39KB; 13pp.) the CST in Science, grade five, and CST in HSS, grade eight, will be added to the API beginning with the 2004 API Base.  

Only Grades Three and Seven NRT Results

California's current statewide NRT is the California Achievement Test, Sixth Edition Survey (CAT/6 Survey). In August 2004, Senate Bill 1448 was enacted, reauthorizing the STAR program. This bill also specified a change in the administration of California's NRT under the STAR program. Beginning with the 2005 test administration, only students in grades three and seven will be administered the NRT. (In previous years, students in grades two through eleven were administered the NRT.) The reduction in the number of grade levels with test results means that the 2004 API Base indicators, weights, and calculation need to match the 2005 API Growth that will use NRT test results in grades three and seven only. In keeping with past practice, only tests given in the growth year will be included in the base year API.  

Table 1 shows the assessments to be used in the 2004 API Base as a result of changes in the STAR program. This table shows all assessments given at the same grades in both 2004 and 2005.

Table 1
Assessments to be Used in the 2004 API Base

STAR = Standardized Testing and Reporting program
CAHSEE = California High School Exit Examination
ELA = English-language arts
HSS = History-social science

API Revisions to Accommodate Changes in the STAR Program

The TDG developed an issue paper about API revisions needed to accommodate the changes in the STAR Program. The paper described key issues to be resolved in determining the 2004 API Base, alternatives to address those issues, and recommendations for calculating the 2004 API Base. The PSAA Advisory Committee met in September 2004, discussed the TDG's paper, and concurred with the TDG recommendations. In November 2004, the SBE discussed the paper and concurred with the recommendations of the PSAA Advisory Committee and TDG. The SBE's decisions are described in this section.

New API Calculation Method

To meet requirements of the PSAA, new indicators are added to the API Base each year as additional test results become available. The new indicators, along with the existing API indicators, establish the baseline components for an API reporting cycle. As new indicators are incorporated into the API, changes to the API calculations occur. The SBE adopts the API Base calculations for each reporting cycle.  

In considering the 2004 API Base, the SBE recognized the importance of maintaining consistency in the API and minimizing any changes in the API calculations. However, it also recognized the need for maintaining the statistical soundness of the API given current and future changes to the state's testing system. After considering three alternatives for 2004 API Base calculations, the SBE adopted a new method of calculating the API. (The three alternatives considered are described in detail in agenda item #3 for the SBE November 9, 2004 meeting http://www.cde.ca.gov/be/ag/ag/yr04/agenda1104.asp.)

The new method adopted by the SBE calculates the API as the weighted average of student scores across content areas and tests results within the school. The following five steps are used to calculate the 2004 API Base (see Tables 5A, 6A, and 7A below for examples of the calculation):

1. Apply performance level weighting factors (i.e., 1000, 875, 700, 500, or 200) and inclusion/exclusion rules to student results to be used in the API (same as in prior API calculations).
2. Multiply each student score by the test weights established by the SBE.
3. Sum the products of #2.
4. Sum the test weights applied to each student score.
5. Divide #3 by #4 to produce the school's API.    

Test Weights

Once the SBE adopted the new methodology, it was necessary to determine the weights that should be assigned to each tested content area used in the API. These weights, which are applied at the student level, are referred to as "test weights." The SBE recognized that the question of the appropriate test weights is a policy issue rather than a technical issue, and the SBE members adopted test weights that they believed reflected the curriculum priorities in California public education.

Test weights are different from the API indicator weights used in prior API calculations because they are applied at the individual student test level rather than at the school level. The test weights are shown as decimals rather than percentages to distinguish them as student test level weights. The test weights set for the API Base will be the same for the API Growth within an API reporting cycle. The SBE adopted separate test weights for grades two through eight and for grades nine through eleven.        

Grades Two Through Eight

For grades two through eight, the SBE adopted test weights shown in table 2.

Table 2
2004 API Base: Test Weights, 2-8 Grade Levels

Note: The test weights shown in this table do not reflect the content area weights for a
school, which will vary based upon the number of valid test scores in each content area.
Test weights do not total 1.00.

The SBE adopted relatively low test weights of .20 for the CST in Science, grade five, and .20 for the CST in HSS, grade eight. The SBE acknowledged that elementary and middle schools and their staffs will need sufficient time and resources to focus greater emphasis in the areas of science and history-social science. In addition, the instructional materials adoption for these content areas is not scheduled until 2006. Comparatively low test weights, therefore, were adopted for the 2004 API Base to provide adequate start up time for schools and districts over the next several years. The SBE will reevaluate the weights after instructional materials are adopted.

Grades Nine Through Eleven

The SBE also adopted test weights for grades nine through eleven as shown in table 3.  

Table 3
2004 API Base: Test Weights, 9-11 Grade Levels

Note: The test weights shown in this table do not reflect the content area weights for a
school, which will vary based upon the number of valid test scores in each content area.
Test weights do not total 1.00.

Content Area Weights for Each School

Content area weights at the school level will be unique to each school, based on the test weights established by the SBE, the school's grade span configuration, and the number of valid test scores in each content area for the school. A school's content area weights are not needed in calculating the 2004 API Base, but they will be provided on the API reports for information only.  

Content area weights differ from test weights because they are school level weights (rather than student test level weights), and they are not the same for all schools. In addition, although the test weights established by the SBE remain the same within an API reporting cycle, a school's unique content area weights within a reporting cycle may be slightly different for the API Base and Growth (e.g., 2004 API Base and 2005 API Growth). The amount of difference will depend on the amount of variation in the counts and grade levels of test takers in the base year (e.g., 2004) and the growth year (e.g., 2005) at the school.

Examples of content area weights are provided in the last section of this document.

Advantages of the Revised API Method

The SBE recognized several advantages of the new calculation method:

• This method takes into account the fact that students at some grade levels are tested in more content areas and/or with different tests (i.e., CST in Science at grade five, CST in HSS, CAHSEE , and NRT). Test results of these students provide more information about the school's achievement than results of students at other grade levels who take fewer tests. Under this method, more weight is attributed to a content area and/or test when more students are tested.

• This method better reflects the combined accomplishments of the students at a school.

• Schools with different grade spans will have APIs that more accurately represent the schools' performances and the contribution of all content areas and tests at each grade level.

•  This method is more sensitive to the number of students taking tests in each content area and/or taking additional tests and avoids instability in the API due to low numbers of students taking certain tests.

•  This method provides a more balanced way of including results of content areas/tests that are not assessed at all grade levels.

• This method easily accommodates the case where a school has valid scores on some but not all tests. (Currently, each school with missing scores needs special data adjustments.)

• This method offers the greatest flexibility in accommodating future changes to the API (such as the addition or deletion of tests or content areas).

• This method, when all students at a school take all tests used in the API, will result in the same API score as the current method.

Examples of the 2004 API Base Calculations

The new method for calculating the 2004 API Base is shown in Tables 5A, 5B, 6A, 6B, 7A, and 7B.These tables are located at the end of this section. For explanatory purposes, each example is based on a very small number of student test results. Tables 5A and 5B are examples of a school (kindergarten through grade five) with four students; Tables 6A and 6B are examples of a school (grades six through eight) with three students; and Tables 7A and 7B are examples of a school (nine through eleven) with three students. These small counts were necessary for illustrative purposes to show the API calculation process. Larger counts of test results would be difficult to display fully on one page. For this reason, the examples are based on very small numbers of student test results for each school.

These examples exclude the scale calibration factors (SCFs) used in calculating APIs.   SCFs are used to maintain consistency in the statewide API average from one API reporting cycle to the next. As a result, the statewide average API Growth and same year API Base match exactly (e.g., 2004 API Growth and 2004 API Base). For the 2004 API Base, the SCFs will be calculated and included in the final calculations. Specification of the SCFs will be provided in the 2004 API Base Information Guide, which will be posted on the CDE Web site prior to the release of the 2004 API Base reports in March 2005.

If all students at a school take all tests, have valid scores, and there are no missing data, then a school's 2004 API Base would result in the same score using the current method or the new method of calculation . If some students at a school do not take one or more tests, the 2004 API Base score would be slightly different using each calculation method.

Inclusion/exclusion rules will be applied to a school's test results used in the API in the same way as in prior API calculations. For example, a grade ten student record showing no CST Math score would have a performance level weighting factor of 200 applied for API calculations.

The same methodology used for calculating the API for schools will be used to calculate the APIs for subgroups, school districts, and county offices of education.

Simple Calculation (Tables 5A, 6A, and 7A)

Tables 5A to 7B, two ways of calculating the API are shown for each school. The first way shows the simplest method of calculating the API (Tables 5A, 6A, and 7A). In these tables, the test weights adopted by the SBE (Column A) are multiplied by the student's individual scores that have been weighted for API calculations using API performance level weighting factors (Column B).  

API performance level weighting factors are applied for Column B as shown in Table 4.

Table 4
API Performance Level Weighting Factors

Note: These performance level weighting factors are the same as those applied
in the current API calculation methodology.  

For example, a student who earns an Advanced score on the CST in ELA would show a 1000 score in Column B in Tables 5A, 6A, and 7A. In these tables, the product of Column A and Column B is shown in Column C. The results of Columns A and C are summed; then the sum of Column C is divided by the sum of Column A to produce the school's API. This is the method that the CDE will use to calculate the 2004 API Base.

Calculation Showing School's Content Area Weights (Tables 5B, 6B, and 7B)

The second way of calculating the API includes the determination of the school's content area weights (Tables 5B, 6B, and 7B). These tables are shown for comparison purposes only and are not required to determine a school's API. Each school's content area weights are displayed in Column E. The API results of these tables equal those of the simple method shown in Tables 5A, 6A, and 7A. For example, the API of 876 shown in Table 5A is the same score result of 876 shown in Table 5B.

Examples of 2004 API Base Calculation - Tables 5A and 5B (PDF; 210KB; 1p.)

Examples of 2004 API Base Calculation - Tables 6A and 6B (PDF; 210KB; 1p.)

Examples of 2004 API Base Calculation - Tables 7A and 7B (PDF; 197KB; 1p.)

School Content Area Weights for the Most Common Grade Spans

Table 8 shows the school level content area weights for common grade spans, assuming there are an equal number of valid scores at each grade level and no missing data. (The determination of a school's content area weights is illustrated in Column E of Tables 5B, 6B, and 7B.) If some students at a school do not take one or more tests, the indicator weights would be slightly different than those shown below.

School Content Area Weights for the Most Common Grade Spans - Table 8 (PDF; 203KB; 1p.)