These Explanatory Notes are designed to assist educators and other interested parties in interpreting the 2002 Academic Performance Index Base Report. The Explanatory Notes provide details with respect to the Academic Performance Index (API) and growth target calculations beyond the explanations and footnotes that appear on the report.
The Public Schools Accountability Act
The API is the centerpiece of the statewide accountability system in California public education. The Public Schools Accountability Act (PSAA) of 1999 (Chapter 3, Statutes of 1999) requires that the California Department of Education (CDE) annually calculate APIs for California public schools, including charter schools, and publish school rankings based on these APIs. The PSAA also requires the establishment of a minimum five-percent annual API growth target for each school as well as an overall statewide API performance target for all schools. A school that meets API growth targets may be eligible for awards under the following programs:
- The Governor's Performance Award Program
- The Certificated Staff Performance Incentive Act (Chapter 52, Statutes of 1999)
- Adopted a 1999 base-year API
- Defined the five-percent annual API growth target
- Established an interim statewide API performance target of 800
API Reporting Cycle
The SBE's actions cleared the way for the publication of the 1999 API Base Report in January 2000. Each annual API reporting cycle includes two reports: a base report, which appears after the first of the calendar year, and a growth report, which appears after school starts in the fall. This pair of reports is based on APIs calculated in exactly the same fashion with the same indicators but using test results from two different years. The 2001–2002 API Growth Report represents the completion of the third API reporting cycle.
The Development of the API
The 2002 Base API adds a number of new components:
- The California Standards Test in Mathematics for grades 2-11
- The California Standards Test in History/Social Science for grades 10-11
- The California High School Exit Exam (CAHSEE) for high schools
In June 2002 the SBE approved a methodology for integrating these new components into the existing API, which had previously included results from the Stanford 9 norm-referenced assessment and the California Standards Test in English-Language Arts (CST ELA) of the Standardized Testing and Reporting (STAR) Program.
Subsequently in January 2003 the SBE finalized the weights of components for the 2002 Base API to minimize the impact of the shift from the Stanford 9 to the California Achievement Test, Sixth Edition (CAT/6) norm-referenced assessment in 2003. Tables of final indicator weights are available in the 2002 Academic Performance Index Base Report Information Guide [http://www.cde.ca.gov/ta/ac/ap/documents/infoguide02b.pdf] (PDF; 897KB; 63pp.).
The final element of the 2002 Base API is a Scale Calibration Factor (SCF). The SCF is a numerical constant that is computed by grade span (2-6, 7-8, and 9-11) and then added to each school's API according to the school's grade span.
The SCF may be a positive or negative number. The purpose of the SCF is to enhance the stability and interpretability of the API by ensuring that the statewide average API does not fluctuate solely as the result of adding new API components.
Mobile Students Excluded from the API
Until the publication of this 2002 Base API, California law excluded from a school's API the scores of students who were not enrolled in the school district in the previous year. In October 2002 the law was amended to exclude scores only of students who have not been continually enrolled in a school in the same district since the preceding CBEDS date. This is the date in October on which Local Educational Agencies (LEAs) submit data to the California Basic Educational Data System (CBEDS).
The 2002 Base API reflects this change in the law. It is anticipated that the definition of mobile students will change again with the publication of the 2003 Base API. This is as a result of an alignment of state law with the requirements of the federal No Child Left Behind (NCLB) Act of 2001. The new definition would hold schools accountable only for the performance of those students continuously enrolled in the school, not in the district, since the preceding CBEDS date.
Other Exclusions
Besides student mobility, there are a number of other circumstances under which the CDE excludes student test results from the API. In determining when this occurs, the CDE employs the same pupil exclusion rules used in aggregating 2002 school-level STAR results [http://star.cde.ca.gov/].
- The Stanford 9 results are excluded if the Stanford 9 test
administration accommodation for the pupil was more than one
grade out of level (e.g., a sixth grader tested lower than 5th
grade or higher than 7th grade).
- The Stanford 9 results are excluded if any of the following
eleven test administration accommodations were marked "yes"
for all Stanford 9 content areas:
Presentation- Braille
- Directions translated
- Other
Response- Marked answers in test booklet
- Scribe marked answer document
- Other
Timing/Scheduling- Additional time
- Additional breaks
- Other
Use of Aids- Bilingual dictionary
- Other
- The Stanford 9 results for a particular content area are excluded
if the percentile rank for that content area was not between
1 and 99.
- The Stanford 9 results for a particular content area of a
pupil record are excluded if any of the test administration
accommodations listed in subparagraph # 2 above were marked
"yes" for that content area.
- The Stanford 9 results for a particular content area of a
pupil record was excluded if any of the following eleven test
administration accommodations were marked "yes" for that content
area:
Presentation- Questions read aloud or signed
- Directions translated
- Other
Response- Marked answers in test booklet
- Scribe marked answer document
- Other
Timing/Scheduling- Additional time
- Additional breaks
- Other
Use of Aids- Bilingual dictionary
- Other
- The Stanford 9 results from mathematics are excluded if the
"calculator/math tables" item was marked "yes."
Accommodations/Modifications on CSTs, CAHSEE
Results from students taking the California Standards Tests with accommodations are included in the 2002 Base API. However, the API treats CST results from any student who is administered a test below the student's grade level are treated as "far below basic."
Results from students taking the CAHSEE with accommodations are included in the 2002 Base API, but results from students taking the CAHSEE with modifications are not included.
Core Elements of the Report
Certain core elements appear throughout the 2002 API Report. They include:
- Number of Students Included in the 2002 API
- 2002 Base API
- 2001 API Base
- 2002 Statewide Rank
- 2002 Similar Schools Rank
- 2002–2003 Growth Target
- 2003 API Target
Number of Students Included in the 2002 API
This is the number of students with valid scores, i.e., scores contributing to the school's API. It is possible for a student to have test results contributing to the component score in one content area and not another. For example, if a student did not receive a test score in English language arts, but did receive a score in mathematics, the student's mathematics results can still be included in the school's API.
2002 Base API
The 2002 Base API summarizes a school's performance on the 2002 STAR and CAHSEE. It is on a scale of 200 to 1000. It is based on the performance of individual pupils on Stanford 9 (all content areas) as measured through national percentile rankings (NPRs) and on the California Standard Tests as measured through performance levels in ELA, mathematics, and history/social science. CAHSEE results are summarized on a pass/no-pass basis.
In some instances, APIs are also calculated for student subgroups at a school in order to ascertain whether the school meets the "comparable improvement" criterion (see below). For details on the calculation of the 2002 Base API, please consult the 2002 Academic Performance Index Base Report: Information Guide [http://www.cde.ca.gov/ta/ac/ap/documents/infoguide02b.pdf] (PDF; 897; 63pp.).
The API for schools with grade configurations that include grade spans 6-7 or 8-9 is the average of the APIs for the grade configuration segments weighted by the number of pupils with valid scores in the segments. For example, for a 7-12 school, the API is the weighted average of the APIs for grades 7-8 and for grades 9-11. This procedure is necessary because the structure of the test varies between grades 7-8 and 9-11.
Statewide Rank
All schools that receive APIs are ranked in deciles by school type: elementary, middle, and high. A rank of 10 is the highest and 1 is the lowest. Each decile in each school type contains 10% of all schools of that type.
Similar Schools Rank
All schools with at least 100 students with test results included in the API are also ranked in deciles by school type when compared to schools with similar characteristics. The PSAA specifies these characteristics to include:
- Pupil mobility
- Pupil ethnicity
- Pupil socioeconomic status
- Percentage of teachers who are fully credentialed
- Percentage of teachers who hold emergency credentials
- Percentage of pupils who are English language learners
- Average class size per grade level
- Whether the schools operate multitrack year-round educational programs
To derive these ranks, the CDE employed standard statistical procedures to generate a school characteristics index. All legally required characteristics were considered as part of these procedures. The characteristics index was then employed in the following fashion to determine the "similar schools rank" of an individual school:
- A comparison group for an individual school was formed by listing all schools (of the same school type) in order (high to low) based on the school characteristics index. Then for each school, the 100 most similar schools are selected, that is, the 50 schools immediately above and 50 schools immediately below a selected school. In the event that the individual school's characteristics index was within fifty of either the top or the bottom of the statewide distribution, that school's comparison group became either the top 100 schools by school characteristics index or the bottom 100 as appropriate.
- The 100 schools in the comparison group were separated into deciles according to the value of their 2002 APIs.
- The API of the individual school was then compared to the APIs of the schools in its comparison group.
- The individual school was assigned the appropriate decile rank.
2002–2003 Schoolwide Growth Target
A school's growth target is calculated by taking five percent of the distance between a school's 2002 Base API and the interim statewide performance target of 800. For any school with a 2002 Base API of 781 to 799, the annual growth target is one point. Any school with an API of 800 or more must maintain an API of at least 800.
2003 Target
The API target is the sum of the 2002 API and the growth target, except for schools with a 2002 API of 800 or more. Schools with an API of 800 or more are expected to maintain a score of 800 or more.
Structure of the Report
Structure of the Report
- County and District Lists of Schools
- School Reports
List of Schools
These lists include all public schools in a district or county for which the CDE has calculated an API. The schools are listed alphabetically by type (elementary, middle, high, and small). Schools with non-traditional grade configurations, e.g., 7-12, have been placed into a school type according to standard criteria established by the CDE. These criteria are available at: Criteria for Standard School Type Definitions for the 2002–2003 API Reporting Cycle [http://www.cde.ca.gov/ta/ac/ap/documents/schltypedef02b.pdf] (PDF; 30KB; 3pp.).
APIs for small schools, i.e., with between 11 and 99 valid STAR scores, are asterisked. APIs based on small numbers of students are less reliable and therefore should be carefully interpreted. A sizeable number of public schools do not appear on the API list of schools. These include:
- Alternative schools serving at-risk, non-traditional student populations
- Very small schools with fewer than 11 valid 2002 STAR scores
Discussions on how best to integrate these schools into the accountability system are currently underway, particularly in light of the provisions of the federal No Child Left Behind Act of 2001.
Schools on the Lists without APIs
Some schools appear on the list of schools without APIs because they have had their 2002 Base API invalidated. Under regulations adopted by the SBE, this may have occurred for one of several reasons:
- Adult testing irregularities in excess of 5% of the students have occurred at the school.
- The rate of students who have been excused from STAR testing by parent request is equal to or greater than 10 percent (schools with rates between 10 percent and 20 percent have had their APIs reevaluated through standard statistical tests to check the representativeness of the tested population).
- The school failed to test 85% of its students in a particular content area (either norm-referenced or standards tests).
- The school has unresolved problems with STAR demographic data.
School Report
A School Report is generated for each school that has API information displayed on the List of Schools. In addition to the common core elements, the School Report includes:
- Data on subgroups
- School demographic characteristics
SubGroups
The PSAA defines a "numerically significant ethnic or socioeconomically disadvantaged subgroup" as a subgroup "that constitutes at least 15 percent of a school's total pupil population and consists of at least 30 pupils." Also, under the law, if a subgroup defined by ethnicity or socioeconomic disadvantage constitutes at least 100 pupils, i.e., at least 100 pupils with valid STAR scores, that subgroup is "numerically significant" and required to demonstrate comparable improvement, even if it does not constitute 15 percent of the school population.
These numerical criteria (15 percent and 30 pupils, or 100 pupils) are calculated on the basis of the number of pupils with valid STAR scores for that subgroup.
The school is responsible for demonstrating comparable improvement only for those subgroups that are numerically significant for both the 2002 Base API and the 2002–2003 Growth API.
Ethnic/racial subgroups include:
- African American (not of Hispanic origin)
- American Indian or Alaska Native
- Asian
- Filipino
- Hispanic or Latino
- Pacific Islander
- White (not of Hispanic origin)
- Neither of the pupil's parents has received a high school
diploma
OR
- The pupil participates in the free or reduced price lunch
program.
Information from the 2002 STAR or CAHSEE student answer document determines whether a student meets either criterion.
A pupil who is a member of the socioeconomically disadvantaged subgroup is also a member of one of the racial/ethnic subgroups. Therefore, it is possible that the total percentage of students in all numerically significant subgroups at a school may exceed 100.
By regulation, "comparable improvement" requires that each numerically significant subgroup must meet or exceed 80 percent of the 2002–2003 schoolwide growth target.
The 2002–2003 subgroup target is calculated by first multiplying the schoolwide target by .8 and then rounding the product to the nearest whole number.
There are four minor exceptions to this rule:
- For subgroups within schools with schoolwide APIs between 790 and 799, i.e., approaching the statewide interim performance target of 800, the annual growth target is one point.
- Regardless of the schoolwide API, subgroups already at or above 800 must continue to meet the statewide interim performance target of 800.
- In schools with 2002 APIs of 800 or more, subgroups with an API of less than 800 must make growth of at least one point.
- In instances where 80 percent of the schoolwide target results in a subgroup target that would exceed the distance from the subgroup API to 800, the subgroup target equals the distance to 800.
School Demographic Characteristics
Along with general subgroup and schoolwide demographic data, the School Report includes the demographic characteristics on which the school characteristics index for the 2002 Base API similar schools rankings is based. The data for the percentages and rates were collected from these sources:
- October 2001 CBEDS data collection (information on teacher credentials, multi-track year round participation, and average class size)
- 2002 STAR student answer document (information on ethnic/racial distribution, parental education level, participation in free or reduced price lunch program, district mobility, school mobility, the number of English learners, the total number of students tested as well as the number of students excused from testing in accord with IEPs or at the request of their parents)
- It is possible for one teacher to be in both the fully credentialed and emergency credential categories; therefore, the total of the percentages for "Fully credentialed teachers" and "Teachers with emergency credentials" may exceed 100.
- Average class sizes were derived from the enrollment data reported on the Professional Assignment Information Form (PAIF).
- The item "core academic courses in departmentalized programs" reflects average class size in the following subject areas: English, Foreign Languages, Math, Science, and Social Science.
Regarding background characteristics derived from the STAR student answer document:
- School mobility is the percentage of students who have continuously enrolled in the school since the preceding school year. Currently it is used as a background characteristic only.
- District mobility is the percentage of students who have been continuously enrolled in the school district since the CBEDS date of the preceding calendar year. Students who have not been continuously enrolled in the school district since CBEDS date of the preceding calendar year are excluded from a school's API.
Some of the School Demographic Characteristics that appear on this report are used in the formation of the similar schools comparison groups for the similar schools ranking on the 2002 Base API Report.
