California's Response to New NCLB Flexibility

Foreword
August 22, 2005

Over the last two years, State Superintendent of Public Instruction Jack O'Connell has publicly urged the U.S. Department of Education (ED) to allow the use of growth models, such as the California Academic Performance Index (API) system, in implementing the accountability provisions of the No Child Left Behind (NCLB) Act of 2001. Instead of the "one size fits all" status model of NCLB, states should be able to employ established growth models to determine whether or not their own schools and Local Educational Agencies are making Adequate Yearly Progress (AYP). In March 2004, at O'Connell's urging, 14 other state departments of education joined California in pushing this initiative. In April 2005, U.S. Secretary of Education Margaret Spellings announced the formation of a working group to consider the growth model issue. This was part of a new ED emphasis on flexibility in implementing NCLB accountability requirements.

The growth model working group has since met twice, with O'Connell's chief deputy, Gavin Payne, attending the second of the sessions. The assignment of this group is to assist the ED in developing criteria that a growth model would have to meet for determining AYP. The intent of the ED is to issue formal guidance to the states in the fall of 2005. On the basis of this guidance, states would submit proposed growth models to the USDE for consideration. ED-approved growth models could then be used by states to establish AYP for the 2005-06 test results.

This paper that follows was presented to the State Board of Education on May 5, 2005. After providing background to the growth model issue, it presents possible modifications in the API to assist in developing a formal proposal to the ED.

Executive Summary
California's Response to New NCLB Flexibility
April 2005

Purpose

This paper advocates that the U. S. Department of Education (ED) grant genuine flexibility to the States in the implementation of the accountability provisions of the No Child Left Behind Act (NCLB) of 2001 to accommodate comprehensive and well-established state accountability systems. It urges that the ED accept a growth or improvement model to determine Adequate Yearly Progress (AYP) for Local Educational Agencies (LEAs) and schools. In California, this is the Academic Performance Index (API) system created by the Public Schools Accountability Act (PSAA) of 1999.

Without this change, we will continue to experience the confusing and frustrating situation where one day a school is lauded for its performance gains and then on the next day condemned as a failure. This situation is not unique to California; it often occurs in other states with established accountability systems that antedated the enactment of the NCLB. In Florida, for example, “less than nine percent of its schools were given a letter grade of D or F based on the state system but more than 75 percent of the schools missed AYP.”1

If this situation is allowed to continue, the resulting frustration will soon turn into cynicism and seriously undermine the credibility of both federal and state initiatives in the area of educational accountability. This paper advocates an end to this situation in California by restoring the priority of the API system for determining AYP and identifying Program Improvement (PI) districts and schools.

This paper is divided into four parts:

• Firstly, it provides background on the difference between a "growth model" and "status model," specifically, the difference between California's API and the federal accountability criteria within NCLB;
• Secondly, it identifies areas in which the API is a more effective agent of change in California education than the status model currently required by the federal government;
• Thirdly, it considers modifications in the API system that would address valid concerns of some of its critics. Modifications include one or all of the following:

1. Calculate separate annual API growth targets for each numerically significant student subgroup at a district or school, thereby requiring higher targets for lower-performing student subgroups;
2. Double the annual API growth target from 5 percent to 10 percent, with a minimum of a five-point gain for any district or school that has not yet reached the statewide performance goal;
3. Require the calculation of growth targets for school districts;
4. Raise the statewide performance target of 800 toward 875.

• Fourthly, it defines the relationship that should exist between the ED and the States, including California, on the implementation of the accountability provisions of NCLB.

Position Paper:
California's Response to New NCLB Flexibility

Purpose

This paper advocates that the U.S. Department of Education (ED) grant genuine flexibility to states in the implementation of the accountability provisions of the No Child Left Behind Act (NCLB) of 2001 to accommodate comprehensive and well-established state accountability systems. This should include accepting a growth or improvement model for determining Adequate Yearly Progress (AYP) for Local Educational Agencies (LEAs) and schools. In California, this would be the Academic Performance Index (API) system created by the Public Schools Accountability Act (PSAA) of 1999.

Without this change, we will continue to experience the confusing and frustrating situation where one day a school is lauded for its performance gains and then on the next day condemned as a failure. This situation is not unique to California; it also often occurs in other states with established accountability systems that antedated the enactment of the NCLB. In Florida, for example, "less than 9 percent of its schools were given a letter grade of D or F based on the state system but more than 75 percent of the schools missed AYP."2

Apologists have defended these apparent anomalies by contending that the federal and state systems simply give different and complementary information about districts and schools, ignoring the fact that for many districts and schools this "additional information" carries severe consequences. Faced with the real prospect that virtually every Title I district and school may eventually be identified for Program Improvement (PI), they respond with the platitude that every district and school can improve in some aspect.

However, our experience is that educators and the general public in California are confused, and not enlightened by the incongruities. They understandably see the federal and state systems as producing inconsistent and not complementary results. Furthermore, finding it impossible to condense the formula "everyone can improve" into a headline, the media find easier to blanket all newly identified PI schools as "failures," no matter how much some of these schools have improved from the previous year or how many of the AYP criteria they have met.

If this situation is allowed to continue, the resulting frustration will soon turn into cynicism and seriously undermine the credibility of both federal and state initiatives in the area of educational accountability. This paper advocates an end to this situation by restoring the priority of the API system in California for determining AYP and identifying PI districts and schools.

The paper is divided into four parts:

• Firstly, it provides background on the difference between a "growth model" and "status model," specifically, the difference between California's API and the federal accountability criteria within NCLB;
• Secondly, it identifies areas in which the API is a more effective agent of change in California education than the status model currently required by the federal government;
• Thirdly, it identifies modifications in the API system that would address valid concerns of some of its critics;
• Fourthly, it defines the relationship that should exist between the ED and the States, including California, on the implementation of the accountability provisions of NCLB.

Background

Definitions

The term "growth model" is often used indiscriminately, with little regard to any generally accepted meaning. Some observers apply the term to any statewide accountability system that has more than one performance "cut point." Such systems give local educational agencies (LEAs) and schools credit for moving students from one level of achievement to another. In the Massachusetts school performance rating process, for example, schools receive extra points for moving students from "warning/failing" to "needs improvement," even though these students still do not score at the proficient level. Other observers have used the term “growth model” in a much more restrictive fashion, applying it only to accountability systems that base LEA and school results on value-added analyses of individual student gains, such as in Tennessee.

For purposes of this paper, a "growth model" is defined as one that measures the academic success of the LEA, a school, or of student subgroups solely on the basis of how much student achievement improves. It acknowledges that not all LEAs, schools, and student subgroups start at the same place. A growth model supports ambitious targets that are individual to each LEA, school, and student subgroup. The term “growth model” encompasses accountability systems based on cross-sectional analyses of grade-level test results from year to year (California) as well as value-added analyses of individual student gains on tests (Tennessee).

A "status model," on the other hand, presumes that all LEAs, schools, and student subgroups should meet common minimum targets, which escalate over time. This is the model upon which the current federal accountability criteria are based. The Texas accountability system is the best example of a well-established state system that uses this type of model.

The PSAA and NCLB

The California PSAA and the federal NCLB have a common set of core beliefs. Both laws advocate accountability models that:

• Establish rigorous academic standards
• Implement assessments to measure whether students meet these standards
• Set measurable performance goals for schools
• Hold schools accountable for reaching those goals
• Hold schools accountable for comparable improvement by student subgroups in meeting those goals

Despite this common set of assumptions, fundamental differences exist between the accountability models in the PSAA and NCLB. The California accountability system is based on a growth model, while the accountability criteria in NCLB rest upon a status model.

The API

The cornerstone of the California accountability system, as mandated by the PSAA, is the API. This single-number index, which ranges from 200 to 1000, summarizes a school's performance over a number of indicators. Currently, these indicators consist of results from the statewide assessment program. Results from the California Standards Tests (CSTs) furnish most of the weight of the API.

Growth or decline in a school's API determines eligibility for the state awards or interventions programs in general. A school's annual growth target is determined by taking 5 percent of the difference between a school's current API and the statewide performance target of 800. Numerically significant subgroups within the school are expected to demonstrate comparable improvement, defined as 80 percent of the school-wide API target, in order for a school to meet its growth target.

Prior to the implementation of NCLB, whether a school met its API growth target determined whether it made AYP. This did not amount to a "free pass" for California schools. In fact, in 2002, the last year that the API alone was used to establish AYP, 178 schools were identified for PI.

Federal Status Model

The federal accountability model in NCLB is radically different from the California growth-based system. LEAs and schools are expected to have the same minimum percentage of students at or above the proficient level in both mathematics and reading/language arts. This applies to all significant student subgroups in the LEA or school. This minimum percentage rises so that within 12 years, 100 percent of the students at all schools should be at the proficient level or above in both content areas. As previously noted, many aspects of this model are similar to the approach of the Texas accountability system.

Within NCLB, there is a brief acknowledgment that the concept of growth has some relevance to the question of AYP — the so-called "safe harbor" provision. The authors of the law appeared to recognize that growth in the end could be the deciding factor in whether an LEA or school could be subject to PI identification and possible sanction.3  However, the current provision relegates the notion of growth to an exceptional procedure that is only invoked if the LEA or school does not meet the Annual Measurable Objective (AMO). From the perspective of the proponents of a growth model, this procedure sends the wrong message to LEAs and schools. Growth should be required of all LEAs and schools, and meeting status targets should be invoked only as an exceptional measure.

Also, the safe harbor provision has a glaring operational fault: it penalizes states with rigorous definitions of proficient and corresponding lower starting points for the percentage of students proficient or above. For these states, a 10 percent reduction in the percent non-proficient may presuppose a virtually impossible level of annual growth.4

The API as an Agent of Change in California

API has fostered a culture of accountability

The API was the result of a yearlong process of deliberations by an advisory committee with the input of stakeholder groups. A technical design group composed of testing experts provided their expertise in support of these deliberations. Implementation of the accountability system was a transparent and open process that enabled public input during the design of the API and the implementation of the accountability system.

As a result, California educators, parents, and members of the public view the API system as legitimate, both because local stakeholders had considerable input into its development as well as the fact that the system has now been in place for six years. Also, the system has been associated with significant and steady improvements in school performance, as reflected in API gains, and therefore widely viewed as a success. From 1999 to 2004 the median API for elementary schools increased by 106 points, from 629 to 735; for middle schools by 70 points, from 633 to 703; and for high schools by 50 points from 620 to 670.

Those API gains reflect that test scores have risen over the last six years, even as our tests have become more challenging. In 1999 the API relied solely on results from a nationally norm-referenced test, the SAT/9, with the rigorous California Standards Tests (CSTs) still in development; by 2004 the API was based overwhelmingly on results from these CSTs and the standards-linked California High School Exit Examination (CAHSEE).

A major factor in the performance gains of the last six years has undoubtedly been the culture of accountability that the API has fostered in districts and schools. Californians, inside and outside of schools, have become generally conversant in the language of the API, including its idioms and acronyms. Newspapers have provided glossaries of the terminology. All of this reflects a general acceptance of the statewide accountability system as legitimate.

LEA and school administrators have become familiar with the basic features of the API. It has given them a common metric to gauge the progress and to compare the performance of their schools. Educators have learned to use API results and associated data to improve instruction. Parents and the general public have also found the single number index to be a useful tool with which to balance more qualitative information they receive from LEAs about local schools.

In contrast, state, LEA and school administrators tend to view the AYP provisions as something imposed from the outside rather than home grown. While they embrace the general tenets behind the AYP provisions, they question why ED saw fit to impose the conditions of meeting them in such a literal fashion in a state with a comprehensive and well-established system of accountability. They also view the federal model as a "one-size-fits-all" model that does not give sufficient consideration to the unique features of education in California. This set of perceptions severely limits the potential that NCLB has to support accountability efforts in California.

API is applicable to all schools

An effective accountability system encourages all schools to improve. In a nod to universality, the federal government now requires that the state make AYP determinations for all schools, not just those receiving Title I funds. The measures a state uses to evaluate improvement in Title I schools must also be applied to non-Title I schools, although provisions of the law on school choice and corrective actions apply only to Title I schools.

In effect, however, in the first years of NCLB implementation, the federal status model is of limited application, particularly for schools in the middle or higher ranges of the performance spectrum. Under the federal model, a school makes AYP so long as it meets the AMOs for that particular year. A school may actually experience a decline in performance over several years and still make AYP during that same period of time. In contrast, a high-performing school that experiences a decline in its API over the same time period is subject to intense media and public scrutiny. This is an inducement to continued growth.

API supports a rigorous definition of proficient

Under the federal AYP model, each state may define "proficient" more or less as it sees fit. However, the net effect of the system of annual status targets has been to encourage some states to adopt more lenient definitions of what students should know and be able to do. In California, the State Board of Education (SBE) refused to lower the performance standards, instead maintaining a rigorous definition of "proficient," despite the fact that a status model implicitly penalizes states with high standards.

Since a growth model considers the net improvement of individual districts and schools, there is no advantage to be gained from lowering expectations. A growth model such as the API is actually more congruent with high academic standards and rigorous definitions of student proficiency than a status model.

API promotes a better allocation of resources

Many if not most states face severe fiscal constraints. This includes California. It makes little sense to identify California LEAs and schools for Program PI or corrective action when they are making significant improvement as reflected in API gains. Instead, the state should devote scarce resources to those LEAs and schools that are incapable of improving on their own.

For example, in 2003 2,200 California Title I schools (45 percent of the total) failed to make AYP. Of these, 1,640 (or 75 percent) met their API growth targets, both school-wide and for all numerically significant student subgroups. Viewed from a slightly different perspective, in 2004 321 California schools:

• Improved their school APIs by more than double their annual API growth targets for the second year in a row;
• Met all subgroup API targets during the same period;
• Did not make AYP.

Using the API to determine AYP would have enabled a more precise identification of those LEAs and schools where program improvement and corrective actions were appropriate, and where resources could be more effectively utilized for higher achievement.

Modifications in the API

Despite its merits, the API is not without its detractors. Interestingly, most of them direct their criticisms not at the API per se but rather at the API performance goals and targets. They contend that these goals and targets are not rigorous enough in the rate of improvement that is required. On a related issue, they also argue that common school-wide subgroup targets do not require the closing of the achievement gap, a key principle of the NCLB Act. Finally, the current PSAA is "school-based: it is largely silent on the issue of school district accountability. For these critics of the API, the federal accountability model is necessary because it specifically redresses what they perceive as key deficiencies in the API system.  

It is important for both the advocates and the critics of the API to recognize that these concerns focus on the rigor and the coverage of the API, not on its validity as a measure. Therefore, it is possible to deal with them without "de-constructing" the API.

Modifications in the state system could take place in response to a USED agreement to enable California to employ growth in the API for determining AYP for its districts and schools. In return, California could agree to one or all of the following:

1. Calculate separate annual API growth targets for each numerically significant student subgroup at a district or school, thereby requiring higher targets for lower-performing student subgroups;
2. Double the annual API growth target from 5 percent to 10 percent, with a minimum of a five-point gain for any district or school that has not yet reached the statewide performance goal;
3. Require the calculation of growth targets for school districts;
4. Raise the statewide performance target of 800 toward 875.

Relationship between the Federal Government and the States

This agreement should take place in the context of a re-definition of the relationship between the federal government and the States, including California, in the area of accountability. As recommended in the NCSL Report on NCLB,

"States should be allowed to use their state accountability system to comply with the spirit of the federal law. These systems should be based on realistic academic goals and not on the broad aspirational proficiency goals of NCLB."5

In the relationship between the federal government and the States, the key elements of the accountability provisions of NCLB would represent guiding principles upon which everyone can agree. They would constitute national goals that all states, LEAs, schools, and students should strive to attain. The NCLB properly focuses on subgroup accountability as a mechanism to reduce achievement gaps. It should continue to require that states design and implement rigorous accountability systems that move students in these states towards the national goals. It should continue to require near universal participation in assessments, so that parents know whether or not their children are attaining proficiency in core academic subjects.

But the ED should make it clear that the status model in Title I accountability provisions is just that — a model. Most importantly, it should offer genuine flexibility by redefining the relationship between federal and state governments in the implementation of these provisions. This does not mean that the federal government should function as a source of funding for the states without any corresponding responsibility on the part of the states to meet national goals. However, the relationship between the federal government and the states as it currently exists is dysfunctional and requires significant adjustment if NCLB is to survive.

Without going into unnecessary detail, it is easy to set forth a set of broad guidelines that would at once address the need for statewide accountability and still accord states true flexibility in how they meet federal requirements. In this new relationship, each state would assure the federal government that it would:

• Adopt the national goal of every student proficient or above in mathematics and reading/language arts by 2013-14;
• Develop and adopt statewide academic standards in accord with state law;
• Administer annual academic achievement assessments in reading/language arts and mathematics;
• Report state-level results annually in a prescribed format corresponding to the national goals;
• Ensure that parents are informed of test results;
• Design statewide accountability systems that effectively measure academic achievement, seek to reduce the achievement gaps between student subgroups, and increase graduation rates.

As long as states accept these conditions, they should be free to adopt any type of accountability system, whether based on a growth model such as California or on a status model similar to that in the current NCLB. States should continue to enjoy such flexibility unless they cannot demonstrate:

• Significant improvement in the percent proficient or above in mathematics and reading/language arts;
• Closing the achievement gap between traditionally higher and lower scoring student subgroups in those two content areas.

What constitutes "significant improvement" and "closing the achievement gap" in a particular state should be informed by the best research in educational measurement, not by federal mandate. In conclusion, California asks that the federal government hold the state responsible for educational outcomes, not for processes. This is, after all, both the basis of genuine accountability in education and also of genuine federalism.

Four API Models

A. Current

• School wide target = 5%
• Subgroup target = 80% of school

B. Model 1

• School wide target = 10%
• Subgroup target = 80% of school

C. Model 2

• School wide target = 10%
• Subgroup target = 10%

D. Model 3

• School wide target = 10%
• Subgroup target = 10%
• Minimum 5 points growth

Percent of Schools Making API Targets
2004

Results by 2003 Decile Rank
Current vs. Model 3

------------------------------------------------------------------------------

Footnotes

1 National Conference of State Legislatures Task Force on No Child Left Behind, Final Report (February 2005), p. 20. (Hereafter referred to as NCSL Report.)

2 National Conference of State Legislatures Task Force on No Child Left Behind, Final Report (February 2005), p. 20. (Hereafter referred to as NCSL Report.)

3 Interestingly, an LEA or school could repeatedly invoke safe harbor, avoid identification for PI, and still not have 100 percent of its students at or above proficient by the end of the twelve-year period.

4 California has experienced this first hand. For 2003, out of about 9,000 schools, twelve qualified for safe harbor. In 2004 only three schools qualified

5 NCSL Report on NCLB, p. 20.

California Department of Education
1430 N Street
Sacramento, CA 95814

Contact Us  |  FAQ  |  Web Policy