In order to achieve the goals we have described, we need useful information to guide decisions; incentives that encourage both the right kind of instruction and the right kind of investments; and the knowledge, skill, and capacity in schools to improve practices and outcomes.
All of these conditions currently pose challenges for the state. California lags other states in the development of educational data systems, including a student longitudinal data system, a teacher longitudinal data system, and a data repository that links preschool, K-12, higher education, and non-educational data and makes it available for policy and research.
The state's accountability system, based on the Academic Performance Index (API), is based only on student test scores rather than a broader array of student outcomes and only takes into account student background characteristics and not the resources of schools, families, and communities. It also provides no diagnostic information on the capacity of schools and districts that could guide improvement efforts.
Finally the state accountability system, like the federal system under No Child Left Behind, relies only on performance standards coupled with rewards and sanctions to improve underperforming schools and districts. These systems provide little information on the source of the underperformance and little guidance on how to improve performance. As Richard Elmore points out:
Low-performing schools, and the people who work in them, don't know what to do. If they did, they would be doing it already. You can't improve a school's performance, or the performance of any teacher or student in it, without increasing the investment in teachers' knowledge, pedagogical skills, and understanding of students. This work can be influenced by an external accountability system, but it cannot be done by that system. Test scores don't tell us much of anything about these important domains; they provide a composite, undifferentiated signal about students' responses to a problem.20
The solution to improving the performance of California's schools and districts is based on the simple premise: improvement depends on capacity. An increasing body of research has identified capacity as the key ingredient in improving school performance.21 This capacity depends on improving both the individual capacity of teachers and school leaders—their knowledge, skills, and material supports—and the institutional capacity of schools, districts, and state agencies to support the delivery of improved education through strong staffing, instructional guidance, well-directed resources, helpful data and information, and productive incentives.22
The underlying problem in California is a lack of capacity at many levels throughout the state's educational system—schools lack the capacity to support teachers, districts lack the capacity to support schools, and the state lacks the capacity to support school districts. A related problem is the lack of coordination among the various levels of the educational system in developing and providing capacity.
As the state faces an extreme fiscal crisis, it may be difficult to implement new school improvement systems in the short-term. However, there may be opportunities for rethinking elements of the testing and accountability systems while fiscal capacity is being recovered. It is imperative that discussions take place as soon as possible in order to come to agreement on the changes to be made once the state has resources to invest in updating these systems.
Accountability and School Improvement Key Recommendations
The Transition Advisory Team believes that California needs both a stronger understanding of how to build school and educator capacity and an improved state accountability system that (1) is based on promoting continuous improvement of a broad array of student outcomes as well as continuous organizational learning; (2) places importance on improving the performance of underperforming students; (3) better identifies the needs and capacity of underperforming schools and districts and provides the appropriate resources and supports to improve; and (4) properly balances accountability—both vertically within the state and district (teachers, principals, superintendents, school boards), and horizontally within the community (schools, businesses, community based organizations, county services etc.). To accomplish this, CDE should:
- Develop a robust system of indicators to inform school accountability, including broader measures of student learning that focus on growth and better assess 21st century skills; measures of school capacity and student opportunities to learn; and measures of resources connected to Opportunity-to-Learn standards that describe the financial and human resources available to schools and help set targets for a system that ensures schools the resources they need.
- Create an Accountability and School Improvement task force to design a more useful and effective accountability and school improvement framework and to develop more productive approaches to building school and district capacity.
- Create a senior data advisor position at CDE to ensure the effective implementation of well-designed, privacy-protected longitudinal data systems (including CALPADS and CALTIDES), as well as integration of these systems with early learning and higher education data sets. Ensure and establish a clear vision of how this database will operate and be used for betterment of education throughout the state.
- Create an evaluation capacity in CDE through internal staffing and partnerships with research organizations, in order to enact policies and propagate practices based on sound research.
21. Malen, B. & Rice, J.K. (2004). "A framework for assessing the impact of education reforms on school capacity: Insights from studies of high-stakes accountability initiatives." Educational Policy, 18, 631-660; Rumberger, R. W. & Connell, J. (2007). Strengthening school district capacity as a strategy to raise student achievement in California. Policy Brief for Getting From Facts to Policy: An Education Policy Convening, Sacramento, October 19. Mountain View, CA: EdSource. Retrieved March 24, 2011 from: Rumberger Connell Brief [http://www.edsource.org/assets/files/convening/RumbergerConnell_brief.pdf] (PDF); Spillane, J.P. & Thompson, C.L. (1997). Reconstructing conceptions of local capacity: The local education agency’s capacity for ambitious instructional reform. Educational Evaluation and Policy Analysis, 19, 185-203.
22. Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2009). Organizing schools for improvement: Lessons from Chicago. Chicago: University of Chicago Press; Knudson, J., Shambaugh, L., & O’Day, J. (2011). Beyond the School: Exploring a Systematic Approach to School Turnaround. "California Collaborative on District Reform, Policy and Practice Brief. AIR; Supovitz, J. (2008). Building System Capacity for Improving High School Graduation Rates in California." CDRP Report 9. Santa Barbara, CA: California Dropout Research Project. Retrieved March 24, 2011, from: California Dropout Research Project [http://cdrp.ucsb.edu/dropouts/pubs_reports.htm#9] , p. 6.