Skip to main content
California Department of Education Logo

PSAA Meeting Minutes – November 27, 2012

Minutes of the Advisory Committee for the Public Schools Accountability Act (PSAA) of 1999.
Members Present:

Co-Chair: Ting Sun, Educational Programs Director, Natomas Charter School
Co-Chair: Kenn Young, Superintendent, Riverside County Office of Education

Nancy Aaberg, Superintendent, Yuba City Unified School District
Vicki Barber, Superintendent, El Dorado County Office of Education
Dennis Cima, Senior Vice President, Silicon Valley Leadership Group
Edward Haertel, Professor, School of Education, Stanford University
Julianne Hoefer, Director, Assessment and Accountability,
Fountain Valley Unified School District
Clifford Kusaba, Teacher, California Teachers Association
Chet Madison, Sr., President, Board of Education, Elk Grove Unified School District
Sonia Ortiz-Mercado, Dean, Matriculation, Student Equity, Early Assessment, Student Leadership, and Division Administration, California Community Colleges Chancellor's Office
Jeff Patterson, English Teacher, SOAR Prep Academy
Charles Weis, Retired Superintendent, Santa Clara County Office of Education
Colleen A.R. You, President-elect, California State PTA

State Board of Education (SBE) Liaison: Ilene Straus

Members Not Present:
Wendell Callahan, Director, Assessment, Research, and Pupil Services, San Diego County Office of Education
Pat Crowder, Principal, Patrick Henry High School
SBE and Legislative Representatives:

California State Senate: Susanna Cooper, Principal Consultant, California State Senate President Pro Tempore Darrell Steinberg

SBE Staff: Patricia de Cos, Deputy Executive Director

Principal California Department of Education (CDE) Staff to the Advisory Committee:

Deb Sigman, Deputy Superintendent of Public Instruction, District, School, and Innovation Branch
Keric Ashley, Director, Analysis, Measurement, and Accountability Reporting Division
Jenny Singh, Administrator, Academic Accountability Unit
Matt Taylor, Administrator, Evaluation, Research, and Analysis Unit

Please note that the complete proceedings of the Novmber 27, 2012, PSAA Advisory Committee meeting, including captioning, are available on the CDE PSAA Meeting Webcast Archive 2012 Web page.

The meeting was called to order at 10:07 a.m.

Welcome and Introductions

Keric Ashley welcomed everyone to the meeting. Deb Sigman welcomed PSAA Advisory Committee members and thanked them for their commitment to serving on the committee. She introduced Keric Ashley as the newly appointed Director of CDE's Analysis, Measurement, and Accountability Reporting Division (AMARD). She said the focus of the committee's work over the next year is the implementation of Senate Bill (SB) 1458 (Chapter 577, Statutes of 2012). The role of the committee is to provide input to the State Superintendent of Public Instruction (SSPI) regarding SB 1458. The immediate task is to make recommendations to the SSPI on indicators and weights for the 40 percent of the Academic Performance Index (API) separate from the Standardized Testing and Reporting (STAR) Program and the California High School Exit Exam (CAHSEE) results. Each committee member, the SBE liaison, the SBE staff, and CDE principal staff introduced themselves. Keric Ashley announced that Kenn Young and Ting Sun would be co-chairs of the committee.

Legislative Intent and Perspective on SB 1458 (Chapter 577, Statutes of 2012)

Susanna Cooper thanked the committee members for their commitment to the work of implementing SB 1458. Ms. Cooper explained why SB 1458 is needed. The national and global economy and workplace have evolved dramatically over the past several years and will continue to evolve. Most high schools in California are not keeping up with the pace of these changes. Students need to understand the "how" and "why" of learning. Many students are not adequately prepared for the workplace, and many require remediation in college. California’s state and federal accountability system currently does not support preparing students for this evolving economy. The state has control over the API, which can be a lever for addressing these issues through the state's accountability system.

Ms. Cooper said SB 1458 provides the opportunity to change the API system to support rigor and relevance for students in high school and to establish a tighter connection between schools and the economy. The API played a key role in establishing higher standards in academic content areas but now must be more comprehensive to include graduation rates and readiness for college and careers.

Public Comment: No public comment.

Overview of SB 1458

Keric Ashley stated that the primary goal for this meeting of the committee was to allow all committee members the opportunity to voice their views on agenda items (without decisions), the advantages/disadvantages of those items, and the sources of data for those items.

Jenny Singh presented an overview of key components of SB 1458. The committee members considered how to obtain broad input on SB 1458. Many members suggested groups or organizations that could provide input or assistance, e.g., California County Superintendents Educational Services Association (CCSESA), California Teachers Association (CTA), Association of California School Administrators (ACSA), and Parent Teachers Association (PTA).

Members discussed ideas for indicators for the 40 percent. Several members suggested reviewing the prior API guiding principles, which specify that the indicators must be valid and reliable. Also, they said new indicators, as much as possible, should fairly measure the diversity of the population of California’s students and schools, i.e., size, language, charter, governance structures, and settings. The PSAA Advisory Committee's Technical Design Group (TDG) should be used to provide advice and recommendations regarding the technical aspects of options for the 40 percent indicators. College and career readiness and workforce skills should be defined first in the broader context. Then review the guiding principles to determine how to assess the important measures and performance levels, and then determine changes to accountability. Because SB 1458 refers to school performance, it was suggested that, rather than concentrating on individual student level measures, the committee should consider school level variables and process variables, particularly for the school quality review. For example, support systems to students, tutorials, partnerships with industry/business, and help for English learners and students with disabilities.

Public Comment was provided by:

  • Christopher Cabaldon, Executive Director of Linked Learning Alliance

  • Fred Jones, Legislative Information Officer for the California Council for the Social Studies (CCSS), representing the California Business Education Association (CBEA); business, labor, and employer groups; and history-social science teachers

  • Lee Angela Reid, Senior Legislative Advocate, Capitol Advisors Group, LLC, represented Gifted and Talented Education (GATE), the California Association of School Counselors (CASC), International Baccalaureate (IB) schools, and the California Regional Occupational Centers and Programs (ROCPs)
Public Meeting Protocols

Amy Bisson Holloway, General Counsel, Legal, Audits, and Compliance for the CDE, presented information on the requirements of the Bagley-Keene Open Meeting Act. She explained key requirements and procedures the state, the committee as a group, and each committee member should know and follow. The Bagley-Keene Open Meeting Act is intended to ensure that the actions of state agencies are taken openly and their deliberations are conducted openly.

Timing of Incorporating New Indicators into the API

Keric Ashley reiterated that the new indicators for the 40 percent may only be added to the API one full school year after the SBE adopts the indicators. Some indicators may be added more quickly than others, such as graduation rate and dropout rate, since those indicators are already under review and discussion by the committee. New indicators could be incorporated all at once (along with the new assessments) for the 2015 Base API or phased in over multiple years. Likewise, weightings could be established all at once or scheduled to increase over time.

Several members showed concern for the short timeline for the committee to decide on its recommendations for the 40 percent. The new indicators for the 40 percent must be incorporated into the 2015 Base API. In order to do this, the SBE must adopt the 2015 Base API new indicators at its July 2015 meeting.1 It may be that all indicators will need to be ready and incorporated all at once due to the short timeline to meet the bill’s requirements.

Committee members strongly suggested that the process for developing recommendations for the 40 percent indicators should follow basic steps:

  1. Review background information. Review past decisions, such as the guiding principles.

  2. Review mandated requirements, regulations, or statutes. First decide on mandated indicators. Prioritize other indicators.

  3. Identify and develop recommendations for new indicators:
    1. Include workplace and stakeholder review.
    2. Check the validity and reliability of data.
    3. Check the impact on different types of schools.
    4. Evaluate the costs of the data.
    5. Check for unintended consequences of indicators.
    6. Evaluate whether the data discriminates/adds value versus redundant.
    7. The CDE should do data runs and evaluate with the TDG a set of indicators and then provide the committee with CDE and TDG perspective on the feasibility of indicators.

  4. Review how Smarter Balanced Assessment Consortium (SBAC) assessments could be implemented.

Committee members expressed the need for Steps 1 and 2 to be done as soon as possible and that all steps should be done halfway into the timeline. The CDE should obtain and provide committee members with information and/or presentations on college and career readiness. Also, committee members would like a list of data currently collected by the CDE and other possible data sources. The committee would also like future meeting agendas to include a summary of prior meeting discussions and decisions to ensure that decisions are consistent and to avoid redundancy.

Public Comment was provided by:

  • Sherry Skelly Griffith, Legislative Advocate for the ACSA
  • Ilene Straus, SBE liaison to the PSAA
College and Career Readiness Indicators

Jenny Singh presented information on possible new API indicators and how the committee might think differently about methods for incorporating new indicators into the API. The committee discussed implications of several of the possible new indicators. For college readiness indicators, completion of a-g was seen as helpful, but a committee member said that many students continue to arrive at college needing remediation even though they had completed all a-g requirements. Using Early Assessment Program (EAP) results as an indicator was viewed as a robust indicator of college readiness; however, not all students take this exam. Results on the California Standards Tests (CSTs) for grade 11 as well as for earlier grades could be valuable indicators, as currently researched by the Riverside County Office of Education. Regarding career readiness indicators, the committee viewed indicators that use student grades as criteria as a problem due to the wide variation of grading policies across school districts. Students completing Advanced Via Individual Determination (AVID) or Career Technical Education (CTE) Pathways were viewed as good indicators, but not all schools have AVID or Pathways programs.

The committee agreed that it would be helpful to define college and career readiness and college and career preparation and to indicate which indicators would apply more/less to career readiness versus college readiness. It was agreed that the accountability system should not have indicators that result in double counting or redundancy of data. Indicators of teacher and principal quality as well as student support systems such as counselors were viewed as important to include in an accountability system. Also mentioned was the work of the Gates Foundation. It has been looking at college readiness defined as academic preparedness, college knowledge, and academic tenacity. The committee should consider process measures for new indicators. The CDE should research and report to the committee what other states have done. For example, Arkansas, Colorado, and New Jersey have incorporated other indicators in their Adequate Yearly Progress (AYP) accountability. It was strongly suggested that the committee do as much work upfront and come to recommendations in advance of the 2015–16 school year.

Public Comment was provided by:

  • Fred Jones, Legislative Information Officer for the CCSS
  • Lee Angela Reid, Senior Legislative Advocate, Capitol Advisors Group, LLC
  • Sherry Skelly Griffith, Legislative Advocate for the ACSA
  • Christopher Cabaldon, Executive Director of Linked Learning Alliance
  • Nicole Rice, Policy Director, Government Relations, California Manufacturers and Technology Association (CMTA)

Keric Ashley summarized the results of the meeting:

  • Most members felt the process of the committee to develop recommendations for SB 1458 should be accelerated by meeting monthly.

  • Committee members look forward to receiving the report of Assembly Bill (AB) 250 regarding California’s new assessment system and further information on SBAC.

  • Committee members want to review the API's guiding principles to use as a basis for continuing forward.

  • Committee members want to review the decisions that were made at the last PSAA Advisory Committee meeting, March 30, 2012.

  • Committee members want a list of currently-collected California Longitudinal Pupil Achievement Data System (CALPADS) data elements.

  • Committee members want definitions of college and career readiness.

  • Committee members want to learn what indicators other states have included in their accountability systems related to college and career readiness.

  • Committee members want to learn about recent research conducted on possible other accountability indicators (such as research on the EAP), data sources of colleges and others, and studies by Michal Kurlaender, Associate Professor, University of California, Davis.

It was suggested by the committee that a matrix of indicators be drafted by the CDE that lists each potential indicator; whether the indicator is used in other states; whether the indicator meets the standards of the API guiding principles; the feasibility of having the data for the indicator; the validity and reliability of the indicator; the added value of the indicator; the pros and cons of each indicator; and any redundancy of the indicator.

The meeting was adjourned at 2:45 p.m.


1 The 2011–12 API is currently posted on the CDE API Web page. The next API cycle is 2012–13, to be posted May 2013 (2012 Base API) and August 2013 (2013 Growth API); however, it is too late to add any indicators for the 2012–13 cycle. There are two remaining API cycles prior to the 2015–16 API cycle (i.e., the 2013–14 and 2014–15 API cycles) when new indicators could be phased into the API prior to the mandated deadline for the 40 percent. New indicators for the 2013 Base API would need to be adopted by the SBE by July 2013, and new indicators for the 2014 Base API would need to be adopted by July 2014. The 2015 Growth API is further complicated due to the implementation of the 2015 SBAC assessments.

Questions: Academic Accountability Team | | 916-319-0863 
Last Reviewed: Friday, February 19, 2016
Recently Posted in Accountability