Guidance for a Quality Improvement ProcessGuidance for implementing a quality, data driven improvement process for afterschool programs.
Guidance for Developing and Implementing a Data-Driven Program Quality Improvement Process for California Expanded Learning Programs
In accordance with California Education Code (EC) Section 8484, all state and federally-funded expanded learning programs serving elementary and middle schools in California shall engage in a data-driven program quality improvement process. This requirement applies to programs funded through both the After School Education and Safety and 21st Century Community Learning Centers grants. This requirement is not required of ASSETs (high school) programs at this time; however, it is highly encouraged.
EC Section 8484(a) further specifies that the department may consider these outcomes when determining eligibility for grant renewal:
A. To demonstrate program effectiveness, grantees shall submit, using the unique statewide pupil identifiers, for participating pupils who are unduplicated pupils, both of the following:
- School-day attendance on an annual basis.
- Program attendance on a semiannual basis.
B. Programs shall submit evidence of a data-driven program quality improvement process that is based on the department’s guidance on program quality standards.
It is noteworthy that program attendance 8484(a)(1)(B) is cited as a key measure of a program’s effectiveness. In fact, poor and sporadic attendance is often a reflection of more systemic problems in a program’s operations. Thus, rather than focus on attendance in isolation, the law makes a direct link between attendance and program quality and requires that programs engage in a data-driven program quality improvement process.
Program Quality Improvement Process
An effective program improvement process is ongoing and involves a continuous cycle of assessment, planning, and improvement. While the process should be carried out at the site level, documentation of this process should be submitted by the grantee.
This cycle of improvement revolves around twelve critical standards—the Quality Standards for Expanded Learning in California—which were developed in partnership between the California Department of Education’s (CDE) After School Division and the California AfterSchool Network (CAN) Quality Committee. They are posted on the CAN Web page, at Quality Standards for Expanded Learning .
Guidance on implementing each step of the quality improvement process appears below.
An assessment of needs consists of the selection of an appropriate assessment instrument; the engagement of key stakeholders in the assessment process; and a critical and reflective analysis of the assessment data.
A. Selecting the Assessment Instrument
The assessment process begins with the selection of an assessment tool. The CAN Quality Committee has reviewed seven assessment instruments that assess program quality and has gauged each instrument's alignment across the twelve quality standards. This crosswalk appears on the CAN Web page, at Quality Standards Crosswalk .
Please note that this crosswalk is not intended as an exhaustive list, nor does any single assessment on the list address all twelve quality standards. In addition, programs are not required to assess their performance on all twelve standards; in fact, it may be more effective to focus more on a few key areas (a suggested target of 1 to 3) each year. It is recommended for sites engaging in this process for the first time to only address one standard. Programs are encouraged to examine a variety of instruments and select one that best suits their needs. They may also develop a tool of their own. Ideally, the administration of any instrument would be supplemented with other assessment strategies, including, for example; focus groups, interviews, and on-site observations.
B. Engaging Key Stakeholders in the Assessment Process
Compose a representative team of stakeholders to be involved in the assessment process. Older students, parents, teachers, administrators, site coordinators and program staff, as well as key community leaders should be consulted to inform the results. Active participation of key stakeholders in the assessment process is important. It will help build credibility and ownership in the results and facilitate needed program changes.
C. Data Analysis and Reflection
Work with a representative team of stakeholders to summarize and reflect on the data. Consider the following questions:
- Are there any clear data trends?
- What are the program’s strengths?
- Which areas require improvement?
- Which areas can be addressed immediately and which ones require longer-term solutions?
Based on the responses to the above questions, identify a few (three to five) high priority needs to focus on that year.
In partnership with a representative group of stakeholders, develop a Quality Improvement Plan to address the high priority needs of the program. The plan should include:
- Specific objectives to address each priority.
- A detailed plan to meet each objective. This plan includes specific and concrete activities for each objective. For each activity, include:
- Specific individuals responsible for carrying out the activity.
- A concrete timeline for completing the activity.
- Estimated costs (if applicable) associated with completing the activity.
- Specific individuals responsible for carrying out the activity.
Critical to the success of any plan is its faithful and thorough implementation. In order to implement the Quality Improvement Plan with fidelity, the following questions should be considered:
- Do all key site personnel understand the priorities of the year and the plan to address these needs?
- Who is responsible for ensuring the plan’s implementation? (Note: This is not necessarily the individual assigned to carry out a specific activity within the plan.)
- How is the plan’s implementation being monitored?
- How is staff being supported to meet the plan’s goals? What professional development and coaching are in place to build the necessary skills in staff to successfully implement the plan?
- How often is the plan reviewed during the year? Is there a process to fine-tune or adjust the plan based on new data or feedback from staff, parents, and/or stakeholders?
Sites should continuously monitor their progress in implementing the Quality Improvement Plan. Is the plan being implemented with fidelity? Is the site meeting its benchmarks? What impact is the plan making? Is it leading to marked improvement in the resources and services that are being delivered to students?
A cycle of improvement means that the process continuously repeats itself. At regular junctures, to be determined by the site (e.g., annually, semi-annually, quarterly), it is important to reassess the needs of the program and begin the cycle all over again.