Transition to a New Assessment System TranscriptTranscript of the live webinar presented on November 14, 2013 from the California Department of Education.
The following is a text transcript of the Transition to a New Assessment System: Focus on Charter Schools Webinar on November 14, 2013 as presented by the Charter Schools Division of the California Department of Education.
Hello. Good afternoon and welcome. I'm Julie Russell, Director of the Charter School Division at the California Department of Education. I want to thank you for taking time out of your busy schedules to participate in the charter school Webinar. There have been several changes in the assessment and accountability system, AB484, Smarter Balanced, Common Core which all impact public schools, including charter schools. The department hosted two meetings this last month the north-south meetings on transition to assessment systems. Many of you may have attended one of those meetings this session will discuss much of the same information but will focus on charter schools and allow an opportunity for you to ask follow-up questions. Today I have with me staff from several divisions here at CDE, they will be providing a lot of background and information on this Webinar topic as we move through the slides if you have any questions please type your question in the Q&A box, to the right of your screen. Staff will be trying to answer as many questions today during the Webinar as possible there are also many frequently asked questions that have already been posted on the CDE Web pages and we will continue to add questions to that link as new questions come up. Also there's a number of resources that are available, we will be sharing some of those with you today during this presentation, and this Webinar will also be posted on our Web page following the presentation, not immediately following, but soon thereafter following the presentation. The focus of this Webinar is to discuss the transition to the new assessment and accountability systems. Another significant change this year was the Local Control Funding Formula or LCFF, which includes a Local Control Assessment Plan or LCAP. However we will not be discussing LCFF today, there's a Web page on CDE's Web site for LCFF that is updated frequently and I suggest if you haven't already, that you get on that Listserv so that you can see the updates as they're posted. State Board staff, CDE staff and local stakeholders are working diligently to develop regulations templates and guidance documents for LCFF. We will plan to host another Webinar in the near future for charter schools on that topic as more information is available. So with that, we'll begin our session for today and I'm going to turn it over to John Boivin who will get us started.
Good afternoon. My name's John Boivin. I'm the administrator for the STAR testing program and I'll give you an overview of AB484 providing you with some information about what would be administered in 2013-14 and then 14-15 and what the future holds for us in regards to assessments. Our hope is to provide you not only with the nuts and bolts about what is expected of you and of local educational agencies but also some other rationale and background for why we're doing what we're doing. Assembly Bill 484 establishes statewide student assessment system. This student assessment system is referred to as CalMAPP or California Measurement of Academic Performance and Progress. 484 outlines the assessments in CalMAPP, some of which are STAR tests which will continue in order to meet federal mandate but introduces Smarter Balanced assessments, or the computer based assessments and then AB484 also allows for schools, districts to focus on the Common Core State Standards and to move ahead and make progress on the Common Core State Standards. The primary purpose of CalMAPP is to promote high-quality teaching and learning through a variety of assessment approaches an item types. The intent is to improve teaching to the full curriculum one of the large criticisms of STAR was that it narrowed the curriculum and that teachers were teaching to the test. The hope with Assembly Bill 484 is that we broaden that curriculum and teachers would then teach to the full curriculum and also 484 allows for a transition over time. AB484 requires participation in the Smarter Balanced field test in 2014. This is an opportunity to test the test, this isn't an assessment for students, and this is to gauge the validity and reliability of the test items and to allow for psychometric studies and analysis. The results of the field test will not be used for any kind of scoring or student assessment or student evaluations. This is an opportunity to experience the computer-based test in an environment that's low stakes that also provides for an opportunity to gauge our technology readiness in preparation for 2015. Consider this sort of a dry run for 2015. Assembly bill 484 also provides interim informative tools at no cost to the LEA's, so schools, charter schools, will have an opportunity to use these tools in the classrooms. It allows for an analysis of the impact of a computer-based test on the EAP, and the EAP is the Early Assessment Program that’s currently run by the California State University System. Something that's new also in Assembly Bill 484 is that it exempts English learners who are in the US for less than 12 months from taking the English Language Arts assessment and this is in alignment with federal law. By March 1, 2016 the Superintendent of Public Instruction will take recommendations to the State Board of Education to expand CalMAPP. The idea is that we would meet with stakeholder groups to develop these recommendations and consider additional assessments such as end-of-course assessments in English language arts, mathematics, history, social science assessments, but also as indicated in law what it’s doing is opening up the opportunities to non-traditional assessment such as visual and performing arts. This is an opportunity for us to explore a variety of item types and assessment modalities, which includes things like matrix sampling or population sampling. In 2013-14 there are assessments that are required, those assessments are certainly the Smarter Balanced field test and Jessica Valdez will follow me with a lot more information about Smarter Balanced field testing, but also what is required is for science, science in grades 5,8,10, in grade 10 it would be the life science test for CAPA for CMA, CFT. Also for CAPA what would be required is English Language Arts and mathematics in grades 2-11, levels 5-1. What's voluntary in 2013-14 would be the EAP, or Early Assessment Program in grade 11 and this has always been voluntary for students in grade 11. Students in grade 11 who wish to participate in the EAP would take the English Language Arts Writing assessment as well as the CST multiple-choice. The writing assessment will be offered, traditionally its offered in the month of March, this year it will be offered from the beginning of February all the way through March in order to provide more flexibility for students in grade 11 who would also have to take the Smarter Balanced field test, so we're offering some flexibility there in regards to the EAP. For the EAP all that would be available are the student results and so this is the CSU results which shows the student readiness for English Language Arts and then if a student is eligible to take mathematics it would be algebra 2 or summative high school mathematics. Also voluntary is the standards-based test in Spanish and that assessment will be available in grades 2-11 for RLA only and this is for English learners in the US less than 12 months or receiving instruction in Spanish. In 2014-15, the science assessments will continue until a successor assessment is implemented, this will also be true for CAPA. The EAP in 2014-15 will be replaced by the consortium assessment. AB 484 requires that the FPS be replaced by a Common Core State Standards aligned assessment by 2017. This is more information about expanding the CalMAPP as I talked to you before, this is, so what will be required is that the CDE convene stakeholder groups to explore different types of assessments and to look also at locally stored performance tasks or portfolios so we're really taking into account a number of different avenues for assessment or evaluating student progress. For 2013-14 and 2014-15 the traditional STAR test that would not be used, like the science test and the CAPA, are available to school districts at the cost and the cost would go to the school districts so you'd be able to purchase these assessments, the old STAR tests if you will, for a marginal cost, but again it would be at the LEA's own expense. Educational Testing Services, our testing contractor, is in the process of building out an ordering and billing system in order for LEA’s to do that work. Something else that was modified in AB484 is the definition of LEA and this definition of LEA was done in order to align the assessment reporting with the accountability reporting, so as you can see the local educational agency means a county office of education, school district, state special school or direct funded charter school as described in Section 47651. As a final reminder all charter schools must participate in the field test and the assessment, all students must be assessed and in the future scores will be produced to hold schools and LEA's accountable for the achievement of their pupils. I'd like to turn it over to Jessica Valdez who will provide information on Smarter Balanced field tests.
Thanks, John. Good afternoon everyone. I'm Jessica Valdez. I'm the administrator for the Assessment Transition office here at the Department of Education and the transition office is responsible for helping to coordinate California's involvement in the Smarter Balanced assessment consortium. So today I'm here to provide you with an update on current assessment development activities as well as an overview of the assessment system itself and so a good place to start is with this diagram which outlines the three-components of the Smarter Balanced system of assessments. It’s important to note that this system of assessments is quite a bit different from the English language arts and math assessments that were part of our prior assessment program, the Star program, assesses a different set of standards, the Common Core State Standards in English language arts and mathematics. As John mentioned it's a completely computer-based assessment system and there are more components to the Smarter Balanced system of assessments. So the first, the primary component, of the Smarter Balanced system of assessments are the summative assessments, so these are the end of the year assessments that are used for accountability purposes, they’re provided in English language arts and mathematics, for students in grades 3 through 8 and grade 11, but two more components to the system are the interim assessment and the formative assessment tools and practices. So 484 require the state to purchase the interim and formative assessment components and offer them at no cost to California LEA’s. These are two completely optional components of the system it is really at the discretion of the LEA to determine whether they're going to use these assessments and how often they'll use these assessments. I first wanted to talk about the interim assessment component; there are actually two pieces to the interim assessments. The interim assessments will be available for students in grades 3 through 11 and there will be a comprehensive assessment that will pretty much mimic the summative assessments. They'll use the same test blueprint as the summative assessments and there will also be assessment item blocks as part of the interim assessment component. These will be basically clusters of items that will assess just a small group of standards. It's important to note that the items that will be developed for the interim assessment component will be developed using the same criterion standards used for the summative assessment items, but again the interim assessments will be solely for use locally at the LEA level and the state will not be collecting any results from the interim assessment component. The third component of the Smarter Balanced system of assessments is the formative assessment tools and practices, and this is going to be essentially a database of resources for K-12 teachers to help gauge the extent to which students are progressing in learning the standards that they're being taught they’ll include instructional modules and I'll speak a bit more about the ongoing development activities for the formative assessment tools and practices. The digital library, which you may have heard about, is the formative assessment component of the Smarter Balanced system and it's scheduled to launch in April of 2014. I may not have mentioned that the interim assessment component is scheduled to launch in fall of 2014 and you can expect that these two components will be developed over time, particularly over the first couple of years of the launch of the new assessment system, so the goal of the new assessment system is really to use these three components of the system together to help teachers and schools get the information that they need to improve teaching and learning with of course the ultimate goal being that all students leave high school ready for career and college. So right now the most significant activity that's going on in the development of the Smarter Balanced assessments is the preparation for the spring 2014 field test which as John mentioned, Assembly Bill 484, requires California LEA's to participate in, and I’ll speak more about the particular participation expectations in just a moment but first wanted to start off by really emphasizing the purpose of the field test. There was a quote recently from Assistant Secretary Deb Delisle for the US Department of Education and she conveyed the purpose of the field test very, very clearly and concisely saying that a field test is not designed to be a valid and reliable measure of student achievement, rather it is designed to help the test developers evaluate whether the tests individual items and the technology platform work as intended before the first operational administration and we really can't emphasize this point enough. Assembly Bill 484 specifically prohibits use of the upcoming field test for any other purpose such as accountability purposes or measuring student achievement. So to the extent that you can help us convey this to school staff we would greatly appreciate it and as I mentioned there will be no results released from the field test and at the same time it's quite important that all participating LEA's and students take their participation very seriously because one of the primary purposes of the field test is to collect item data to help inform test development and standard-setting. There of course are benefits for all involved in the field test, students will get that hands-on experience with the functionality of a computer-based system, the teachers and administrators will get exposure to the administration logistics during a low stakes environment, during a trial run, and the participating LEA's will benefit from having that time to identify where gaps may exist, particularly in the areas of technology and to help prepare them for the operational assessments which will begin in the 2014-15 school year. So the field test will take place between March 18th and June 6th of 2014. CDE in conjunction with its testing contractor, Educational Testing Service, will be assigning schools shorter test windows within that time frame and that's something that's being worked on right now and we expect to be able to release that information to LEA's soon. So there are different participation expectations for different groups of students so we laid them out here on this slide and just keeping in mind that these participation requirements are for all public schools, including charter schools. So Smarter Balanced, before I get into the participation requirements, I do want to acknowledge that Smarter Balanced is going to be identifying a scientific sample of students to participate in the field test and it'll be about 10 percent of students for English Language Arts and about 10 percent of students for mathematics but with the passage of 484 far more students beyond that scientific sample are required to participate. So for grades three through eight, all students are expected to participate in the spring field test. For grades 9 and 10, only students selected for the scientific sample will be expected to participate and just a very small percentage of students in grades 9 and 10 will be expected to participate. For grade 11, this is sort of a special group of students because as John mentioned in his highlights of 484, the law does require that grade 11 students still have the opportunity to participate in the early assessment program. So for grade 11 students the expectation is that all students selected for the scientific sample participate. All grade 11 students beyond the scientific sample may participate and keep in mind that school sampling will be done at the school level by grade so it would be an entire class of students for a particular grade that would be selected to participate in the scientific sample for every single grade. Now the only exception to the requirements that you see on this slide are for students who currently participate in the CAPA, California Alternate Performance Assessment, and for recently arrived English learners there's the exemption from participating in the English language arts portion of the field test. Otherwise the requirements as you see them on this slide stand. We just really want to emphasize that all grade 11 students even those who participate in the field test are to have the option to participate in the early assessment program. Now each participating student will be assessed in only one content area either English Language Arts or mathematics. Content area is going to be assigned by CDE and testing contractor ETS as I mentioned for each school by grade. So if we have an elementary school, grade 3, would be selected for either English Language Arts or mathematics. Grade four for either English Language Arts or mathematics and so forth. Within a particular district it is not necessarily going to be the case that all grade 3 students at every school would be selected for the same subject so the subject assignments will vary by school but the same subject would be assigned for the entire grade within the school. As I mentioned there is going to be that scientific sample of students that is going to be selected by Smarter Balanced in California there about 680,000 California students in the scientific sample and it's the data from this sample of students that is going to be used to determine the item reliability and validity and will be used to help inform test development and standard setting. So the field test will take approximately three hours for students to take and that's from beginning to end. The English language arts assessment is a bit longer, but in general you can count on about three hours of testing time. There's not going to be any paper and pencil version of the field test available. Assembly Bill 484 does require that for the operational assessments which begin in the 14-15 school year that a paper-and-pencil version be available for the first three years. However the purpose of the field test this school year is really to test the computer based administration system so there’s really no paper-and-pencil version that will be available for the field test. As mentioned earlier there will be no student, school or District score reports produced for the field test and results are not going to be factored into any state or federal accountability calculations as I mentioned 484 specifically prohibits this and after my presentation you'll hear a bit more about how 484 impacts accountability calculations. Please note that CDE does have a Smarter Balanced field test Web page available including a set of questions and answers which are updated regularly so we encourage you to check those out. So I'm going to move on, now switching from the field test to the operational assessments which as I mentioned begin the following school year in 2014-15. This past September, Smarter Balanced governing states approved the operational testing windows and there's a separate window for two different groups of students. The testing window for students in grades 3 through 8, the summative testing window, can begin only after 66 percent of the schools instructional days have been completed. For grade 11 students its eighty percent, so for grade 11 students summative assessments can begin only after eighty percent of the school’s annual instructional days have been completed. Now it's important to emphasize that individual states within the consortium can establish more specific Windows within that time frame that has been approved by governing states and it is very likely that CDE will be taking regulations to the State Board to establish more specific windows for California LEA's. So I mentioned at the beginning of my presentation the third component of the Smarter Balanced system, the formative assessment tools and practices. And those are being housed in the digital library that's under development by Smarter Balanced, as I mentioned the digital library is going to contain formative assessment strategies and professional learning and instructional resources for educators to help gauge the extent to which students are grasping the standards that are being taught to them and really the intent is that these be resources that teachers can use on a daily basis in the classroom to get immediate feedback. Just to reiterate, that access will be available to the digital library for all California LEA's at no cost and the library is scheduled to launch in April of 2014. Smarter Balanced is ensuring that all resources that go into the digital library meet quality criteria and I've listed a few of them here on the slide as far as they have to incorporate high-quality formative assessment practices, reflect differences in learning and support personalized learning and they have to demonstrate utility and engagement and user friendliness. Now the way that Smarter Balance is ensuring that all of the resources in the digital library meet these quality criteria is through the establishment of a state network of educators in each member state. In California we have 150 educators from throughout the state that are a member of our state network of educator team, and that team of educators is responsible not only for reviewing resources that are submitted against the criteria that have been established but also submitting resources themselves. So in addition to the formative tools and practices, the digital library is going to contain Web-based professional learning and instructional modules on a variety of topics I've listed a few of them here. The Common Core State Standards, assessment, literacy, understanding Smarter Balanced content specifications and formative assessment processes within the context of the Smarter Balanced assessment system. A document that we wanted to be sure you are aware of is actually a document that was approved by governing states back in November of 2012. These are the preliminary test blueprints, the blueprints of course being crucial to test development because they really provide the guidance for test development, including the test content, the number of items, the types of items, the score points that will be assigned to items and even contain the estimated student testing time for each subject by grade band. I want to emphasize that the test blueprints are considered preliminary until after the completion of the field tests. Smarter Balanced anticipates collecting findings from both this past spring's pilot test and the upcoming field test and using that to fine tune the test blueprints but the preliminary test blueprints in their currently approved state are available on the Smarter Balanced Web page at the URL at the bottom of the slide. Now switching from paper and pencil assessments to the Smarter Balanced computer-based assessments really opens up the ways in which students can respond to items so this slide lists a variety of the item response types that will be found on the Smarter Balanced assessments and will be able to be experienced firsthand by students in the upcoming field test, but the best way right now all for students, parents, anyone really to experience these different item response types is through the Smarter Balanced practice tests. These are the tests that were launched on the Smarter Balanced public Web site this past May, there were some enhancements added in August 2013 including scoring ties for each of the practice tests. Again this really provides the public with an opportunity to get familiar with this new testing environment that we do want to emphasize that the practice test really should not be used to guide instructional decisions regarding individual students. Now there is a practice test available for English language arts and mathematics for each of grades 3 through 8 and grade 11 and there are about 30 questions per test including a performance task. Shortly after the practice tests were released by Smarter Balanced, CDE developed a letter template to assist LEA's in announcing the availability of the practice test to parents and guardians and the template's available in English, Spanish and in a number of other languages on the CDE practice test Web page which is listed in the second bullet on this slide. Now for California LEA's there is a specific helpdesk for not only the practice test but the field test and questions are fielded from our testing contractor ETS, so contact phone number and email address are at the bottom of this slide and again that's for both the practice test and the field test. Another resource that I wanted to let you know about quickly is a document that was approved by governing states this past September and if the Smarter Balanced usability, accessibility and accommodations guidelines. The intent of these guidelines is that they be used beginning with the spring 2014 field test and this document outlines three types of support that are going to be available to students on the Smarter Balanced summative assessments and the first type of student support are universal tools and these are tools that are going to be available to all students, there are embedded tools and non-embedded tools. Many of the universal tools are embedded; examples include spell check, highlighter, embedded ruler and so forth. Again these are available to all students and many of these tools which are embedded, the students can turn on and off and enable or disable as they please. The second type of support are designated supports and these are supports that are specifically for students who have been identified as requiring this sort of support when taking assessments. They would need to be identified by an educator or instructor, someone who's familiar with the students’ needs when taking state assessments. Again there are embedded and non-embedded designated supports, some examples are listed here color contrast text, blocking of distracting content, so forth. Now the third type, the third category, of support that's available to students are accommodations and this group of supports is specifically for students with an IEP or 504 plan that have the need for such an accommodation listed in that plan or program, examples include closed captioning, braille, embedded calculator and describe. There are many more supports in each one of these three categories that are listed on these slides so I encourage you to check out the guidelines in their entirety at the URL at the bottom of the slide. And then the last resource that I wanted to bring to your attention is a Smarter Balanced Spanish Web page which was recently launched by the consortium it’s a great resource for Spanish reading audiences, contains downloadable fact sheets for teachers, parents and students and also provides the description of research that really helped design the assessment items and reduce linguistic burdens. So the URL for that Web page is listed at the bottom of the slide. If you're not already a member of the CDE's Smarter Balanced ListServ I highly encourage you to do so just need to send a blank email to the address at the top at this last slide here and always feel free to contact my office, the assessment transition office either by email or phone and the Web addresses for both the CDE Smarter Balanced Web page and the consortium main Web site are listed here and with that I’m going to turn it over to Keric Ashley to provide you with an update on accountability.
Thank you, Jessica. This is Keric Ashley, Director for Analysis Measurement and Accountability Reporting Division. My division's responsible for calculating Federal Accountability, what we call adequate yearly progress or AYP. State accountability which we refer to as academic performance index or API and then some other data reporting that we do specifically Dataquest, the school Accountability Report Card template is done through my division as well as school quality snapshots. So now that you've heard about the new assessments both the field test in the operational tests, let's talk a little bit about how these new assessments are going to impact accountability. There are actually three bills that have an impact on our accountability moving forward. The first before we get to the most recent assessment bill that’s already been talked about last legislative session SB 1458 was passed, that had in it a big impact on high school API, and so if you're in an elementary school or middle school you can take a nap for the next couple of slides. So what SB1458 says is that for high school API’s beginning in 15-16 the 2015-16 school year, the API can only be made up of no more than 60 percent of the API based on assessment scores or assessment results only 60 percent could be less than that but no more than 60 percent which means we need to identify at least 40 percent of a high schools API through other indicators such as graduation data or college and career readiness so that's by 2015-16. We do have a public school Accountability Act Advisory Committee and that advisory committee has made a recommendation to the superintendent about a proposed method for including graduation data into the API and what you see on the screen is that proposal, which looks at each individual student who has gone through a four-year cohort and in that four-year cohort if they graduated with a diploma they would earn 1000 points toward their schools API. A special education student who has earned a certificate of completion would also earn thousand points. A high school equivalency test or the GED would receive some credit, not as much as a graduate, but we thought it was worthy of them receiving some credit for that accomplishment and they would receive 800 points non-graduates would continue to receive 200. Along with those points structures in the API there’s an incentive for graduating students who are in disadvantaged groups. In other words bonus points will be awarded in three specific categories: English learner, students with disabilities and socioeconomically disadvantaged students. Each of those students would earn bonus points of fifty and if they're in two groups then they would earn a hundred points and if they happen to be in all three of those groups they would earn an additional 150 points which means that student would earn 1,150 points toward their schools API. So this methodology has been approved by the public school accountability act Advisory Committee but it is yet to receive board approval. Along with the graduation data, the public schools Accountability Act Advisory Committee is also discussing a college and career readiness indicator as well as some possible other indicators to be included in the API. The state board is interested in discussing not only graduation data, but these other indicators in context with the local control accountability plan as many of you may be aware the local control funding formula in the budget bill AB97 requires an accountability plan in which data is part of that plan so the state board is interested in discussing how the data in that local control accountability plan and the data included in the API relate to one another whether they want it to be duplicated or whether the they want that data to be in either the accountability plan or the API so those are decisions to be made by the State Board in the future. So let's get back to the assessment bill AB484 that was just recently signed by the governor last month. You've already heard that the main purpose of this is to improve teaching and learning in to assist our teachers and administrators in identifying the needs for their students and then again to be used to not only promote high-quality teaching but then that data will eventually be used for accountability as well. So 484 does replace our current CST or the STAR system with CalMAPP assessments, which means for accountability the CalMAPP assessments will also be replacing STAR in the future. But AB484 had a couple of very specific prohibitions; one is not using the field test results for accountability purposes. You've heard this already but it doesn't hurt to repeat it. This is a low stakes year in which the field test will be given for purposes of testing the test and a schools ability to administer the test through their computers. It is not to be used; the results are not to be used for any purpose other than to see how well they can administer these assessments. It also prohibits the comparison of the results for next year's assessments to California Standards test so we will not be using future results that you would get from the 14-15 operational administration of CalMAPP. We will not be comparing those results with the CST's, that’s prohibited specifically in AB484. If we do not have the ability to calculate an API for a school that does create some challenges for us. There are about forty Education Code references that refer to the use of the API, whether that's API scores, whether that's the state goal of 800, it might refer to the rankings that are given by a school either statewide rankings or similar school rankings but there are about forty references in law and some additional references in state board regulations to use the API for one or more purposes. So AB484 does recognize that if we cannot calculate an API next year that there are some alternatives to doing that. The first of which would be to use the most recent API, so we have calculated an API and posted it just recently in the last six weeks that API could then be used the following year. Another alternative will be an average of the three most recent years of API so that's data that my division has available and we will be calculating a three-year average just in case that needs to be used for one or more programs. If the most recent API doesn't work and if an average of the three most recent API's doesn't work then there is a third alternative and that is, and this is right out of the law, to use alternative measures that show increases in pupil academic achievement school-wide and for all student groups. So any other type of assessment whether it be benchmarks, benchmark assessments or some other way of showing an increase in pupil academic achievement can be used in place of an API if we are unable to calculate one based on the field test data which of course won't be used for API calculations. You could also use a combination of these three, so in other words a program might be able to say well we will use the most recent API for making a determination or an average of the three, of the last three years of API or the greater of the two, or the lesser of the two depending on what the program is. So there is possibility to use one or two or all three of these in conjunction with one another to make a determination whether it's a determination to fund a program or to be eligible for a program or perhaps if it's needed for charter renewal or revocation these alternatives were put into law for that purpose. So let's talk about Federal Accountability, AYP, the federal government is making available double testing flexibility waivers that states can apply for. They would have to apply for it by November 22nd and should California apply, what this would do is the double testing waiver allows states to not have to test students in two different assessments and in our case California Standards test in and the new CalMAPP field test. Of course we've already made the determination by the state that we will not be giving the CSTS, at least in most cases, one example would be: grades 5, 8 and 10 science CST continues. So it'll be very difficult to make a determination for AYP based on results that we won't have, so in that case we would then carry over the API and any program improvement status that goes along with it from the previous year's results, in other words schools would freeze in place. So if a school was year two program improvement and we weren't able to calculate AYP, which is very unlikely will be able to calculate one for elementary and middle schools because we won’t have test scores, in that case the school would remain Year two program improvement and they'd be responsible for continuing to provide whatever extra services those students would receive now. In other words they would stay in program improvement year two, no schools will advance in program improvement, no new schools would be identified for program improvement and of course no schools could exit from program improvement either. AYP for high schools is different altogether. We will be calculating AYP for high schools, for those of you who may not know, for federal accountability we use the CAHSEE, as well as CAPA in English language arts and math to make our adequate yearly progress determinations, along with graduation rates. So we can and will calculate AYP next year for high schools, based on those same criteria. In the future we may want to move away from CAHSEE and use the new smarter balancer or CALMAPP assessments, but for now we have CAHSEE and will use CAHSEE to calculate the AYP determinations next year for high schools, which means of course that new high schools can be identified for PI, they can advance and of course they can exit as well. So let's move to API, the Academic Performance Index, our state's accountability system. Since again we have CAHSEE data for high schools we could produce an API for high schools now notice and on the slide that could is underlined, doesn't say we will calculate the API for high schools next year it means that we have assessment data that would allow us technically to do this but this is a state board decision and it would be a recommendation that would have to come from our public schools Accountability Act Advisory Committee that would then be taken to the State Board for approval. So we have the data available, we could use CAHSEE, even without CST scores to produce a high school API but that decision has not yet been made, that’s a decision that at some point this spring will go to the State Board for approval. The last bill is of course the budget bill which created the local control funding formula as well as the local control accountability plan but that budget bill, AB97, also made some very specific changes to our state’s API. One, it reduced the numerically significant subgroup threshold to 30 students, in the past it was one hundred students, so this means more student groups will be included in the API for different schools bringing that threshold down to thirty students. It also added a new numerically significant subgroup and that's the subgroup for foster youth, so this is a new student subgroup that will be added to API and the law specifically says that the threshold for that student group is a size of 15, so unlike the other student groups that are being reduced to 30, foster youth will be considered numerically significant if there are 15 students at a school. The one last word on the foster youth data, CDE and the California Department of Social Services are currently working on an MOU that would provide a data match between our data system here at the state Department of Education called CalPADS and we would link that data with Department of Social Services data, DSS, they have the most recent an up-to-date data on which students are foster youth through that data match we will then provide reports back to schools and districts as to who their foster youth are. Right now schools may or may not know that they have students in their school that are foster youth and if schools are going to be held accountable for them it's only fair that we let them know which of those students are foster youth to allow them to focus some extra services for those students. So the contact information for my division is on the screen, and is cleverly not my phone number and not my email, but it will get to those in my division who can answer your questions.
Thank you, Keric. Hello. It's Julie again. We have several staff going through and answering the many questions that are coming in so hopefully we'll get to the question you ask either by the end of the presentation or it will be posted on the frequently asked questions, after the Webinar. There's also some follow-up questions on when will the PowerPoint be posted, so we are planning on posting the recorded Webinar session, we're going to post it on our charter school Web site we have a listing of everybody that's participating, your email, so when we get that posted on our Web site we will send out an email letting you know the link so that you can go ahead and log on access it. So with that we have two more sessions that were going to be going over, so a lot of that was the program, the AB484, the API, AYP. So now we're going to get a chance to do the technology requirements and there have been a lot of questions so hopefully this will answer many of the questions on technology requirements and IT readiness and then we're going to do CalPADS. So I'm going to turn it over to Rodney Okamoto.
Good afternoon everybody. My name is Rodney Okamoto. I'm going to talk about technology readiness as it applies to both the field test and the operational system. So this is just a fun slide about how we've been doing statewide testing for a long time where it's a multiple-choice fill in the bubble type of response. We do have a writing sample on the CST but its written on paper and it's graded on paper. So what we're going to go to now is a computer-based assessment system so this slide shows from a graphic standpoint a high-level overview of the system. So I Just want to point out the major pieces to this diagram here, so at the bottom there's a student, in his nice looking red sweater, using a computer connected to the internet. This student’s computer will be running a custom software application called the Smarter Balanced Secure Browser. So this software application is different from a standard Web browser such as Internet Explorer or Firefox or Chrome or Safari it's a custom application that's installed on the student’s computer. Up in the middle-right part of the diagram is a test proctor, or a test administrator, this proctor is using a computer connected to the internet. One of the differences between the test proctor's computer and the students computers, is the test proctor's computer is using a standard Web browser such as Internet Explorer or Firefox or Chrome to administer the test so that this person will be monitoring the progress of all students that he or she is responsible for. Then up on the upper part of the screen is a depiction of servers also connected to the Internet, so this is showing that both the student’s computers and the test proctor's computers are connected to the Internet and accessing some centralized servers also on the Internet. So a couple bullet points on what technology is needed, just from a general standpoint, the most obvious is computing devices, both for the student and test administrator. Along with those computing devices, there are accessory requirements and I'll go over that later on. Network bandwidth, because the student's computer needs to connect to servers on the Internet, so both from an internal network perspective for example wireless access points and also bandwidth to the Internet. The soft costs for technical support and training, so technical support from a technology standpoint, and also training for test administrators from a technical standpoint and then facilities you know adequate space, physical space, electrical power, are you able to run cabling to certain places at your location and so forth. So now from the student’s computers perspective I'm going to go over the minimum operating systems. So Windows XP SP3, SP stands for service pack, that's the last update to XP, came out in 2006 I believe, XP has been around since 2001 so you could have a fairly old computer still running XP Service Pack 3 that will work for the test. Smarter Balanced is proposing a phase-out date for Windows XP at the end of the 2015-16 school year and just a note here from a general technology standpoint, Microsoft itself is ending support of Windows XP in April 2014, so if you have XP machines at schools right now, there should be a plan in place to upgrade those machines, upgrade the operating system at least, because after April 2014 Microsoft will no longer support that operating system which from a network security standpoint means no security patches and any vulnerabilities in Windows XP will not be fixed by Microsoft. So XP is the minimum Windows operating system that means newer operating systems such as Windows Vista, Windows 7, Windows 8, and their recently released Windows 8.1 are all supported. I do want to note that there are tablets that run an operating system called Microsoft Windows RT and that’s different from Windows 8 so a tablet can run either Windows 8 or Windows RT. If you're interested in a Windows based tablet you want to get the tablet that runs Windows 8 because at this time Windows RT is not supported by the Smarter Balanced assessments. From a Macintosh standpoint, 10.4.4 is the minimum, that operating system has also been out for many years now. There's a proposed phase-out at the end of the 2014-15 school year and that kind of goes with the next version because 10.4.4 and 10.5 both are based on a processor chip known as the PowerPC chip from Motorola and to continue to support that chip causes a lot of development efforts so post phase-out is the end of the 2014-15 school year and similarly to Microsoft Windows the operating systems for Mac OS 10 after 10.5 are all supported so 10.6 all the way up to 10.9 which is the latest operating system called Maverick for Apple those are all supported. From an iPad perspective the operating system that runs on iPads is known as iOS, 6.x is the minimum version that has the guided access feature to secure the application that was released September of last year, recently Apple released iOS 7 so either iOS 6 or 7 if you're going to use iPads. If you're interested in using Android devices that one gets a little trickier because there's no one type of Android operating system. Android is known as an open source product where a manufacturer can download the source code and make changes to it and put it on that manufacturer’s tablet. So even though Google developed Android and continues to make changes, a manufacturer such as Samsung or Lenovo or Sony can download Android and they do this, and make it work on their hardware. So the point here is if you're considering an Android device it has to be Smarter Balanced certified and I have an email address later on where we can give you information about that site so that the manufacturer you're interested in can be notified, and both for Apple an Android tablets, and you know Microsoft has tablets themselves, an external keyboard is required. So the use of the virtual keyboard takes up valuable real estate and the items are designed to use the complete display of all the devices. Recently there's been a lot of interest in devices known as Chromebooks so this looks like a lightweight laptop but instead of running Windows or MacOS 10, it run's an operating system known as Google Chrome OS. So a lot of schools are getting Chromebooks they're relatively inexpensive compared to a Mac Book Air or Ultrabook Windows machine, so Chromebooks are supported for the Smarter Balanced system. A few schools use an operating system called Linux, its freely available you might have to pay a little bit for support but instead of running Windows on desktops or laptops they might be running Linux. VDI stands for Virtual Desktop Infrastructure you might hear terminology such as zero client or thin clients those are also supported, and just another note on Windows RT, so that operating system is not supported. So a good test on your student devices is to try the Secure Browser application and access the practice test because that's a close simulation of what the field tests and operational test will be, so if you have an unusual setup with for example virtual desktop infrastructure, the best thing to do is to try to load up the practice test or access the practice test using the Secure Browser application. Here's some of the requirements from the student perspective, ten inch diagonal display, so that's your standard iPad tablet, is 9.7 inches, so that's a ten inch class many tablets are available in a ten inch display. 1024 by 768 resolution, that’s also the iPad 2&3 resolution, it's a lot better now, or greater now, but 1024 by 768 is an iPad 2 resolution. Some older what's known as netbooks they look like laptops but they're a little smaller form factor called netbooks They may not meet that requirement so if you're planning to use netbooks please send an email to that email address I'm going to tell you later so there's a special email address for IT related questions and we can forward some information that you may be able to use if you're using an older netbook with Windows seven and if that netbook doesn't meet the display resolution there may be a workaround. Physical keyboard, I mentioned that before for tablets, also be aware if you're planning to use Bluetooth keyboard there is possible interference issues if your wireless data network runs at a frequency known as 2.4 gigahertz so if you have, well let's say 30 Bluetooth keyboards in one room and your wireless network runs at 2.4 you want to test that to make sure there isn't any interference issues. Currently there's no stylus or drawing device requirement, it’s possible in the future that there may be some items, specifically math items, more complex math items where the student will need to so-called show their work to figure out how they got to the answer but at this time we haven't been given any time frame as to when those items might come about. Headphones for the English Language Arts test, there are some items that have audio clips, and if the student has the text to speech accommodation, headphones are required so that the audio coming out of the computer won’t distract other students nearby. A printer would be connected to the test administrators computer either via network mapping or local connection so if a student has an accommodation where the item needs to be printed on hard copy a printer needs to be available in that test session. So now let's talk about the Secure Browser application, this is a software application that is installed on each student's computing device. A new version does come out at the beginning of each school year so you have to make plans for having ways to deploy that application, so for the field test the anticipated release date is Monday actually, November 18, when the field test version will be released, the Smarter Balanced Secure Browser field test version, will be released. If you have the Secure Browser installed because maybe you were trying it out or maybe you were part of the pilot, you’ll need to either uninstall that or if it's on a Windows machine we found out that the field test version will uninstall the pilot test version so you won't have to worry about uninstalling on a Windows machine. So the main takeaway here is a Secure Browser Web application installed on each student’s computer and updated at the beginning of each school year. Again I mentioned that this is a good way to access the practice test, in fact since this slide was made there's a slight change where when you install the field test version of the Secure Browser and launch it, it’ll send you directly to the practice test so you won't have to worry about finding the link to the training test. If you want to experience text to speech or the Braille option you need to access the practice test through the Secure Browser, as Jessica mentioned the practice test is also available through a standard Web browser, but if you want to experience, for example, text to speech you need to use the Secure Browser. One of the main reasons to have a Secure Browser application is to make sure that the test is secure, for example once you start the Secure Browser you can’t jump out of it and start another application such as a Web browser or calculator. In fact both at startup and during execution of the Secure Browser application it checks for a list of so-called prohibited applications and if it notices maybe a screen capture utility running in the background or Web browser was started it won't let you continue with the test. From a network balanced perspective there's an estimate of 20k bits per second average for each student connected to the system and I know that by itself doesn't mean a lot, I do have a slide later which shows the calculator that estimates the percent of your current bandwidth that would be utilized given all the computers accessing the test so that’ll be more clear when I get to that slide. There's a spark in network activity at startup and remember at least for operational it's an adaptive test so there’s some prefetching going on for future questions. It's always hard to estimate the network requirement because it’s depending on how fast students answer questions, on some of these ELA questions it's multi-part responses so you might read a five paragraph passage on the left hand side and have six questions on the right hand side to go through so it could take a while between the time that the student actually generates network traffic as they are reading the passage and answering questions. So I know a lot of schools are using the Secure Browser on the practice test to try to get a gauge on network activity. So when you add more devices that increases your internal network hardware requirement so if you add more tablets to your environment, more wireless access points or more desktops you might need more access layer switches and then remember internet bandwidth is shared typically from one school location out so as you add more devices it will impact your internet bandwidth. So this diagram gives a high-level overview of a typical school network to get a better understanding. On this picture here, on the upper part, we have desktop computers, Mac or PC, that are hard-wired to the network with a network cable. When you start adding tablets and wireless laptops and Chromebooks those are wireless devices, they don't connect directly to your network but they go through a wireless access point so this is showing that the more wireless devices you have you probably need to purchase more wireless access points. Then finally the last piece here is that Internet bandwidth is shared from the school site so as you add more devices that might impact your internet bandwidth also you want to make sure when you do assessments and if you think you're going to have internet bandwidth issues to make sure there's no other students doing non assessment work that maybe can be delayed to another day or that your business staff isn't doing some big upload of data during that time frame when you're doing assessments. So are schools ready technology wise? There is a tool called the Smarter Balanced Technology Readiness Tool and if you don't have access to this tool we’ll definitely set you up with a user account. You do need to add pretty specific information about the type of devices and network capacity at the school level so it could generate the reports to get the figures out if you have enough computers and if you have enough network bandwidth. You also enter the number of days that you expect to test, your testing window, and the number of sessions per day that you think you're going to run. Here are some benefits of the TRT, you get these nice school level readiness reports and I have a screenshot later on, you can use it to justify spending decisions, also it allows the CDE and counties to identify schools that might have insufficient technology. There is a so-called snapshot on December 13th, where all this information is aggregated up so if you're in that situation where you don't think you have enough technology, one-way to inform the state is to complete the technology Readiness Tool by December 13 so once the snapshot is taken we'll be able to run reports and find these schools that have insufficient technology. Here's a sample report, it's a test district with a list of schools in that district, there's a column to the right which is the number of students, the percent of students, that can be tested given the network capacity at the school and then second from the right is a column to say, the percent of students who can be tested given the number of devices at that school. So at this district the bottom two schools are in the green, the first two schools need some more technology. There's also a what if calculator so if you want to, let's say your test elementary school 3, with 29 percent of students being able to take the test what happens if you add 20 more computers how does that percent change or maybe you increase your testing window by a week how does that affect the percentage. The other question is how many days will it take to test, so there is a Web-based calculator called the Smarter Balanced Technology Readiness Calculator, this one's more general-purpose you don't need any login account and it takes a lot less time to enter information because you basically enter four different fields; the number of students at the school, the number of computers at the school, how many hours that computer is available per day and your internet bandwidth. So what comes back is the number of days that is expected to take to complete the ELA and Math assessments. So on this example on the screen, there's a 10:1 student to computer ratio we're estimating that the computers can be used four hours a day and the Internet connection speed is 10 megabits. After you hit calculate, you get information underneath those buttons and it says we estimate it takes 20 days to complete both the ELA and math assessments and it's going to take up 12 percent of your total bandwidth. So this is a quick and easy way to a get an estimate of how many days it would take given the number of students and computers and hours per day available for the computer. So remember this estimate, this calculator is based on operational system for the field test its pretty much half this number of days because it's going to take either the math or the ELA and not both assessments. This is a slide to show how student registration will work for the Smarter Balanced system. The independent charter schools on the left submit CALPADS information to the CDE CALPADS system and we're going to use the CALPADS system to register students into the Smarter Balanced system. So there'll be no need for charter schools to register students into the Smarter Balanced registration system, we'll be using the information from CALPADS, however the charter school will need to set the student level accommodations so they will access the registration system to set for example does this student need text to speech, does this student need braille or maybe the hard copy option. So there's a nice document called the Smarter Balanced usability, accessibility and accommodations guidelines that list out all the options there so the charter school will set those options for each student. They'll log into the student registration system, find the student and mark off the options, for larger schools there will be a File Upload option to set the accommodations. So here's some resources, there's a technology readiness Web page on CDE's site the Smarter Balanced technology strategy framework and system requirements specifications document that lists out the minimum specifications it'll be updated here in the next month or so but it's pretty much all the information I've given you here. There's a key local considerations document and also reiterating the field test Technical Assistance Center not only for assessment questions but also for technical questions and here's his email address you want to make note of it firstname.lastname@example.org for your IT related questions as they pertain to smarter balance. Okay so I'm going to hand this over to Jessica Barr. She's going to talk about CALPADS.
Good afternoon everyone. You'll see on your screen it was Cindy Kazanis. I'm going to replace Cindy today, this is Jessica Barr. I'm the administrator of the CALPADS, CDS, CBEDS Operations Office and I'm just going to talk briefly about some of the implications that all these changes have on CALPADS. So first I just wanted to touch base and take this opportunity to provide you a reminder about the current data collections that are happening through CALPADS. The Fall 1 and Fall 2 are currently ongoing. We have a certification deadline for Fall 1, December 13th, so that's a very critical deadline that you need to meet and this requires enrollment, grads, drops, etc. Fall 2 certification deadline is February 21st and you'll note the dates of the end of year as well. So with all these changes and all of the modifications that are happening with 484, CALPADS becomes more integrated into the entire system so before we hadn't used CALPADS in the full capacity that we have available to us. Rodney mentioned that the CALPADS data will now be used to be uploaded into the test registration system for Smarter Balanced. That will start with the field test and continue forward in the years to come for operational testing. We are currently working on an implementation plan and that will include some aspect of reporting. Reporting in CALPADS for the new assessment has yet to be determined so it may look a little bit different and that’s yet to come but we will be communicating that out to the field as soon as we have more information. Assessment and accountability implications, regarding CALPADS and how charter schools report that’s primarily what I wanted to discuss with you today. The CALPADS data are being used for assessment and accountability purposes more than they ever have before. Each June charter schools may elect to change the method of how they report CALPADS and CBEDS. Charter schools can elect to report independent of their authorizing agency or the charter schools authorizing agency reports data directly to CALPADS and CBEDS. This has no relation to how charters are funded so with the change in the definition of a local educational agency direct funded charters will now be responsible for testing students and this can have an impact depending on how you report to CALPADS. The CALPADS security model only allows us to show results to the LEAs who actually report independently in CALPADS. So for example direct funded charter schools who do not report independently in CALPADS will not see their students’ assessment results. So this is something to consider now as we move forward for 13-14, everything has been established, the deadline was June 30th to change how you report to CALPADS, each year again in June you will have the opportunity to make that change so as I mentioned before the data are being used for high stakes purposes, so it's absolutely critical for LEA's to update their CALPADS data on a very frequent basis and as it gets closer to the assessment time, in the windows, it's very important and critical that you have data that are accurate and up-to-date. In terms of CALPADS data and accountability, demographic and programs student-level data, those corrections are actually occurring in CALPADS and accountability is pulling data from CALPADS and this was the first year that the corrections actually happened inside of CALPADS so again this is a change and this will basically become the status quo for future years. The CALPADS student level data that are used for accountability, focused on student enrollment and exit data so specifically it is used for accountability purposes to determine if a student is continuously enrolled and that's basically where you get your accountability main data set from and it's also used to identify the four-year graduation cohort rate for AYP. Secondly it is used for demographic and program participation information and this is to determine the student sub-groups that accountability uses, specifically race and ethnicity, gender, English Learner status, socioeconomically disadvantaged, which means the FRPM eligible students in a combination with a specific parent Ed level: and the students with disabilities sub-group. If you have specific CALPADS questions, especially regarding the information moving forward with assessments, please contact the first phone number, the 916-324-6738 that is my office or you can email, the email address right below that. If you have any kind of CALPADS related questions that have to do with data collections, and how you actually provide data to CALPADS please contact the lower number, the Service Desk and this is operated by our partner at CSIS or you can choose to use the email below. Thank you, Jessica.
This is Julie again. This last slide is a one page snapshot of all the contact emails and phone numbers for the programs that we've talked about today during the Webinar.