Skip to content
Printer-friendly version

Smarter Balanced Tech Readiness Webinar Transcript


The following is a text transcript of the Smarter Balanced Tech Readiness Webinar on November 4, 2013 as presented by Rodney Okamoto and Cindy Kazanis of the California Department of Education.

Webinar starts.

Rodney Okamoto:

This is the Smarter Balanced Technology Readiness Webinar and my name is Rodney Okamoto at the California Department of Education along with Cindy Kazanis who will be helping to present some of the later slides. So before we get started, due to the number of people on the call we won't be able to take any questions over the phone or verbally so you will need to use the Q and A box on the right hand side to ask your questions.

So these are the agenda items for today, first off, a brief overview of the Smarter Balanced Assessment System, and then some information on the Smarter Balanced Field Test, then we'll get into the minimum system requirements for the field test and operational system. We will talk about some tools you can use to help gauge your readiness, the technology readiness tool and also the technology readiness calculator, and then we will talk about the student registration process as it pertains to Smarter Balanced, and then finally some guidelines and helpful tips about how to prepare for online assessments not only from a technology perspective but from an overall district perspective.

So first off, just some basic information about the Smarter Balanced Assessment System. It's a system that's being developed as a consortium of 25 states. The development is funded by a federal Race to the Top grant and the goal of the assessment system is to assess students in grades 3 through 8 and grade 11 in mathematics and English Language Arts with the assessments aligned to the Common Core State Standards. The system will be operational in 2014-15 but also the way that California is going to use the field test it will have a greater impact on the districts in this school year. And we'll also be talking about how the Smarter Balanced System plays an integral part in Assembly Bill 484 or what is known as the CalMAPP Assessment System.

So this is how California does statewide student standardized testing right now, primarily paper pencil with multiple choice questions. There is a writing assessment involved but it's all done through paper pencil.

The new way is a computer-based system and what this diagram is showing is how students’ computers are connected to the Internet. And I also wanted to point out that the student's computer is running a software application called the Smarter Balanced Secure Browser so it's not your everyday Internet Explorer or Firefox, Chrome Web browser. It's a custom application that is loaded on each student's computer. On the right hand side in the middle is a depiction of a test proctor, also on a computer, but in this case the test proctor is running a standard Web browser such as Internet Explorer or Firefox or Chrome or Safari. And that test proctor is using the Web browser to monitor the student's progress as he or she goes through the test. And finally up upon the upper part of the screen you see a depiction of centralized servers, also connected to the Internet. So this is showing how the student's computer goes to the internet, connects to centralized server somewhere on the internet, how the centralized server sends items to the student, how the student responds to those items, and those responses are stored back on those centralized servers, and how the test proctor in that classroom environment or lab environment, or wherever the students are being assessed, monitors each student's progress and is able to pause or restart individual student's test.

So on to the field test. The objective of the field test is, or so called "test the test", which means not only the items but also the technology involved and it's important to know that the field tests is not meant to measure individual student achievement. So in California, the field test is a little different than the other consortium states because AB 484 requires that all the students in grades 3 through 8 in all the schools in California participate in the field test for grades 9-10. AB 484 requires students in the scientific sample to participate. And in California, the scientific sample is probably about four percent of grade 9 and grade 10 students overall so very small percent of grade 9 and grade 10 students will participate. And then in grade 11, AB 484 requires all students in the scientific sample to participate. So in California, we expect the grade 11 scientific sample to account for approximately forty percent of all grade 11 students. AB 484 gives the option for other grade 11 students who are not part of the scientific sample to participate also.

So the field test window starts on March 18, 2014 and runs through June 6. For California, schools will be assigned to a six-week window the students will take either the math or the ELA assessment and that subject area will be assigned by CDE. Also the assignment will be by school and grade within that school.  So for example, in a given elementary school who serves fifth-grade students, CDE will assign, for example, grade 5 ELA to that specific school. Another school in the district might have grade 5 math and another school might have grade 3 ELA. So the assignment is by grade in that school and not by the individual students.

Also special to California, there's a helpdesk phone number and e-mail address that deals with field test questions so I have this on the screen. Also I have it later on the end of the presentation, but this phone number can be used for technology questions or questions about how to administer the field test, what window a school was in, some training opportunities, and other information regarding the field test. So remember, this is for California only. You may see information from Smarter Balanced and the field test that's a little different. And that's because that's at the consortium level and here California is doing a little bit differently, things a bit differently on the field test including this technical assistance center.

So let's get into some of the minimum system requirements for the student's computer. So first off, from Microsoft Windows operating system perspective, Windows XP Service Pack 3 is supported. However, remember that Microsoft is ending support for Windows XP in April 2014 which means no security updates for Windows XP after April 2014. So you should have a plan in place already to update to either Windows 7 or Windows 8 or some other operating system. From a Smarter Balanced perspective, Windows XP will be phased out. At least the proposed phase-out date is the end of the school year, the end of the 2015-16 school year.

So the other Windows operating systems that are supported are Windows Vista, Windows 7, Windows 8, and the recently released Windows 8.1. However, please note that Windows RT, which runs on a Microsoft-based tablets, they're not supported at this time. Also make note that there has been some confusion about an operating system called Windows 8 RT and I think that's maybe just a marketing. Some Web sites might have erroneous information that Windows 8 RT is not an actual operating system so you’re either running Windows 8 or you’re running Windows RT.

For Apple Macintosh operating systems, 10.4.4 is supported--although the proposed phase-out date is the end of the school year 2014-15, so the end of the first operational year. 10.5 actually came in two flavors: based on the processor chip, the 10.5 for PowerPC is proposed to be phased out also at the end of the school year 2014-15. There's a 10.5 that runs on the Intel-based chips along with 10.6 all the way up to 10.9, which is the recently-released Maverick operating system. So those are all supported with no proposed state updates in the near future at least.

There's a question about RT and the way that I understand it is Windows RT is significantly different from Windows 8 which requires a new development effort and currently there's problems in trying to secure the operating system so that the test times can be secured, so no immediate plans to support RT.

From an Apple IOS perspective, IOS is the operating system that runs on Apple devices such as the iPad 6.x release came out last September 2012 that was significant because that version included the guided access feature. So this is a feature which, for example, it disables the ability to press the home button to jump out of an application. So that was required to secure the test items also. In September 2013, 7.x came out. So that's being released, and also remember for all tablet computers or any computer that doesn't have a keyboard, a physical keyboard is a requirement for the Smarter Balanced Assessment. Also, I have a note here that IOS 6 requires at least an iPad 2 or second-generation iPad. So if you have iPad one first-generation, they can't run IOS 6, so you will need to upgrade the device.

Android. Android gets a little bit more complicated because there's no one version of Android. So Android is an open source operating system. And Google maintains Android but any manufacturer or you or me can download the Android operating system source code, make changes to it, and get it to run on whatever tablet I have from a manufacturing standpoint. So the complication comes about because there are many different tablets that run Android, yet there's no one Android operating system.

So to make sure that the Secure Browser and the assessment work on an Android device, Smarter Balanced will be releasing a certification process where manufacturer can contact Smarter Balanced and run their device through some test to make sure that it meets all the requirements and then it will get on the certified list. So if you are considering Android-based tablets for the assessments, please contact CDE. We have an e-mail address at the end of the presentation so that we can point you to this certification Web site. It’s a pre-release now but if you're considering Android devices, we could give you that address and you could work with your manufacturer so that they know about this certification process. And then similarly to the iPad tablet, Android tablets requires a physical keyboard.

On to the Google Chrome OS. So this is different than the Chrome Web browser. Although under the covers, there's probably a lot of shared code, version 19.x or higher. More commonly, this operating system runs on Chromebooks which are wireless devices and also recently, there's been more movement or interest in Chromeboxes which has a hard wired connection to the network. So Chromebook is similar to a laptop in form factor, a lightweight laptop, although is typically not as expensive. The basic design is that all the storage is up in the Internet cloud. There's very little on board storage and then your apps are similar to a tablet, where you download apps. On a Chromebox, it's a device where you have connections to a monitor; so if you want a larger monitor data than what a Chromebook would give you or if you want a full size keyboard or use a mouse, and of course, if you want to be hard wired to the network.

Linux. There's a small percentage of schools running desktop using Linux operating system. This is also open source. We may have to pay for support from the manufacturer. Fedora 6 or Ubuntu 9 or higher is minimum operating systems for Linux.

Virtual Desktop Infrastructure or VDI. You might also hear terms about thin client or zero client technology. This is where the majority of the processing is done on a server-side instead of a processing done on the desktop side or the laptop side. So this is where you have storage area networks and large servers doing all the processing and the main advantages is easier deployment of applications you know you could give. You could push down an image really easily of a set configuration. From a testing perspective, there's many manufacturers who offer solutions but as I get into the Secure Browser, we are really asking that you test your specific VDI solution using the Secure Browser on the practice test.

Okay from miscellaneous student system requirements standpoint, 10 inch class diagonal display, that's the minimum. So the iPad measures out at a 9.7 diagonal -- that's a 10 inch class device. There are many tablet Android based tablets that have 10 inch device displays. Chromebooks come in at 11.6 and now 14 inches. There are devices such as the iPad Mini which is less than 8 inches -- that an iPad Mini doesn't meet this 10 inch class requirement.

1024 by 768 display resolution, even your iPad 2, that's the resolution that iPad 2, that's what they run in and most of your current desktops should be no problem. However, if you have older netbooks that are maybe two or three years old, they may not meet this requirement. They might run at 1024 by 600. So if you do have netbooks that you're thinking about using for the assessment, please contact CDE because there may be a workaround if you're running Windows 7 on these netbooks. But we will want to work with you one-on-one because this workaround is one of those things where it works on some netbooks based on the display chip and not on others. And we would want to make sure that you tested it on one device before you try to deploy it throughout your enterprise. Again, if you have older netbooks that don't run at 1024 by 768 or greater, contact CDE.

Again the physical keyboard requirement, you want to point out that if your wireless data network runs at the 2.4 gigahertz frequency, that you should be aware of, there could be conflicts between a concentrated number of Bluetooth devices and your wireless data network. So certainly if you have tablets and you're thinking of using Bluetooth keyboards, please check to see what your wireless data network what frequency your wireless data network runs at. Because if it’s at 2.4, you probably want to do some testing to know exactly how many Bluetooth keyboard you can run without interfering with your data network. Of course, the other option is to have hard wired keyboards for tablets.

There's currently no stylus or device or drawing device requirements. I know there's been previous documentation, maybe a year or so ago, that came out where at a certain year it said that you will need a stylus or some type of drawing tablet for math items specifically so that you can show your work on some of these more complicated items. But currently Smarter Balanced hasn't published any specific date on when those type of items will be developed.

Headphones definitely required for English Language Arts items -- there are certain items where there's audio clips. So unless you’re doing assessment with only one person in the room, you want to have headphones so one student, when one student is listening to an audio clip, it doesn't interrupt other students. Also there's a text-to-speech accommodation, both for ELA and math. This is where the Smart or the application reads the items back to the student audibly. So if a student has a text-to-speech accommodation, that student will need headphones even if it’s for a math assessment.

Also I remember at another conference, there were talks about if there’s a microphone requirement because they're considering a headphone plus microphone combination device. At this point, there's no items that require microphone input. And I haven't heard of any discussion on what year that might be when there could be items with microphone input, so for now headphones only.

And then finally, the printer need so this is the one a student has an accommodation that the items be printed. If you have a student like that in the classroom or the lab, you'll need to have a printer nearby so that the test proctor can print the items and give it to a student.

Okay on to the Secure Browser application. Remember this is the custom application that runs on each student's computing device. It's actually a modified version of Mozilla Firefox built for assessments. But every year, at the beginning of the school year, there are plans to have an updated version released. So that means it would be useful if a school or district had ways to deploy software through the network instead of having to touch each machine. Certainly, if you have a manageable number of computers that won't be a problem. The more computers you have, the more need there is to have some way of deploying applications. So for the field test of the Smarter Balanced, the Secure Browser version will be available, scheduled for November 18. So we'll get information out to you when that's available. There will be a website where you can download the application so you can get started installing the Secure Browser for the field test.

This Secure Browser is also ran on tablets – IOS, Android. And then new for the field test, there will be a Smarter Balanced applications for Chromebooks. In the pilot test of Chromebooks, the assessments were administered on the Chromebooks using especial Google account log in. So for the field test and operational, that design has changed. So now the Chromebooks will be like the other devices where application is installed to secure the device.

The application installed in a specific folder. So to avoid confusion, you want to uninstall the pilot test version otherwise you will have two Secure Browser applications installed on the computer and the student will have to know which version to run for the field test. So to make things easier, it be better if you uninstall the pilot test version if you have that version installed on student computers.

So liked I mentioned on a previous slide, when I talked about VDI, the Secure Browser application can be used to access the practice test. So when you start your browser, it looks like there's no test available that but there's actually a link to the training test which really means the practice test in this example. So you can start your browser, access the training test link, and then you could go through some items using the Secure Browser. If you want to test the text-to-speech or the Braille combination, you have to use a Secure Browser. So the practice tests in itself is available through standard Web browsers but to get a better simulation of what a student will expect, and to test your computers, I would recommend using the Secure Browser to access the practice test.

One of the reasons to have the Secure Browser application is to insure test securities. The Secure Browser put the machine into so-called kiosk mode. So for example on a Windows machine, you can’t control-alt to a different application or control-alt-delete. If you do find a way to start an application, the Secure Browser periodically check to make sure these applications, prohibited applications, aren't running, not only during execution, but at startup.

So this is one of the reasons to test the Secure Browser on one of your desktop images because you may have a application already running, like a screen capture utility or something like that runs in the background at startup. If that's on the prohibited application list, then the Secure Browser application won't be able to start-up. That’s one of the reasons to test using the Secure Browser. Also just a note that the Secure Browser is all HTML 5 and JavaScript based. So if you don't have Flash Player or a Java or Sun Java-VM on your desktop, no worries because those aren't required.

I do have a comment that I might have to check on that Secure Browser will uninstall the previous version. So I'm gonna get a user assessment test opportunity on November 11. So I'll try that out on my machine to see if when I install the field test version, if the pilot test version is uninstalled or not. We'll get that out to you.

Okay from a network bandwidth perspective, there's an estimated rate of 20 Kilobits per second per each student connected to the system. This is a hard one to measure because it’s dependent on how long it takes each student to answer items. The faster that a student answers items, the more network bandwidth will be used. On some of the ELA items, if you've gone through the practice test, you might notice that there's quite a bit of text on the stem to read on the left hand side. So unless the student is just guessing at the answers, it's gonna take a little while for the student to read the left hand side passage and there might be four or five questions on the right hand side that relates to that passage. So it's gonna take a little while for students to progress to the next question. But overall we've been told that if you average everything out, 20K per each student connected. Although at the start-up, there may be a little spike because the Smarter Secure Browser is gonna check to see if there's any a new JavaScript to download.

And then also during operational, when it's an adaptive test, we've been told that there are some so-called prefetch of items to make the system more responsive. So you might see spikes not only because the next item is coming, but because the next item and item after that have been downloaded to the computer. Also on the ELA test, there are audio clips which would take up more bandwidth then just straight textural passage. So depending on if a student gets an audio clip, that would create more traffic. But the good thing is there's no video clips, so audio clips only, no video clips. There are some audio clips that the graphics change every now and then to make it kinda look like a video but no traditional video in any of the items.

The next two bullets are pretty much common sense. This is where if you add more devices, your internal network will have to be, you may have to enhance your internal network. For example, if you add more tablets or wireless devices, you might need more wireless access points. If you add more desktops you might need more switches. And also from an internet standpoint, you may, the more devices you add at the school level will have an impact on the school to district connectivity, and then from district to county also. It all flows up to the county to the Internet.

So that's what this diagram is showing. There's over 80% of schools in California connected to the K12 High Speed Network or K12 HSN. Most of those schools are connected via the district office up to the county office. L.A. Unified School District is the exception where they connect directly to the K12 HSN. And then there are a handful of districts that connect directly to the K12 HSN instead of going to the county. And that's because of either geographic constraints or from a circuit perspective is cheaper to connect directly to the K12 HSN instead of the county. Particularly, this is how schools are connected to the Internet and on the right hand side you see a couple boxes depicting Smarter Balanced servers. So California has a certain set of servers and then at the consortium level there's other set of servers.

So this diagram breaks out some of the components that are at the consortium level such as, there's a common item bank for all the Smarter Balanced items at common data warehouse and reporting system, the digital library for the formative assessments are housed at the consortium level. But each Smarter Balanced state is responsible for the state hosted components such as test delivery, the scheduling, registration, and scoring.

So let me hand this over to Cindy to go over some of the tools that are available and some other information.

Cindy Kazanis:

Thank you, Rodney. I'm Cindy Kazanis. I'm the Director for Educational Data Management Division and I think Rodney forgot to mention at the beginning that he's also the National Co-Chair for the Smarter Balanced Technology Readiness and Reporting Work Group. So we have somebody who is helping make some of the decision at the consortium level and we are really fortunate to have Rodney do this in addition to his state job here at the California Department of Education.

So to go on to the next slide, I'm going to talk through the Technology Readiness Tool. We recently ran a report and we have just less than 20% Local Educational Agencies who have responded and submitted data on the tool. So one of the highlights of it or the takeaways for today is that, if you were to complete the tool, hopefully you will after this conversation, you will get some very good data that will come as a result of completing the tool. And there is some specific information that is required that is listed here: user account, the number of devices, your current testing Windows, and what you think your current test sessions per day will be.

The TRT has been available for those of you that have filled it out before since April of last year. And in the beginning, the results were used to help, actually created the minimum specifications that Rodney already walked through with you today. The readiness tool was a joint effort by both consortium by PARCC and by Smarter Balanced, and was developed by Pearson. And what we will be seeing in the next, what I am going to talk with you about in just a little bit is a snapshot that is coming up in December. But the Technology Readiness Tool is again requires some specific information and outputs are the next slide.

So what you'll see as the result of completing the technology readiness tool is reports based on the information you've input, about how ready your district and school sites are to provide this, the assessments for students. What we saw at a local school district here in Sacramento area was, as a result of completing this tool, they were able to develop sort of minimum specifications for how they are thinking about spending their Common Core money which of course has gone out to districts of 1.25 billion dollars. And one of the ways that districts are allowed to spend that is on devices and increasing bandwidth. So the reports can be used, as the second bullet here says, to justify those spending decisions. And also what we see at the state level and what we're trying to relate to the field and those also decisions and lawmakers is where we have and we think insufficient technology across the state. And so at a state level we do have, again based on the twenty percent that have completed the TRT, we know where some of those technology gaps are and we'll be providing I think further guidance as we move along with implementation of Smarter Balanced.

Noted on here is the snapshot. So there are two snapshots annually for the TRT and the one that is coming up is on December 13. So we would encourage you to complete the TRT so that your information is included in the snapshot so we have a good idea again from the California perspective what, how ready we are for getting, not only the field test but also the operational test comes next school year.

This next slide gives an example of a sample report. And I forget the name of this district, maybe it is called, Sample District, but we have a couple of elementary school and middle school and two other schools. And what you can see from based on the color of the third column there it says number of devices to test takers readiness at the school site. So based on the information that was input, this district has two sites that are not so ready for the test and two that are over 100 percent ready or rather at 100 percent. So they can take the exam within the current testing window that they have established and devices that they currently have. What you can see is the number of devices that are lacking at elementary school three. And so again, special attention can be made to that school now that now that this information has been inputted.

This also allows the report and with the calculator that you can see right next to the magnifying glass towards the left hand column is the, “what if calculator”. So if were to add decreased or rather increase your testing window, will you have enough bandwidth does that help with that issue. If you add a number, more computers, more devices will you have enough devices then to give the test window. And so that calculator is really, I think, a very helpful tool, again, within the TRT for districts to use to, again, start making some consciousness decisions on how you are going to, on a technology perspective, implement this test at each of your school site.

In addition to the TRT there is also a Technology Readiness Calculator. It is open and available for the public. The TRT again requires a login and it is specific to the LEA level and we have done some pre-population in there and I think that is one of the reasons why it is limited access. But the Readiness Calculator is again its web based. If you put in just some simple numbers as in your hundred students and certain number of devices, number of hours that you think will be in each device, you will get the output that, you know based on that very high-level of information, you will have a good idea of how many days you will need to administer the test and the number of bandwidth required.

This slide shows you the actual calculator so this shows that we had in our sample 600 students testing,  we had 60 students, 60 computers that were available and we determined that those computers were only available for four hours per day and the connection speed at 10 Mbps (Megabytes per second). So based on that calculation we estimated, again this is a very high level and a publicly available calculator, it is going to be a requirement of a minimum of 20 days to complete both tests. So the English language arts and math assessment for the students enrolled at this school in which 600 students are required to take the exam. And it also gives you the total bandwidth that is needed. Good useful tools to help you with your planning processes as you begin to implement for spring of this year. Keep in mind too that the field test estimate is 50% of this since we're only, at this point, giving students are only taking one subject for the field test. That is something to keep in mind.

This screen I’m going to move on to talks a little bit about student registration. And for those LEAs that participated in the pilot test, the registration will be similar in that CDE will preload information to the system using CALPADS data. And so the main thing to keep in mind as we again, as you put together your plan for implementation of Smarter Balanced Field Test and also operational test come next year is that you have good communication with whomever is responsible for your CALPADS administration. And such that, if you have new students that come in that you register them and you get them an SSID in CALPADS immediately. Especially during the testing window, we're going to be doing and what we think are probably nightly updates to the system of new students. And so if you are a district and only put information in CALPADs once a week, once a month, you are going to have to be doing it a lot more regularly to make sure that the students are in the system because at this time we don't have another process to allow students to get added to the system otherwise. We are going to depend on the CALPADs system to show that these students are existed at a particular school site or registered at a particular school site and also have their designated grade levels for the exam.

And the last two slides before the resources is more of how to start thinking about the implementation planning process. And this is different than I think in any other assessments that we have been privy to in a long time in that, you’re intimately working with. If you’re the IT person, you should be talking directly with your assessment staff as well as your curriculum staff on how to make sure that the policies that are in place or maybe that are not in place and procedures that you develop to anticipate issues as well before we, before spring and before the field test actually begins. And so, like I said, we recently were at a school site, we had a really great discussion with a district locally about the kind of group they have put together, they actually have a technology committee at the district level that includes parents and that's how they are starting to develop some of these policies as well as procurement specifications for devices. This is at the district level and any at the site level making sure that each, everybody knows their role and you can put together a plan that has some what ifs in there so that you're ready for those questions prior to the implementation of the test next spring. So the second bullet talks about what the goals of the groups are we've already given you some timelines to prepare for it. November 18th is when the Secure Browser will be available and then in early march is when the field test window begins.

The second part of this slide is asking you to think about reviewing your current policies. Do you have protections in place to ensure that equipment and bandwidth are allocated consistent with the established priorities? Do you know who has the authority to resolve issues? Are you going to allow students bring their own devices? And so those are some of the policies that you want to be prepared to have in place ahead of implementation of the field test. And then at the state level we're in the process of trying to put get together some best practices and working with our County Office of Education to figure out some best practices for the field. And so you will probably see some information forthcoming on that in the next couple months.

So finally I would encourage you to take a look at our resources on our Web page specifically the Technology Readiness Web pages listed there. It’s got a great deal of information including links to the Smarter Balanced Web site and that also includes a link to the Smarter Balanced Technology Strategy Framework and System Requirements Specifications which is where a lot of the information from the devices and such came from. That document is going to be updated in the near future and they're going to rename it to Smarter Balanced Technology Strategy Framework and Testing Device Requirements. So there will be some clarifications on the Smarter Balanced Certified Devices and a couple of other things that I think Rodney talked about earlier in the conversation.

Additionally the TRT, there is a document up there with key local considerations which is designed for TRT users and it gives guidance on how to determine network speed and utilization, wireless network considerations, BDI network considerations, and length of window planning. So that's also a good document for those of you that are completing the TRT. We've also included on here the field-test phone number and e-mail address that Rodney went over earlier that is being staffed by our testing vendor, ETS, so that's a great number to put to memory. And then finally, we got the word out about this webinar through our SBAC IT Readiness e-mail address. So if you heard about it from another source perhaps you came to our South Assessment Meeting and you heard Rodney mention this webinar, if you're not already on the distribution list, we encourage you to get on the distribution list.

Webinar ends.

Related Content
Questions:   CDE Smarter Balanced Technology Readiness | sbac-itreadiness@cde.ca.gov
Download Free Readers