CMT Introduction for AgenciesThis document is the text transcript for the CDE Monitoring Tool (CMT) Introduction for Agencies presentation.
Speakers: Malik Abdul-Khaliq, Consultant & Stacy Savoca, Analyst
Location: California Department of Education, Sacramento, CA
Malik: Welcome to the CMT Webinar Training.
My name is Malik Abdul-Khaliq, I’m a Regional Team Leader in the FPM Office. Presentencing with me today is my esteemed colleague Stacy Savoca.
Stacy: Hello. I’m the analyst in the FPM Office. Myself and a developer at CDE have spent the last 2 years working to bring CMT to fruition. We hope that it serves the users well.
The FPM Office intentionally named the California Department of Education Monitoring Tool or CMT with the term tool instead of the term system as a reminder that CMT is a tool used in the monitoring process. There are many activities performed by Agency staff and CDE staff in the monitoring process outside of CMT.
Malik: These are the topics we’ll cover in today’s presentation. We will also take this opportunity to point out recommended activities related to CMT use in the monitoring process.
CMT is the primary repository for monitoring reviews. Changes that occur during the year will be updated in CMT first. Schedules and other information on the Web may not be current.
The Webinar today will present information in a specific order to show the functionality in CMT. Actual work flow by users may differ from the direction presented.
Stacy: CAS is user maintained system. Users can update CAS directly when information such as a phone number or e-mail address changes occur. All information displayed for users in CMT is based on the user information set up in CAS. When creating an account, users need to use title case and check spelling.
Stacy: When a user logs in to CMT, the tool accesses CAS to verify if the user has a valid CAS user account. It’s important to note that having a CAS account doesn’t give a user access to CMT. The user must be assigned. The FPM Office will assign all Agency Review Coordinators to CMT. The Agency Review Coordinator will assign additional users as needed.
Stacy: This screen displays the CAS User Logon page and path to access CAS.
Malik: Users have access to functionality based on the user level and assigned role. There are two User Levels in CMT, Agency and Review with multiple roles in each level.
CMT has two roles at the Agency level:
1-Agency Administrator—Basically this is the super user for the agency.
2-User Administrator—Provides support in managing users
The FPM office will assign each Agency Review Coordinator as an Agency Administrator. The Agency Review Coordinator will add and manage all additional users. User management instructions will be provided to all Agency Review Coordinators.
Malik: CMT supports FPM reviews as well as Title III, SIG, CRR, Adult Education and Before and After School Non-LEA reviews. There are three roles at the Review level:
1-Agency Review Coordinator—Basically the super user for the review.
2-Agency Review Program Staff—Program knowledgeable with access to all features except user management.
3-Agency Review Staff—This user is the support staff person. This role cannot manage users or certify evidence requests.
Malik: These are the key LEA staff and their roles.
The FPM Office is working with the COE Monitoring Leads to provide CMT training in a train-the-trainer capacity. At this time, the COE Leads have only had minimal training. The COE Monitoring Lead may refer agency staff with CMT technical support questions to CDE.
Malik: These are the CDE key staff and their roles.
All questions related to the logistics of the review are directed to the Regional Team Leader, the RTL.
Questions related to the program instruments or evidence requests, please contact the assigned program reviewer. CDE Program Reviewers have the ability to view agency screens for technical assistance purposes in a Read Only format.
Stacy: This is a print screen of the CMT Logon page. The user enters a valid CAS username and password. User information is specific to the user and must never be shared.
CAS user links are also available on the Logon page as well.
Stacy: Most users are assigned to one agency and are directed to the Agency Home tab upon logging in to CMT.
CMT provides information based on the user role. The Home screen only displays open and active reviews with the most current review on top. If this agency had a prior review which had findings that weren’t resolved, the review would display just below the current review.
Selecting the “Review” tab will display all open and active reviews as well as closed reviews.
The Home screen contains a Bulletin board which the FPM Office will update with content relevant to the Agency staff. Feel free in advising the FPM office of content that would be of interest to agency staff.
Stacy: Users assigned to two or more agencies must select an agency from the assigned list before accessing a review.
Stacy: Select “View Summary” from the “Home” tab to display the review details.
Information, The Agency Name, and CDS code. The review title and stage. There are three review stages; Pre-Review, In-Review, and Resolution. CMT screen and functionality changes based on the stage. For instance, in Resolution, instead of focusing on Evidence Requests, the agency focus will shift to resolving the Instrument Item findings.
Information related to the FPM Website, any specific On-site review information, Review dates and Evidence Details is provided.
A list of instruments is displayed (make note of the instruments for later work) with the number of Incomplete Evidence Requests. Evidence Requests are the documents, such as Web pages, and hard copy references that the agencies provide in the monitoring process such as an LEA Plan, SPSA, Inventory, etc., CDE Reviewers and Agency Program Staff.
The “View Comments” link. This functionality will be covered later in the presentation.
The “View Resources” link is available as well. This link will be discussed later in the presentation.
The side bar contains the RTL, Agency Coordinator, Review Sites and programs reviewing the specific sites.
All CMT screens will contain contact information for the FPM Office and COE leads.
Once the agency user has entered the “Home” screen, selected the “View Summary” to become familiar with the review details, the user will review the program instruments for the review. Select the “Compliance Monitoring” link.
Stacy: On the Compliance Monitoring Web page, select the “Program Instrument” link.
Malik: The 2016–17 Instruments are posted above the 15–16 instruments with the exception of Physical Education (PE). The PE Instrument is pending.
It’s the agencies responsibility to select and review the program instruments. Program requirements and evidence requests are displayed in the Program Instrument PDF files. Program instrument items and evidence requests are viewable in CMT.
The View Resources link is used by programs to share samples, templates, and other resources.
The FPM Office has provided an Excel export of all evidence requests as a resource for all reviews. This spreadsheet is not specific to a review but contains all evidence requests for all fourteen program instruments. The spreadsheet can be sorted or filtered to assist agencies when collecting responsive files.
At the conclusion of the review, the Notification of Findings is archived in the View Resources link.
Malik: This is sample of the Master Evidence Request List.
Malik: The Agency “Documents” tab is the Agency work space for uploading documents, indicating Web page links, and identifying Hard Copy references which will be linked to the Evidence Requests. The documents in the Agency “Documents” tab are not visible to the CDE Reviewers until linked to an Evidence Request and Certified. These steps will be covered shortly. CDE Reviewers do have a read only view of information related to the documents: title, description, links to view the document, updated, and Edit/Delete options.
This print screen displays documents added previously. Filter, search, and sorting options are available to assist users.
CMT is not a secure environment. Documents with student identifying or otherwise sensitive information must not be uploaded. Three document types are available to add in CMT. 1) The most common are files. File size is limited to 25 MB. Allowed file types are standard Microsoft products. 2) Web address. Provide a complete Web address to the appropriate item. 3) Hard copy reference. This option is for On-site reviews only and documents that cannot be uploaded to CMT due to sensitive nature or inability to upload such as building plans. Make sure to indicate in the description who the reviewer will work with to view the files and the location where they will be accessible.
All documents must contain a title and description. When initially uploading enter a general description. Instrument and item information is provided in a link description. This functionality will be covered later in the presentation. CMT is designed with version control. Document should never be uploaded more than once. Users cannot upload documents with the same title or file name. Additional files must contain a version number, revision name, etc.
Enter a document title, description, select the “Document Type”, then “Browse” to locate the document on the local computer or server. Agency staff may use tools or programs such as Google Docs in the collection process, but the actual file must be moved to CMT. Please do not list a link to other systems. Select “Add Document”. The document will be added to the Agency Document Tab. Users can Edit or Delete documents unless locked by the CDE Reviewer. Locking of documents will be covered later in the presentation.
Stacy: The Evidence Request tab in CMT displays the Evidence Request by program instrument. Filter, search, and sorting options are provided to assist the user in selecting files for uploading.
Information will display regarding the number of linked documents, the request and response status. This functionality will be covered later in the presentation. Selecting an Evidence Request from the list will provide more detailed guidance related to the Evidence Request.
Once the user has selected the files need for each Evidence Request, the “Documents Tab” is selected.
Stacy: Selecting General Ledger in the Evidence Request tab will navigate to the General Ledger Evidence Request Detail Screen
This screen displays documents previously linked to the General Ledger Evidence Request. To link another document, select “Add a Link.”
CMT also provides quick links to share an evidence request with other instruments that ask for that evidence request and link the same document. This allows users to share documents with other evidence requests where appropriate and not share where it is not appropriate.
the document by clicking “Select”. In this print screen, Affirmation of
Consultation was selected. Large doc file 7mb shows that it was previously linked
to this Evidence Request. This screen also displays the area where comments can be posted related to the evidence request. This functionality will be covered later in the presentation.
Stacy: During the process of linking a document from the Agency document tab to an Evidence Request, CMT allows the user to update the description. This is where item and instrument detailed information can be entered when linking this document to this evidence request.
Select “Add Link” to link the document to the Evidence Request. Include references to page numbers for large documents.
If this document was selected in error, select “Cancel.” Selecting “Add Link” will complete the process.
Once all Evidence Request have been linked to the appropriate documents, the user Certifies the Evidence Requests.
Stacy: Select the “Certify” tab to access the certify evidence request. An Evidence Request such as “LEA Plan” will most likely only have one document linked. An Evidence Request such as “SPSA” will most likely have several documents linked. The user certifies the Evidence Request and all documents linked to that Evidence Request. Documents linked to certified Evidence requests are considered final. Agencies no longer submit a program instrument.
When the Agency has certified the Evidence Requests with linked documents, the CDE reviewer can evaluate the Program Instrument Items. At this point, the reviewer will lock documents relevant to the review. Once a document is locked, an agency user cannot edit or delete the document in the Agency Document tab. Additional documents can be provided to the Evidence Request containing locked documents.
CMT will display the term “locked” for all documents in which the link has been locked. In the previous slide, the “SARC” document has been locked. Reviewers may choose to lock documents that do not meet requirements as a matter of record during the review. The proposed records retention guidelines in the monitoring process is the current year plus four previous years. All documents not locked after five years would be purged from CMT. Agencies are advised to retain all documents uploaded to CMT at the local level. Agencies will be advised once records retention guidelines are established.
As Reviewers evaluate Program Instrument Items based on the documents linked to Evidence Requests, the Request Status will change by the Reviewer to “Incomplete” or “Currently Sufficient.” The status “Incomplete” indicates that the Reviewer needs more documents and must be accompanied by a comment with guidance regarding the additional documents needed. The status “Currently Sufficient” indicates that the Reviewer doesn’t need more documents at that time. The status “Currently Sufficient” may change to “Complete” as the Reviewer evaluates other documents or based on the On-site review.
The “Response Status” will display the information related to “Draft” and “Certified” for each Evidence Request including Agency user information, date, and time.
Stacy: Based on Agency user activity in uploading, linking, and certifying documents, or during the period that Reviewers are evaluating items, additional documents may need to be uploaded. The same steps are followed as shown in the previous slides. The difference is when an Evidence Request has been previously certified. CMT will provide a reminder that draft links of documents to Evidence Requests are not visible to the CDE Program Reviewer. The Agency user with a role that allows the user to certify must select “Certify” for each Evidence Request with the draft link. If a CDE Reviewer has locked a link to a document, the agency cannot remove the link.
Malik: Comments are the primary communication tool in CMT during the monitoring process. All conversations, outside of CMT such as phone conversations, must be summarized and memorialized in CMT comments for audit purposes.
Comments are posted related to an activity not to a person. All CDE and Agency staff assigned to a review can view comments.
Agency users will post comments most often at the Evidence Request level in the Evidence Request Detail screen.
Malik: Enter comment text in the “Add a Comment” field.
Three comment types are available to add in CMT. 1) Review; these are comments related to the whole review. Such as a change to site that the team would be reviewing that will be closed due to an urgent situation. This would include a phone call to the Regional Team Leader with a follow-up Review level comment. 2) Instrument Level; these comments would be related to an entire instrument. For instance, if an agency program staff was providing documents for a review and was unexpectedly unavailable due to an urgent situation, a comment would be posted at the Instrument level advising of the situation that effects the whole Instrument. 3) Evidence Request; these comments are the most common and would be posted when Agency staff have questions or have information to share related to an Evidence Request.
Select the appropriate option in the drop-down menu.
Comments are displayed descending, chronological, and in a thread fashion similar to text messaging on screen space permitting. All comments can be viewed by selecting “View Comments.”
Malik: The View comments screen displays all comments related to the review. Filters and sort options are available to manage comment views.
Malik: The Recent Activity tab provides users with current information related to the review. Information can be filtered by Timeframe or the number of items per page.
Malik: Agencies with findings at the end of a monitoring review will move into the Resolution stage. Only instruments with findings will display on the Agency Review Summary page. Information related to findings will display, such as Resolution Deadline, Unresolved Findings, Incomplete Evidence Requests, Reviewer information, and Agency staff information.
The complete Notification of Findings is available in the “View Resources” link on the Agency “View Summary” page.
Select the Program Instrument to review each finding.
Malik: This is a print screen the Program Instrument Items with findings. The status of the finding and whether additional documents are needed is displayed.
Select the Instrument Item to view the finding text.
Malik: The Finding page displays the finding status, resolution deadline, the finding text, and the Instrument Item Requirement.
Resolution Agreement Requests are submitted by the Agency via CMT comment feature. The request must describe activities that will occur and date when evidence will be uploaded. For example, “San Gregorio USD would like a resolution agreement to revise Goal 2 of the LEA Plan. This will be done and approved by the Board by March 15, 2016. Resolution documents will be certified by April 1, 2016.” Requests cannot extend beyond 225 calendar days after last day of the review.
Reviewers approve or deny Resolution Agreement Requests within 5 days and update the Resolution Due Date in CMT. An agency that has not resolved findings within 225 days may have their names submitted to the State Board of Education, SBE, for possible action.
[No presenter commentary for this slide]
Stacy: This is a print screen of the current items in the CMT parking lot. [Note: This statement is an error by the presenter, slide presented does not show a print screen of the CMT parking lot]
Slide #34 & Slide #35
Stacy: CMT notifications have been implemented as shown in the next two slides. [Clarification: Presenter is referencing slide 34 and 35, not 35 and 36]
Stacy: This slide provides a list of resources available to Agencies participating in a monitoring review.
This concludes our presentation. Thank you for joining our training opportunity.