Meeting of the Board of Regents | February 2009

THE STATE EDUCATION DEPARTMENT / THE UNIVERSITY OF THE STATE OF NEW YORK / ALBANY, NY 12234

 

TO:	EMSC Committee
FROM:	Johanna Poitier
SUBJECT:	Critical Improvements to the K-12 Data System
DATE:	February 2, 2009
STRATEGIC GOAL:	Goal 1
AUTHORIZATION(S):

 

SUMMARY

 

Issue for Discussion

 

What short-term actions is the Department taking to improve the K-12 data system?  What additional long-term actions are planned?

 

Reason(s) for Consideration

 

For information

 

Background Information

 

The Problems

 

The Department’s data collection and reporting system is a critical tool to ensure high standards of quality education and accountability in schools across the State. The data system has experienced many problems during the past four years  In addition to working on fundamental solutions; the Department surveyed the field to find out what parts of the system were creating the biggest problems. The survey revealed considerable concern about the timeliness of data and the processes associated with the K-12 data collection and reporting system. These concerns were also shared with our partners, customers, and Department staff during a series of conversations with the Senior Deputy Commissioner.

 

Chief problems cited by the field included:

 

• Data has not been released to the schools and the public in a timely way.
• Districts have not been able to see very quickly the data they entered into the system, thus leading to errors in data entry.
• The data system itself has not electronically reminded users of obvious errors they have made in entering the data.
• The “spinning cube.” Schools have had a difficult time verifying their data through nySTART as the data progresses through the system. 
• When schools did get access to their data, they sometimes doubted the accuracy of the results because of the elapsed time.
• Correcting the data has been laborious and time-consuming for schools and the Department.

 

The Solutions

 

              To focus on areas of greatest concern, we  formed a “Datacore” group drawn from schools and BOCES to make recommendations for changes to the data system. It was essential to have the input of people closest to the day to day operations share their perspectives on practical ways to systematically improve our data system. In addition, we engaged highly respected data experts, notably the Center for Educational Leadership and Technology and the Parthenon Group, to suggest solutions. All of these partners offered useful input to re-engineer the data system.

 

To eliminate the problems and create a state-of-the-art data system, the Department is now putting into effect a series of critical short-term and long-term solutions to improve the data system. Altogether, these actions constitute major advances, organizationally and technologically, all of which will foster timely, more accurate and reliable data collection and reporting. 

 

 In order to implement these changes, we have applied for a three-year grant of $7.8 million from the U.S. Education Department (USED) to help improve the K-12 data system.  If we receive the grant, we will use the funds to streamline the data system, increase data quality and accuracy, make local reporting easier, support Regents policy and school district practice to improve student achievement, and reduce the cycle time for reporting to the public.

 

All of this will be coordinated with the ongoing work to develop a P-16 data system, designed to improve college-ready graduation rates and college completion rates statewide. The Parthenon Group, a strategic advisory firm with expertise on systems and data used to drive education reform, will facilitate the creation of an implementation plan in 2009 to identify the priority objectives of an integrated P-16 data system and detail the structure, resources, and technology needed to support the effort. Funding for this process will be provided by the Bill & Melinda Gates Foundation.

 

A.  Short-term Actions

 

The changes accomplished to date fall into two key categories: (1) leadership and organization and (2) the redesign of the data infrastructure.

 

(1)  Leadership and Organization

 

We have eliminated the “silos” in the Department to make major leadership and organizational changes to support a state-of-the-art data system. It is essential that all parts of the USNY enterprise that collect data are coordinating efforts and that these actions are aligned with test administration. 

 

• Project Team

 

To oversee all of the work being done to improve the data system, we have formed a new internal Project Team that includes staff from the EMSC data unit (Information and Reporting Services), Information Technology Services (ITS), and the Special Education data unit from VESID.  The Project Team works under the overall direction of the Senior Deputy for Education–P16 and the Deputy Commissioner for Operations and Management Services. The Chief Information Officer (David Walsh), and, Assistant Commissioner for Policy and Strategic Planning - P16 (Alan Ray), provide joint leadership to the Team. The Team also works closely with the Office of State Assessment to coordinate data reporting on State tests.   Finally, the ongoing support of the Regional Information Centers (RICs) makes the improvements possible.

 

In addition, two excellent ITS programmer/analysts and a supervisor now provide direct technical support in creating in-house reports; troubleshooting nySTART and other data management and reporting issues; and providing valuable analysis of the overall implementation of technology to the work of the office.

 

• Project Managers

 

We have also hired a Grow Contract Project Manager from a nationally recognized technology company (CGI), to manage the contract with the Grow Network/McGraw Hill, and help find solutions to the problems. He started the first week of December. CGI is also providing a Data Architect, who is helping to analyze and, where necessary, restructure databases used for the production of district and public reports.  In addition, with the help of the Bill & Melinda Gates Foundation, we have hired a P-16 Project Manager who is focusing on outreach to the field, data governance, local resources and technical assistance to schools and Regional Information Centers.   

 

• Chief Data Director

 

              We have selected an outstanding candidate to be the new Chief Data Director and are awaiting final approval to hire him.  This is a key leadership position that has been vacant for a year and a half.  This person will provide strategic and management oversight for the Statewide collection, evaluation, implementation and reporting of all P-16 data.  The Chief Data Director will oversee and direct the Office of Information and Reporting Services and staff involved in activities related to the collection, evaluation, implementation and reporting of educational data oversee the transformation of New York State’s data collection, management, analysis and reporting processes that contribute to its systems of school improvement, school accountability, policy development, and resource allocation.    Along with other responsibilities, the Director will:

 

• Lead and implement the major reform and expansion of the Department’s educational data collection and reporting to ensure the use of best practices.

 

• Lead the development of Department policy related to accountability, research and data analysis, collection and reporting for consideration by senior managers and the Board of Regents.

 

• Direct the planning and development of public reports related to the data-driven statewide discussions, student achievement trends, accountability issues, etc.

 

• Work closely with the Office of Information Technology Services to improve the Department’s use of technology in data collection and reporting, including the development and monitoring of effective, efficient data collection, management, and reporting systems.

 

• Direct and oversee the work of project managers and any vendors involved in building a process or perform a service as part of the accountability function.

 

• Develop systems to effectively communicate data policy and practices to all affected parties in the field. Lead statewide advisory group meetings to ensure that all data collection statewide is consistent, timely and accurate.

 

(2)  Redesign of the Data Infrastructure

 

We are redesigning the data infrastructure to alert districts when data they have entered contains obvious errors, provide reports in a more timely way, help districts select superior data management systems, and train district users in data collection and reporting.

 

This new infrastructure will allow us to:

 

Create a system to report the Grades 3-8 test scores within 10 weeks of test administration

 

It is important that school districts and the public receive test score information in a timely way and have a reliable timeframe for receipt of this information.  In the first year of the Grades 3-8 tests, 2006, it took 33 weeks to release the ELA scores and 28 weeks to release the Math scores because new psychometric processes were required for the new tests.  Those timelines have subsequently decreased.  Beginning with the next Grades 3-8 exam administration, the Department will shorten the time for test scoring and scanning, psychometric equating, and quality assurance based on new systems now in place.  This will allow us to reliably report the test results within 10 weeks after the tests are administered, which is a significant improvement.

 

Provide electronic reports to districts so they can see very quickly the data they report.

 

In the past, districts have entered their data but have not been able to see in a timely way exactly what they entered. This situation produced data submission errors. With the help of the Regional Information Centers, we are now developing simple data validation reports that will show districts exactly the data they entered and how it appears in the data system. Districts can then revise the data more quickly as they find problems. These new reports will be available to districts this spring.

 

Introduce electronic checks and edits to help districts improve data accuracy.

 

A major advance from previous practice is the creation of important electronic data checks for accuracy and reasonableness. When districts enter the data for the first time, they will be alerted if the data are incomplete, inconsistent, or obviously inaccurate. Districts will also be alerted if there has been a significant increase or decrease in a data category (e.g., graduation rate) from the previous year. In addition, districts will be able, through the use of a student’s ID, to determine whether a student has actually dropped out or transferred and enrolled in another district in the State. This will help districts keep more accurate graduation and dropout rates. Some of these changes have been made already and the rest will be put into effect by spring 2009.

 

Eliminate the long delays districts have experienced in verifying their accountability and other data (the “spinning cube”) by providing faster alternative reports.

 

Without going through the nySTART system, districts will more easily be able to print and verify their data and the data will be consistent with that contained in the nySTART reports.  This will allow districts to avoid the “spinning cube” that causes delays when districts attempt to verify their data through the nySTART system.  The more simple alternative reports, that districts can use to verify both their individual and aggregate data as the data progress through the system, will be made available systematically and revised as data are updated in the statewide repository, beginning with the spring-summer 2009 data verification. 

 

This is a temporary, short-term solution to the data verification problem. We are also working on a long-term revamping of the data system. Please see more on that later in this report.

 

Coordinate the collection of both special education and general education data to eliminate duplication and confusion.

 

In the past, EMSC and VESID collected data in different systems and at different times of the year. There was inadequate coordination between the two units. Our internal Project Team is creating a 2009-10 electronic calendar of all relevant data collection and reporting activities, including the State assessment schedules.  The calendar will be provided to districts via the website during the spring of 2009.  SED will be able to notify districts quickly of any changes to data collection deadlines or data release schedules.

 

Make consistent and uniform the different Student Data Management Systems that districts use.

 

School districts use a wide variety of student data management systems, school lunch systems, and special education management systems, all of which vary greatly in their ability to collect and export required State data elements.  If we receive the USED grant, we will establish a Student Management Systems Certification Center.  This Center will work with the student management system vendors across the state to establish criteria for certifying systems for use by school districts. The center will be responsible for alerting vendors to proposed changes in the data collection or accountability rules and will seek vendor advice on the implementation of those changes, thus making accurate data reporting easier for districts.

 

Support and training for school districts

 

We will create a Technical Assistance Center responsible for the development of standards for school-based staff who enter data, along with activities to support the implementation of those standards.  If received, the USED grant will provide the resources to implement this long-term design and infrastructure improvement. The Center will be charged with developing, disseminating, and maintaining a series of training activities for district staff responsible for the collection of data and its transmission to the State. The Center will create written materials as well as websites and other structures as needed to provide more uniform support for school districts.

 

These activities will be focused on two themes: a) technical training and support for school district officials who deal with data, and b) the clear communication of data business rules. The quality of these support services varies greatly across the eleven Regional Information Centers and the Big Five. The Technical Assistance Center will be responsible for establishing a standard set of support activities across all Centers and the Big Five school districts and provide quality assurance procedures for the ongoing maintenance of those services.

 

Better communication with school districts.

 

Timely and consistent communication with school districts across the State is critical. We are expanding the advisory panels that provide input and advice on our data collection and reporting practices to include users of data, data collection staff, and technical staff.  This will help to expand the discussion in order to improve how the data is used and provide better and more immediate feedback from the field as we continue make improvements.

 

B.          Long Term Plans to Re-engineer the Data System

 

Through a major re-engineering of the data system, we will develop a streamlined system that will be more responsive to the needs of schools, increase data quality and accuracy, make local reporting easier, support Regents policy and school district practice to improve student achievement, and reduce the cycle time for reporting to the public.

 

During conversations with the Senior Deputy Commissioner, stakeholders as well as outside consultant groups have commented on the multiple levels of data repositories in use in the state and have recommended streamlining this long data trail. We are planning to revamp the technology so we can (a) decrease the collection/management cycle time, and (b) provide more flexible access for all to the data. Districts would have unfettered access to their data, with the ability to update it, right up to the moment it is released to the state and becomes “official.”

 

The contract with the Grow Network/McGraw Hill will end in October 2010. In exploring options to reform the data system, we are reviewing ways to bring some portions of the work back to the Department and the regional data centers. This would provide greater control over the data collection, accountability calculations, and data reporting. The Project Team is determining which options are feasible, given the State hiring freeze, State budget constraints, and current and potential federal support.

 

              All of this work will be coordinated with the ongoing work to develop a P-16 data system.