Meeting of the Board of Regents | January 2009

THE STATE EDUCATION DEPARTMENT / THE UNIVERSITY OF THE STATE OF NEW YORK / ALBANY, NY 12234

 

TO:	EMSC Committee
FROM:	Johanna Duncan-Poitier
SUBJECT:	Grades 3-8 Testing Policy Overview
DATE:	January 7, 2009
STRATEGIC GOAL:	Goals 1 & 2
AUTHORIZATION(S):

 

 

 

SUMMARY

 

Issue for Discussion

 

Should the Board of Regents consider making changes in the Grades 3-8 Testing Program as follows:

 

1. Revise the current testing calendar and administer assessments in mid  spring (March) or late spring (April-May) starting with the 2010-11 school year;

 

• Take additional steps to change the test scoring and scanning process for the Grades 3-8 Testing Program In order to return test scores sooner; 

 

1. Review and revise the current levels for proficiency in the grade 3-8 tests and raise the standards; and/or

 

1. Redesign the Grades 3-8 Testing Program to allow for more refined information regarding student progress and achievement, especially measuring “growth for all?”

 

Reason(s) for Consideration

 

Review of Policy.

 

 

Procedural History

 

During the May 2008 and subsequent meetings of the Board of Regents, the Senior Deputy described progress to significantly improve the Grades 3-8 ELA and Math Testing Program. In May, the Department also described a goal to establish a system to improve the reporting of the grades 3-8 test scores so that test scores are reported within 10 weeks of the last make-up exam, as opposed to the lengthier and unreliable schedules experienced in the past. At the December 2008 Regents meeting, staff reported that a system has been created to achieve this goal for this testing cycle.

 

 

During the December discussion of the EMSC Committee, the Board requested additional information on the following issues to inform their policy discussion:

 

1. The purpose of the grades 3-8 tests,
2. What would be required to move the testing period toward the end of the school year if the Board of Regents were to change the current testing cycle (with information on benchmarking the timelines that other states follow),
3. Additional ways to speed test results to the schools,
4. Levels of proficiency in the grades 3-8 tests, and
5. Other improvements to measure “growth for all.”

 

This information is provided below.

 

Background Information

 

1. Purpose of the New York State Grades 3-8 Testing Program

 

The New York State Testing Program (NYSTP) is designed to evaluate the implementation of the State Learning Standards at the student, school, district, and statewide levels. These tests:

 

• Annually assess the implementation of the State’s learning standards;
• Help to determine the accountability status of schools and school districts;
• Measure individual student and cohort progress; and
• Provide information on student readiness for study at the next level.

 

New York State is required to administer tests in English Language Arts (ELA) and mathematics under the No Child Left Behind Act (NCLB).  These tests must:

 

• Address the depth and breadth of the State’s Learning Standards;
• Be valid, reliable, and of high technical quality; and
• Be designed to provide a coherent system across grades and subjects. 

 

The Grades 3-8 Testing Program is administered to approximately 300,000 students per grade in both public and nonpublic schools.  With the exception of the small population of severely disabled students who qualify to take the New York State Alternate Assessment (NYSAA), virtually all students (general education students, students with disabilities, and English Language Learners) in all public school districts are required to take these tests. 

 

The ELA tests measure student progress toward three of the four content standards: reading, writing, and listening (the Department does not test speaking). Many states only measure reading progress or performance. The mathematics tests measure student progress toward the five content bands.  The established cut scores classify student proficiency into one of four Performance Levels based on their test performance:

 

Level 1:  Not Meeting Learning Standards

              Student performance does not demonstrate an understanding of the

ELA/mathematics knowledge and skills expected at this grade level.

 

              Level 2:  Partially Meeting Learning Standards

              Student performance demonstrates a partial understanding of the

ELA/mathematics knowledge and skills expected at this grade level.

 

              Level 3:  Meeting Learning Standards

              Student performance demonstrates an understanding of the

ELA/mathematics knowledge and skills expected at this grade level.

 

              Level 4:  Meeting Learning Standards with Distinction

              Student performance demonstrates a thorough understanding of the

ELA/mathematics knowledge and skills expected at this grade level.

 

All students who score below the State-designated Performance Level 3 must be provided Academic Intervention Services by their school districts.

 

              New York State has developed its testing program to reflect the depth and breadth of the New York State Learning Standards.  All assessments contain both multiple-choice and constructed-response questions.  Some states limit their tests to only multiple choice questions. Multiple-choice answers are scored by computer and the constructed-response answers are scored locally by NYS teachers.

 

                The Grades 3-8 Testing Program has been externally peer reviewed and fully approved by the United States Education Department.  The Department has adhered to the Joint Standards for Educational and Psychological Testing to ensure that the tests are valid and reliable to ensure accurate classification decisions for both student support (Academic Intervention Services) and accountability determinations.

 

 

 

 

1. Should the Department modify the current testing calendar and administer assessments in mid (March) or late spring (April-May) starting with the 2010-11 school year?

 

At the December 2008 Regents meeting, the Board discussed whether to move administration of the grades 3-8 tests to the spring. Doing so would permit more time to teach the new year’s curriculum prior to the administration of the test. This would also align administration of the grades 3-8 tests with the time period for other exams.

 

Currently, ELA tests are administered in January and math tests are administered in March in a staggered format requested by the field; first grades 3-5 and then grades 6-8 are tested.  There is also a make-up period for the tests to ensure that the school systems have time to test students who were absent and provide accommodations to students with disabilities and English Language Learners. 

 

An analysis of other large states (Chart 1) shows that many states test later in the year, from mid to late spring, with test administration periods of varying length. California allows schools to select the dates of their administration period and requires a 21 day test administration period for both reading and mathematics. Illinois allows 10 days for both reading and mathematics. Ohio allows 15 days for both reading and mathematics. Pennsylvania and Texas allow 10 days for both reading and mathematics.

 

Many states choose to administer ELA and math at the same time, while others separate the test administration. Most states place the exams at a time when students have had a significant amount of instruction but not at the end of the academic year. This allows states the time they need to score the exams and to make more timely accountability determinations. Some of these large states return scores by the end of the school year, but others post their results during August.

 

The Department is working to find ways to issue test results as soon as possible after the tests are given and within the same school year. Last month we reported to the Board about the creation of a new system to speed test results to the schools within 10 weeks after the tests are given; that system will be in place for this testing cycle. This represents a major improvement over previous years, when it took from 17 to 33 weeks to return ELA scores and from 13 to 28 weeks to return math scores. We will weigh various options to reduce still further the time it takes to return test scores. Staff will continue to work with stakeholders to gather input regarding the administration of these exams for Board review.

 

 

 

 

 

 

 

 

CHART 1

 

LARGE SEVEN STATES*

COMPARISON OF ELEMENTARY AND INTERMEDIATE STATE EXAMINATIONS

 

State	Average Number of Students Tested per Grade	Test Content	Subject	Test Administration Window	Date Schools Get Results
California	481,000	MC	Reading and Math	California Department of Education does not establish testing schedules for school districts.  Schools administer the California Standards Tests within a 21-day window that is comprised of ten days before and 10 days after the day on which the school completes 85 percent of all instruction for the year.  District coordinators set the testing period within the 21-day regulatory window for the Standardized Testing and Reporting (STAR) Program.	The state, county, school district, and school summary reports will be posted on the Internet on August 15th each year at http://star.cde.ca.gov/.
Florida	200,000	MC, CR	Reading and Math	Late February and early to mid-March	May
Illinois	150,000	MC, CR MC, Short Answer	Reading and Math	March 2-13	August
New York	300,000	MC, CR	ELA and Math	Jan. 12-16 (Grades 3-5); Jan. 20-23 (Grades 6-8); Mar. 2-6 (Grades 3-5); Mar. 9-13 (Grades 6-8)	May: ELA June: Math
Ohio	132,000	MC, CR	Reading and Math	April 20 – May 8	June 30
Pennsylvania	130,000	MC, CR	Reading and Math	March 16-27	June 10
Texas	305,000	MC	Reading and Math	Reading: March 3 Math: April 7	March 20: Reading April 24: Math

* Largest states by population

 

Key:      MC = multiple choice

              CR = constructed response

Please note other states administer reading tests.  New York administers tests that measure English Language Arts (reading, writing, and listening).

 

 

• In order to return test scores sooner, should we take additional steps to change the test scoring and scanning process for the Grades 3-8 Testing Program? 

 

                Last month we reported to the Board that we have created a system to return test scores much more quickly (within 10 weeks after test administration), beginning with this school year. However, in future years, if the tests are administered later in the year, we may need to take additional actions to shorten the time even more in order to release results before the end of June, unless we release results later in the summer. (Please note that our benchmarking revealed that two large states release their grades 3-8 test scores in August.)

 

                Under the current system, the scoring of the extended-response questions by teachers is costly and both time and labor intensive.  The majority of school districts must complete this scoring during the contracted work day or incur additional labor costs.  We have consulted with the field, and they have asked that we not shorten the scoring timelines due to the volume of work and scheduling difficulties.

 

                In order to shorten the time it takes to return test results, the Board of Regents has asked about the potential to employ a single vendor that will scan multiple choice answer sheets, score constructed response items, and prepare a file for submission for psychometric research.  School districts would save money by not having to hire substitute teachers in order to free-up the teachers needed to score the exams. However, there would be considerable cost to the State to use this scanning and scoring process.  In addition, since many states administer these exams at the same time of year, demand on the vendors to scan and score is great and this could impact time frames.  Estimated State costs run between $17 and $25 million per year; however, the Department would not have exact costs unless it puts forward a Request for Proposal (RFP) and receives proposals from vendors.  We would need to secure additional funding in future State budgets to support this option.

 

 

1. Should we review and revise the current levels for proficiency in the grade 3-8 tests and raise the standards? 

 

              The Board of Regents raised standards for all students in 1996 and implemented the new grades 4 and 8 testing program in 1999 (later grades 3-8) to measure those higher standards. Prior to that, the Department gave much more basic tests that were designed only to determine whether students needed remediation.

 

              Since then, student performance across the State has improved markedly. Some observers have suggested that it is appropriate to raise the standards and the corresponding knowledge and skills required to achieve proficiency on the tests. Does the Board of Regents want the Department to revisit the cut scores of the current testing program to create more rigor?

 

1. Should we redesign the Grades 3-8 Testing Program to allow for more refined information regarding student progress and achievement, especially measuring “growth for all?”

 

The Regents have directed the Department to revise the accountability system in order to measure “growth for all.” This will mean giving recognition to all students who demonstrate appropriate growth from year to year throughout their schooling.  We are working with Dr. Brian Gong and other testing experts and will have preliminary options for the Regents to review at their meeting next month. In creating these options, we are reviewing and evaluating possible incorporation of the accountability programs used by other states and cities throughout the nation.

             

              Accomplishing this revision will require some important changes in the grades 3-8 tests. Most important, we must increase the number of multiple-choice items in each test so that we can more sensitively measure growth across the spectrum, from level 1 through 4.  In addition, according to Chapter 57 of the laws of New York State, we must create and maintain a vertical scale that will measure progress across all the grades 3-8 tests.  Moving to this more sophisticated vertical scaling method will help to provide the detailed information needed to make these determinations.

 

              The current contracts with the testing vendor, CTB/McGraw Hill, will expire after all work is completed for the 2009-10 school year. We can make the necessary changes described above when we produce revised tests under the new contract.