In an excellent article in the New York Times last week, Michael Winerip presented a 10 Years of Assessing Students With Scientific Exactitude--a synopsis of NYSED's recent testing efforts. It's difficult to fully enjoy Winerip's wit because the story presented is so sad--false hopes and failure at every turn.

In the last decade, we have emerged from the Education Stone Age. No longer must we rely on primitive tools like teachers and principals to assess children's academic progress. Thanks to the best education minds in Washington, Albany and Lower Manhattan, we now have finely calibrated state tests aligned with the highest academic standards. What follows is a look back at New York's long march to a new age of accountability.

DECEMBER 2002 The state's education commissioner, Richard P. Mills, reports to the state Regents: "Students are learning more than ever. Student achievement has improved in relation to the standards over recent years and continues to do so."

JANUARY 2003 New York becomes one of the first five states to have its testing system approved by federal officials under the new No Child Left Behind law. The Princeton Review rates New York's assessment program No. 1 in the country.

SPRING 2003 Teachers from around New York complain that the state's scoring of newly developed high school tests is out of whack, with biology and earth science tests being too easy and the physics test too hard. The state Council of School Superintendents finds the physics scores so unreliable, it sends a letter to colleges for the first time in its history urging them to disregard the test result. Dr. Mills does not flinch, calling the tests "statistically sound" and "in accordance with nationally accepted standards."

JUNE 2003 Scores on the state algebra test are so poorly calibrated that 70 percent of seniors fail. After a statewide outcry, officials agree to throw out the results. The Princeton Review says that ranking New York first was a mistake. "We're going to have to come up with a fiasco index for a state like New York that messes up a lot of people's lives," a spokesman says.

...

NYSED has published test results from the 2010-2011 school year. You can download the results from this NYSED page. The NYSED documents are quite large, though. For your convenience, here are the pages which cover the Newburgh School District schools (pages 1334 to 1338 of the PDF version).

This year the cut scores for performance levels were raised substantially, because SED determined that they had been set too low. So comparing to previous years is difficult. You can see how different the 2009-10 results are by viewing the pdf linked in this old post.

One thing that jumps out in these stats is that of the 108 "Out of District Placement" students tested, only one student on one exam scored at level 3 or 4. Oh, wait, that's nearly exactly what happened last year. One out of 95.

Excerpt from the NYSED press release on the changing of exam "proficiency" cut off scores:

As a result of raising the bar for what it means to be proficient, many fewer students met or exceeded the new Mathematics and English Proficiency standards in 2010 than in previous years. Across Grades 3-8 statewide, the majority of students, 53% in English and 61% in Math, met or exceeded the new Proficiency standards this year. By contrast, in 2009, 77% of students met or exceeded standards in English and 86% of students did so in Math.

On one hand, this seems to be a necessary step to make statewide assessments more meaningful. If the standards were set improperly it is good to correct them.

At the same time it seems misguided to expect that "raising the bar" will somehow cause improved achievement. How will this more meaningful information be put to use...

Ensuring that student achievement information provides meaningful information about student progress is just one element of the Regents' broader reform agenda. The State Education Department is working to enable educators and parents to make the most of that information - for example, by developing statewide curriculum models aligned with college and career-readiness standards, and by implementing a teacher and principal evaluation system that will provide differentiated professional development.

Oh.

There is a webcast of the July 28 news conference and here is a slide presentation and links to the data.

For your convenience here is pdf version of the assessment data for schools in the Newburgh School District. Of the 95 "Out of district placement" students, only one student on one exam scored at level 3 or 4.

Additional commentary on this topic at NYSSBA, Jerry Moore, EdVANTAGE, and recordonline.

Interesting article from The Buffalo News, July 8, by Mary B. Pasciak, Flawed tests distort sharp rise in scores by students

Over the last few years, student performance has soared on math and English tests across New York State, with the most dramatic improvements evident in urban districts such as Buffalo, leading many to celebrate the progress.

But now, state education officials say the progress may not have been quite what it seemed.

Weaknesses in the state's testing and scoring systems over the last several years created what Education Commissioner David M. Steiner equates to systemic "grade inflation."

Students who score at the "proficient" level in middle school math, for instance, stand only a 1-in-3 chance of doing well enough in high school to succeed in college math, he said.

"This is about telling the truth about how far we are from where we need to be for kids to succeed in college," said John B. King Jr., senior deputy education commissioner.

Steiner and King are calling for a host of reforms to the state's testing system, as well as the eventual adoption of a statewide curriculum.
...

An article at the New York Times, New Diploma Standard in New York Becomes a Multiple-Question Choice, raises several questions about NYS Regents exams.

• How will the new "five regents exam" graduation requirement affect graduation rates?
• Should testing standards align better with college requirements?
• What do the exams really test?

...
For a time, Mr. Decker encouraged teachers to spend more time delving into the philosophy behind the Bill of Rights, a line of inquiry that seldom leads to points on the test in United States history. But he drew back when he saw that many students were failing the exam.

Educators are also debating how the tests are graded. Teachers currently grade their own students' exams, which many critics say encourages grade inflation, particularly on tests that include essays, whose scoring is somewhat subjective. Because schools can be closed for sagging graduation rates, teachers have a stake in the scoring, and will have an even larger one two years from now.

"We've raised issues about scoring of the test for the last 10 years, but I don't see sufficient pressure for them to really examine it," said Howard T. Everson, a senior research fellow at CUNY and the chairman of a committee that advises the state on testing. "The problem is it would be expensive to do it any other way, but I don't see a plan for them to address it."

More important, Mr. Everson said, there has been little evidence that state officials are certain about what the exams really test and what their impact is.

"People have always taken the Regents as a fact of life," he said. "They were always seen as a benchmark of quality - it has always meant something. And does it still? That's the real question."
...

The Buffalo News reports that the State Education Commissioner, David M. Steiner, is concerned about too much focus on assessments. "Assessments" is the word which you use instead of "tests" when you want to pretend that there is something scientific about them. Do you know how many "benchmark" tests students in the Newburgh School District are given? Are the benchmark tests achieving their purpose? Does the Newburgh School District officially encourage any form of assessment other than tests?

Excessive testing in schools gets 'F'

Too many assessment exams can stifle curriculums, says state's new education chief

By Peter Simon

The state's new education commissioner says he wants to de-emphasize assessment testing, saying too much "teaching to the test" is going on in schools.

"In too many cases, the assessment becomes the curriculum," David M. Steiner said here Wednesday. "If the test is the curriculum, then you're tempted to teach to the test."

Steiner, who became education commissioner Oct. 1, told The Buffalo News Editorial Board that assessment testing still has a crucial role in the state's classrooms but that a more balanced approach is needed.

Robert M. Bennett, a member of the state Board of Regents who accompanied Steiner during his Buffalo visit, was more blunt in his comments, saying assessment testing "has become an obsession" in the state.
...

The press release at the Newburgh Schools site about improving ELA results contains this gem:

The accountability system, which results from federal “No Child Left Behind” legislation, requires school districts across the country to assess their students in grades 3-8 and at the high school level yearly in mathematics and English Language Arts and to rate them according to Performance Levels 1, 2, 3, and 4 with 4 representing advanced proficiency (85-100), 3 representing proficiency (65-84), 2 representing basic proficiency (55-64), and 1 representing basic performance (0-54). Performance Levels 1 and 2 indicate unacceptable performance.

That can't possibly be right can it? "Basic proficiency" used as a label for what old-timers would call a failing grade, for performance which NCLB categorizes "unacceptable".

According to the Accountability Peer Review document submitted by New York State Education Department and available from the federal Department of Education it is indeed possible:

The State has also defined an additional level of achievement: basic proficiency. Basic proficiency is defined as the performance of a student who scores Level 2 on the State assessments in grades 3-8 English language arts, mathematics; or scores Level 2 on a State alternate assessment; or for certain Limited English Proficient (LEP) students grades 3-8 shows Level 2 growth on the NYSESLAT; or scores between 55 and 64 on the Regents comprehensive examination in English or a Regents mathematics examination; or 65 or greater on a Regents competency test.

That about destroys any meaning "proficiency" once had.

The Times Herald-Record reports Newburgh School District improving rapidly in English Language Arts and there's a corresponding press release at the Newburgh Schools site. There are a couple of comments on the Record Forum here.

It is no doubt a good thing that a computer in Albany has churned through the numbers and categorized Newburgh Schools' 2006-2007 performance as better than its 2005-2006 performance.

Do the students, parents, and teachers in the district perceive this improvement as well? If so, to what do they attribute the improvement? If not, well, then what do we ask?

An excellent post titled Can Data Threaten Good Teaching? can be found at the Teacher Leaders Network website.

...
In order for data to be meaningful, as many variables as possible need to be controlled. If you are to compare students in different classrooms, the instruction needs to be as similar as possible. So as we go forward, if we view the solution to our educational troubles as being the generation of meaningful data, then we will tend to seek to standardize curriculum, instruction and assessment as much as possible, so the data will have the greatest possible validity.

But there are some huge problems with this approach. One large problem is that we are attempting to standardize all these factors in order to generate reliable data. But it is not clear that the new data is substantially better than the data we have had in the past, or that we have interventions that are appropriate responses to the data.

But the bigger question I would raise is the hidden cost of all this standardization. From the edu-technocrats’ point of view, it is no problem to expect the teachers to follow this or that timeline, or use this or that scripted curriculum. But as accomplished teachers, we know there is a huge cost paid by the teacher and students. The teacher has lost discretion to craft a sequence of lessons that reflects her best knowledge and her understanding of her students -- and the students have lost access to her special knowledge, and her ability to respond to their interests, strengths and weaknesses.

I think the standardized teaching model is having trouble. Its flagship vehicle, No Child Left Behind, is in grave jeopardy, and the serious problems of equity in our schools have not been substantially shifted, in spite of the "bright light" high-stakes test data have shone on them. As the model begins to sputter, its proponents look to rescue it by intensifying the standardization, making the systems even more thorough and reliable, unwilling to recognize the flaw in the premise.

The flaw in the premise, from my point of view, is that we are dealing with an immensely complex human enterprise. In a single classroom with 20 learners, we have a wide range of abilities and aptitudes, interests and backgrounds. No matter how good the scripted curriculum, no matter how sophisticated the tests, the decisions made by a strong teacher based on skillful classroom assessment practices will, on the whole, be better than those that would be made as a result of the standardized system.

In order to take advantage of this expertise, of course, you must empower and trust that teacher. You must invest in that teacher (and the development of more teachers like her), and create the conditions that sustain and support her. You can ask her to be accountable, but only if you give her the support and autonomy she needs to be effective and in charge of the learning for which you are making her responsible.
...

The teaching strategies promoted by the Newburgh School District Central Administration often incorporate this data driven approach. "Curriculum mapping", "Curriculum Alignment", the "Audit of Curriculum", the attempted quarterly district-wide subject tests all tend to view standardization as either something desireable or as a given. How are we empowering and trusting our teachers? How are we taking advantage of their exprtise? How are we encouraging them to develop skillful classroom assessment practices?

In a recent blog post titled The Inmates Who Want to Run the Asylum, Gerald Bracey takes to task the "idiots and professional fear mongers who are trying to control curriculum and instruction and education policy in our nation's schools..." Mr. Bracey is always worth reading carefully.

Journalist Fareed Zakaria noted that kids in Singapore score much higher than American kids on tests, but that 20 years later, it's the American kids who are ahead of the game. He asked the Singapore Minister of Education why this should be. The Minister said that while his kids had high test scores, American kids had talent. "We cannot use tests to measure creativity, ambition, or the willingness of students to question conventional wisdom. These are areas where Singapore must learn from America." Zakariya also quoted a father who had lived in America for a while and then moved back to Singapore: "In the American school, when my son would speak up, he was applauded and encouraged. In Singapore he's seen as pushy and weird. Schooling in Singapore is a chore. Work hard, memorize, test well" (the father put his kid in an American style private school).

Keith Baker, a retired U. S. Department of Education researcher got the same message from a Swedish father living in L. A.: "He holds a high position in a bioscience company," Baker wrote in the October Phi Delta Kappan. "He told me, 'There is no doubt that graduates of European high schools know a lot more than American grads, but I prefer my kids in American schools because Americans acquire a spirit that other countries lack.' Other anecdotal sources suggest this 'spirit' involves ambition, inquisitiveness, and perhaps most important, the absence of a fixation on testing and test scores."

Recall that psychologist Robert Sternberg called our high-stakes testing programs "one of the most effective vehicles this country has created for suppressing creativity."

How do Newburgh Schools promote innovation and creativity in our students?

More specifically, is promoting innovation and creativity a goal which the Newburgh Enlarged City School District Board of Education espouses?

Excellent article from the Santa Barbara Independent. Hat tip to the Schools Matter Blog.

The Failure of Testing to Address Actual Proficiency

My wife had a group of elementary school kids do a scavenger hunt at the Farmers Market to gather fruits and vegetables for a "tasting day" at school. On the morning of the tasting, big banners hung from the cafeteria ceiling for all to see--Tasting Day. A child walked into the cafeteria and visibly slumped, his smile vanished, and his eyes clouded. When my wife approached him, he said, "I thought we were done with testing."

Welcome to the post-accountability No Child Left Behind (NCLB) world, where no child is left untested. Are today’s students better readers and mathematicians than 10 or 20 years ago? Has all this emphasis on content standards, annual testing, and exit exams worked? No. It is time to recognize that the standards-based, test-driven, school-accountability movement has failed. After 10 years, these measures have not improved education.

NCLB is based on the idea that after 12 years of testing and punishment for under-achieving students and schools, 100 percent of the nation’s students will be English and math proficient. For the last six years of NCLB, national reading proficiency has remained nearly flat. Math proficiency has improved, but at the same pace as before NCLB. Now we face a balloon payment of proficiency. For the next six years, students climb the Everest of proficiency, during which all students--regardless of language, ethnicity, economic status, or disability--must rise to 100 percent proficiency.

Given that only 24-34 percent of the nation’s students have been proficient readers for the last 20 to 30 years, it would take a miracle to achieve a national policy of Only Half Our Children Left Behind. On this preposterous road to universal proficiency, we abandoned the idea that education is about more than annual test performance. Millions of tax dollars were given to private test companies like ETS and McGraw-Hill. Art, music, and shop classes vanished. Average class size grew. More kids dropped out.

In 2005, Harvard’s Civil Rights Project conducted a study that tracked California 9th graders during four years to gauge how many received their diploma by the end of 12th grade. Only 71 percent of these 9th graders graduated with their peers. Twenty-nine percent--nearly a third--had disappeared from public school four years later. Minorities fared worst: 42 percent of African Americans, 40 percent of Latinos, and 48 percent of Native Americans did not graduate high school.

No one noticed that the blind academics-only routine was driving fragile, disadvantaged teens out onto the streets. Consider the schedule of a below-proficient high school student: two English classes, two math classes, a science, a history, maybe physical education, and no electives, sports, art, shop, or music. Or consider the teacher--required to teach only specific standards at a mechanical pace. The pace dictated by the "if it’s October, this must be Standard 5" calendar. No adaptation was made for the learning needs of individuals. Teachers were told to teach the test, not students.

Decoupling what is taught from what students know is the single greatest flaw in the accountability movement. Having a common set of standards for every subject makes some sense. It is easy, however, to list everything we’d like kids to know. The hard part is getting them to care about learning it.

...

The Sentinel published a not so well reasoned editorial on October 26. This prompted a spirited response from an elementary school principal in the Newburgh School District on November 9. The Sentinel deserves credit for publishing such a critical response, and for their usually more supportive attitude toward educators, which the response letter acknowledges. The principal has provided a necessary dose of reality and reason to the conversation, well worth a read.

An article in The Guardian, reports on "pervasive anxiety" experienced by youngsters and cites national tests as a cause. The article does suggest some steps which might help alleviate stress...

Researchers found that pupils in schools which tackled the problems they worried about, such as those with eco-clubs and recycling schemes to teach children about environmental problems, were happier.

"Where schools had started engaging children with global and local realities as aspects of their education they were noticeably more upbeat. In several schools children were involved in environmental projects and the sense that 'we can do something about it' seemed to make all the difference," they write.

School staff told the researchers that some parents were not involved enough, while others were too "pushy" and demanding of their children academically. The General Teaching Council for England opposes national tests for seven, 11 and 14-year-olds, the results of which are published in league tables and scrutinised by parents when choosing a school. Wales has scrapped them. Ministers in England are adamant that they will remain.

Hat tip to Schools Matter Blog.

At a dedication ceremony last week for a new playground at Horizons-on-the-Hudson Magnet School a student is quoted:

"At first there was nothing out here," said sixth-grader Savannah Ordonez, a member of the school's "Gardening Option" program, "and then we started clearing it out and planting. Now it's the most beautiful part of the school."

As she, and all students in grades 2 through 11 thoughout the district suffer the first bout of new district mandated "benchmark assessments" this week, we hope that Savannah remembers that there was at least one time when she and her classmates took a place where there was nothing and created something beautiful. And there was at least one time when teachers took the risk to try something with no guaranteed outcome. And there was a time to design, create, build, and accomplish something real.

We hope she remembers, yet we fear that by the third or fourth bout of "benchmark assessments" (there are four scheduled this school year!) she may succumb, and be tricked or brainwashed into thinking that filling in bubbles with a number two pencil is as important as having planted a seed.

Thanks to the staff of Horizons-on-the Hudson and all the volunteers who helped, for teaching creatively; thank you Senator Larkin, for finding funds for this project; and thank you to the Record for reporting this story.