To the Editor:
Although Designs for Change virtually never complains about newspaper stories, I feel compelled to say that I am astonished by the negative reporting in your Dec. 9, 1992, article titled “Chicago Principals Finding Reform Demands Overwhelming.’'
The Chicago principal survey analyzed in the article contains an extraordinary set of findings that you ignored. Chicago principals were the individuals most radically affected by reform and thus are the individuals one would predict would be most negative about it. Further, when the reform law passed, many analysts predicted principals would be at war with the local school councils that now have the power to hire and fire them, and that a sharply negative impact on the quality of education would result.
Yet in this survey, the principals themselves offer opinions like the following:
- “Since reform, this school is getting better.’' (Eighty-three percent agree; 17 percent disagree.)
- “Since reform, I am more optimistic that this school will improve.’' (Sixty-two percent agree; 38 percent disagree.)
- “Since reform, staff development is more responsive to teachers’ needs.’' (Sixty-six percent agree; 34 percent disagree.)
- “Since reform, there is more conflict in this school.’' (Only 29 percent agree; 71 percent disagree.)
- “The [local school council] and I have a similar understanding of the principal’s responsibilities and rights.’' (Eighty percent agree; 12 percent disagree; 8 percent neutral.)
- “The L.S.C. has a clear understanding of its role and responsibilities.’' (Sixty-one percent agree; 24 percent disagree; 15 percent neutral.)
- “The L.S.C. pressures me to spend money in ways that I think are inappropriate.’' (Five percent agree; 87 percent disagree; 8 percent neutral.)
- “The L.S.C. meddles in internal school matters.’' (Fourteen percent agree; 76 percent disagree; 10 percent neutral.)
- “The L.S.C. contributes to academic improvements.’' (Fifty-eight percent agree; 20 percent disagree; 22 percent neutral.)
- “The L.S.C. is an effective policymaking body.’' (Fifty-five percent agree; 18 percent disagree; 27 percent neutral.)
These positive results are coming from the principals themselves, notwithstanding the fact that the old-guard Chicago Principals Association has spent hundreds of thousands of dollars in court seeking to have the reform law overturned, and that the association’s president, Bruce Berndt, who is the only person quoted in your article, constantly seeks to cast Chicago reform in the most negative light possible.
Yet, although the principals’ unexpectedly positive reports about successes in implementing a massive organizational change constituted a substantial portion of the survey, none of these specifics were reported. In fact, these positive results don’t even rate a full sentence in your article, which dismisses them with the following dependent clause: “Although Chicago elementary and high school principals give the city’s school reform high marks, ...’'
Another major set of findings virtually ignored relates to the extent of changes that have already occurred in schools and classrooms, over and above successful implementation of the new decisionmaking structure. According to the survey findings, the percentage of principals reporting “extensive or moderate’’ implementation of innovative classroom teaching practices has increased from 24 percent of schools before reform to 52 percent now.
Similarly, extensive or moderate restructuring of teachers’ work has increased from 15 percent of schools before reform to 35 percent today, and initiatives to build parent and community involvement have increased from 26 percent of schools before reform to 60 percent today.
The article does state one highly encouraging result related to these findings: "[T]he most dramatic restructuring tended to take place in the least-advantaged schools--schools with low pre-reform achievement levels or with enrollments comprising primarily minority or low-income students.’' But since no specific findings are presented to clarify what “restructuring’’ means or what percentage of schools are affected, the import of the statement is obscured.
Having virtually ignored the sizable range of positive results in the study, the article focuses almost entirely on issues and problems highlighted by the study. While results pertaining to some of these issues are reported accurately, the article in other instances contains statements that generalize far beyond the findings reported, and fails to report findings that don’t fit such over-generalizations.
For example, principals cite a variety of constraints that are making their job difficult, including increased work demands caused by reform, lack of sufficient resources for staff development, and difficulty in removing nonperforming teachers (problems that, incidentally, are not unique to Chicago). However, the article leaps far beyond such specific survey findings by asserting that “the principals also say they lack the authority and resources to carry reforms further, the study found.’' Nowhere was a question resembling this sweeping assertion asked in the study, and contrary evidence to this assertion recurs in specific study results. For instance:
- Sixty-two percent of principals agreed with the statement that “since reform, I am more optimistic that this school will improve.’'
- Ninety-four percent agreed with the statement that “I am making a difference in the academic development of my students.’'
- Sixty percent disagreed with the statement that “my success or failure as a principal is due primarily to factors beyond my control.’'
In other instances, the article chooses to highlight the specific response to one particular question, while failing to report the other related responses that place that finding in a broader context.
For example, it states that “80 percent of the administrators hired before 1988 said they received less respect from the public than they did before the reform.’' But it fails to report specifically that 78 percent of principals agreed with the statement that “in this community, I have high status.’'
When weighing the full range of related study responses, the pattern that emerges is that most principals feel they are well respected by parents in their schools and by their immediate community (with which reform has typically brought them into a much closer working relationship), but that they have less respect from the Chicago public in general.
One particularly unbalanced aspect of your article is that none of the study’s authors were quoted. Nor was any representative of an organization that supports Chicago reform quoted.
As a whole, this study documents major progress of Chicago school reform, attested to by the individuals one would expect to be the most negative about reform. It also documents significant problems that reform faces. Your article reports the problems as if they were the whole story, systematically ignoring the striking documentation of progress.
Donald R. Moore
Executive Director
Designs for Change
Chicago, Ill.
To the Editor:
I am surprised that the findings of the Claremont University study (“Study ‘From Inside’ Finds a Deeper Set of School Problems,’' Dec. 2, 1992), regarding the primacy of “relationships’’ in the restructure of schools did not mention the Schools Development Program, one of the most successful reform efforts in the country.
Established in 1968 by the Yale University psychiatrist James P. Comer, S.D.P. identified relationships as the major problem in schools. Dr. Comer believes that good relationships lead to orderly and supportive school environments, which, in turn, support student development and provide the necessary foundation for academic achievement, regardless of the socioeconomic status or racial background of children.
Dr. Comer’s program creates structures and processes which promote good relationships in schools. At the most successful of the more than 200 Comer schools across the country, the atmosphere for learning is inspiring, academic achievement is high, and children experience adults working in a collaborative, purposeful manner.
I applaud the Claremont study. It clearly addresses the critical importance of good relationships; however, it is important to acknowledge that for nearly 25 years, James P. Comer has been pointing out and demonstrating that improving relationships, more than anything else, is the critical issue in transforming the schools.
Beverly Caffee Glenn
Howard University
Washington, D.C.
To the Editor:
The saddest part of Susan Ohanian’s Commentary on the teaching of literature in schools (“The Classroom Structures That Really Count,’' Commentary, Dec. 2, 1992) is that it echoes a battle I fought as a beginning English teacher 25 years ago. With all the talk of reform, curricular diversity, and change, teachers are still forced to mete out Charles Dickens to students--teaching them, as John Dewey once pointed out, that the cost of teaching only “great works of literature’’ is students’ learning that all “great literature’’ is tiresome, boring, and dull.
Ironically, many of the books traditionally taught in high schools were initially included on book lists because, at the time the lists were constructed, they were “contemporary literature.’' In the 1870’s, when
Dickens became recommended preparation for the Harvard admissions examination--a widely used precursor of the Scholastic Aptitude Test--Dickens was still alive and writing, even touring the United States. The Red Badge of Courage became standard high school reading in the 1910’s, as part of the 1918 curriculumrevision; the book was less than 25 years old at the time.
Likewise, the ink was barely dry on All Quiet on the Western Front when that book was first handed out to high schoolers. And so it goes.
After battling a department chairman and supervisor in the 1960’s, I was granted “experimental permission’’ to operate a free-choice reading program in my 11th- and 12th-grade classes; a year later, the permission was nearly revoked because the program was too costly: Students were stealing the books, lending them to family and friends! Do we never learn?
Stephen E. Phillips
Brooklyn, N.Y.
To the Editor
There is much that can be criticized in Barry Gross’s Commentary, “Our Students Are Better Than Our Schools’’ (Nov. 25, 1992).
One of the most illogical recommendations is Professor Gross’s embrace of a combined Scholastic Aptitude Test score of 1200 as an absolute arbiter of scholarship eligibility. This would virtually guarantee that the nation’s classrooms would become even more like test-coaching centers. White males, who already receive higher S.A.T. scores, in part due to the test’s biases, would be further advantaged. The profits of commercial “test prep’’ businesses would soar as well-to-do parents sought to give their children another leg up.
Most outrageous is Mr. Gross’s insistence that “no attack on the S.A.T. would be tolerated.’' This attitude is all too typical of what has been called “the cult of mental measurement.’' Rather than addressing the dozens of academic studies demonstrating the lack of validity, inaccuracy, and “coach-ability’’ of the S.A.T., Mr. Gross would decree silence.
Serious students of educational reform should reject any scheme based on such a narrow-minded, “Don’t confuse me with facts, my mind’s made up’’ analysis.
Robert A. Schaeffer
Public Education Director
National Center for Fair and
Open Testing (FairTest)
Cambridge, Mass.
To the Editor:
As an administrator in special education, I am sensitive to how students with disabilities are portrayed. For example, in a caption for a photograph in your Across the Nation section for Dec. 9, 1992, the phrase “handicapped children’’ was used.
I would like to offer the following suggestion: When describing students with disabilities, it is best practice to put the person first. For example, a child with Down’s Syndrome, a high school student with cerebral palsy, and so forth, instead of putting the disability first, as in, “a handicapped child.’'
This manner of speech and written expression is vital in the continued movement to bring all students and educators together. As your publication is in the forefront of education, attention to this detail is necessary.
Betsy J. Shelley
Assistant Director
Integrated Program Models
Plainville High School
Plainville, Conn.
To the Editor:
Your front page article on RAND’s evaluation of the Vermont portfolio assessment program (“RAND Study Finds Serious Problems in Vt. Portfolio Program,’' Dec. 16, 1992) misquotes me and, in doing so, seriously overstates the negative conclusions of the RAND study.
Your article asserts that “Daniel Koretz, ... the report’s author, said that the low levels of reliability indicate that the scores are essentially meaningless [emphasis added].’' Neither I nor the report characterized the Vermont data as meaningless. As I noted when interviewed for the story, sufficiently severe unreliability would indeed render scores meaningless, but we did not conclude that the Vermont data were that unreliable. Indeed, the report distinguishes in some detail between results that are too unreliable to use and others that are acceptably reliable. For example, the summary of the report states that “despite the unreliability with which individual students’ work was scored, statewide average scores are quite reliable because of the large numbers of students represented.’'
Given the importance of the Vermont program to both the state’s reform agenda and the national debate about assessment, it is important to clarify what the report--and I in my presentations of it--actually did say about the quality of the portfolio scores. It is fair to say, as your article does, that we found serious problems with the portfolio scores. The level of inter-rater agreement was indeed low--low enough to place severe constraints on appropriate uses of the data. Moreover, the low level of inter-rater agreement precludes most of the analysis that is needed to validate the assessment, and it badly biases some of the statistics the state had hoped to report.
Nonetheless, it is important not to overstate the bad news. Rater reliability showed considerable variation, reaching a high (given the state’s preferred methods of reporting) of 0.57. (This was for the criterion of “usage, grammar, and mechanics’’ in the writing portfolio program.) Moreover, if the state had opted for simpler reporting strategies--for example, if students had been given a single total score for each piece of work rather than five to seven scores on qualitatively distinct criteria--inter-rater agreement would have been improved somewhat.
We reported that combining scores across rating criteria, combining “best pieces’’ and the rest of the portfolios in writing, and changing the method for combining scores across pieces in mathematics would have resulted in correlations between raters ranging from 0.49 to 0.60--not stellar, but hardly an indication of “meaninglessness.’'
In my view, these findings underscore a need for caution, for realistic expectations about innovative performance-assessment systems, and for independent research that will inform policymakers about the strengths and weaknesses of their programs. As I said when I was interviewed for your story, I have been concerned for some time by the excessive optimism about performance-assessment programs shown by many parties to the national debate. State Superintendent Rick Mills and other policymakers in Vermont have avoided that overconfidence, and they deserve commendation for their uncommon insistence--apparently not shaken by this report--that their program be subjected to ongoing, independent scrutiny.
But just as it is important to be realistic about the difficulties that confront performance-assessment programs and to investigate their successes and failures, it is important not to overstate the problems that are uncovered.
Daniel Koretz
Resident Scholar
Ráîä Institute on Education and
Training
Washington, D.C.
