Kevin Bushweller (Moderator):
    Welcome to today's online chat about the impact of high-stakes testing. We have a large volume of questions, so let's get the discussion started ...

---

Question from Stacey Wilson, Teacher, Desert Edge High School:
    Science high-stakes testing is next on the NCLB list, and already in my district there have been recent curriculum changes, such as fund cuts for all dissections in Biology, simply because, "Those lesson objectives are not related directly to a specific standard and therefore will not be tested, so we will no longer be doing them." How do you see high-stakes testing affecting science curriculums?

David Berliner:
    I am sorry but high-stakes testing, as is true of all NCLB testing, always--yes always--narrows the curriculum. The state, the district, your principal, any teachers that fear for their scores, all will do the same thing when the stakes are made too high--They will teach what is on the test and ignore what is not. Our book documents some of the most egregious of these cuts, but your story ranks right up there. The best administrator I knew in NYC said to all her teachers ignore the test--"lets provide a rich and rigorous curriculum and let the chips fall where they may." Even with her low income kids she always had kids that did Ok so she got away with that. You'll need to protest. Just like if the football team lost its jerseys, parents can be mobilized over this issue too. They need to know that the time and money sspent for dissection is a part of the science curriculum standards that the national science groups expect, thus such activities need to be in the local curriculum. I wish you luck. its a sad time and someone in your district is making what i believe to be foolish decisions.

---

Question from Robin Blum, Resource Specialist, Marin School, Albany CA:
    I have just spent the past 3 days monitoring a number of my 4th grade special education students who are taking high stakes testing with accommodations. One of my biggest concerns is for the pressure I am given to test these students with grade level tests and without modifications. I feel that the negative impact on self worth is huge when they are told that guessing answers is more important (this is what students must do when they are reading 2 years below grade level and can't read the lengthy passages that are part of the elementary tests). I believe that allowing the use of out of level testing or reading the lengthy reading comprehension prompts to these students would give them the opportunity to show what they do know - rather than encourage arbitrary guessing. When the experience is so beyond their level and they must so many hours completing the process - there is a huge impact on self esteem. How do you feel about this?

Sharon Nichols:
    We couldn't agree more. This is a terrible situation that affects so many of our students. It is terrible to put students in a position that simply tells them over and over again "you really can't do this" or "you don't know how to do this well" We worry excessively about this.

we didn't spend a lot of time on this in the book--i.e., on how these kinds of tests are socializing our youth. We suspect special education students' sense of motivation, self esteem, and confidence are being seriously harmed by high-stakes tests.

It simply makes no sense to put these kids in this position. One reason is that the test ends up not being valid--doesn't represent what they know or don't know well at all. But, as you point, out, this has serious consequences for the dispsitions and attitudes of our students. it is terrible.

---

Question from Bob Frangione, Teacher/Parent:
    Is "accountability" really necessary for educational reform? If so, what are some of the reasons that high-stakes testing is not working to make schools, districts, and teachers more "accountable."

David Berliner:
    I think accountability is needed for any expenditure of public dollars. But having people give an account--telling what they are doing to educate kids is not the way we have chosen to view accountability. Its our narrowing of this concept of accountability to tests thats one problem and then we make it worse by narrowing our notion of acceptable data to tests that are cheap to make and score. Both forms of narrowing make the system we have created awful. By putting high-stakes on the results of these tests we have school personnel avoiding real accountability and giving the politicians what they want--high test scores! So every state has its own state test scores going up ( through smoke and mirrors, drill and practice, cheating, forcing kids out, etc--all described in our book) and this allows all politicians say "see, we are holding the schools accountable" and many folks are happy. But the kids are getting a lousy curriculum and the teachers are not allowed to use their best professional judgment and their morale has been destroyed. If we really believed in accountability we would hold communities accountable for sending to the schools kids who are healthy and have health care, who have not been fed junk food and watched tv until late, who dont work over 20 hours a week in high school, and so forth. We dont hold our communties accountable for sending kids to school ready to learn, but then we hold the schools accountable for the results of that. It's a one way accountabiltiy system we have and we need a two way accountablity system, where community health and welfare is demanded before we punish schools for a lot that is out of their control.

---

Question from Jill Schwarz, curriculum committee member, STMA School District:
    I think that there is a problem with high stakes testing only because the morality of our population is so low. So many teachers and administrators are willing to cheat or look the other way. Is there not an ounce of honesty left in our public schools? NCLB assumes people will be honest. Isn't that the bottom line with this topic?

Sharon Nichols:
    We think this is crucial. In our book we talk about how high-stakes testing creates a culture where cheating is more likely to happen and seems to be more justified. This is exactly the problem we worry about under high-stakes testing.

I think we have to be careful about condemning our teachers. we learned that although some adults are cheating, we suspect that not all acts of cheating are equal. Maybe it is appropriate to "bend the rules" in the name of helping a student be successful? We don't condone outright cheating, but again, we also won't blanketly say, adults are cheaters....but this is problematic under high-stakes testing for sure.

---

Question from Terrill S. Wyche: Wayne County Community College District:
    What purpose does the testing system serve? Couldn't a portfolio assessment system serve the same purpose as the testing system?

David Berliner:
    The issue i think is how important is the decision to be made. I wouldn't want to base a decision on the knowledge base of a dental hygienist on the basis of the portfolio she presents to me were i the state examiner--its too ideosyncratic and may not cover all the areas Id want info about. But were i her instructor Id want to see it and could use it to make sure that there was evidence she was progressing well in covering all the things she should be covering. Portfolios so far don't have enough of the psychometric properties needed to make important decisions about kids, schools or a states' performance on academic standards. But if there wern't high stakes attached to these tests then a portfolio could be used. So they are great in a classroom, for sharing growth of kids with their parents, and even in a university, to show how scholars or artists or architects have made progress in what they can do. But for external audits of how a school is doing they seem to fail the test of reliabilty and validity. So if we werent seeking higher precision to make important decisions about people, schools and districts they might work. In more informal evaluations and accountabilty settings they work just great.

---

Question from Bob Alexander, President, MaxTheTest.com:
    In states with high stakes graduation requirements using state-developed tests, have we seen any indication that there's been an increase in national testing performance, e.g., SATs and ACTs?

David Berliner:
    Actually the opposite seems true. There are three studies I encountered that show a LOSS in SAT scores and i think there was one that showed a loss in ACT scores as a function of high stakes testing. This actually makes some sense. If a state develops standards and tests to match the standards, then they have a more circumscribed curriculum because their curriculum must reflect those standards and those tests. On the other hand the SAT and the ACT (especially) have a conception of the common high school curriculum and base their tests on that common/50 state/ typical curriculum. If your state moves into high-stakes testing you've narrowed your states curriculum and it stands to reason that you've lost a point here and there on the SATs and the ACTs. Thats what seemes to have happenned. There is good evidence that the NAEP hasnt moved as a function of making all states high-stakes testing states. In the journal Education Policy Analysis Archives is an article by Nichols, Glass and Berliner which makes that point quite well.

---

Question from Mary Thomas, third grade teacher, District of Columbia Public Schools:
    No Child Left Behind has sent elementary schools and their administrations into testing frenzies that I think consume large portions of the school year and increasingly reduce opportunities for instruction; however, students are constantly tested to enter private school, or high school, or colleges and universities. Shouldn't we broaden our discussion of testing and rethink its role in our society, its value, and what it really measures?

David Berliner:
    yes--you are quite right. We seem to be the worlds leader in numbers of tests that our youth take, but the world is catching up to our nuttiness. This is all part of the technological age we live in where it assumed that the testing technology, like other technologies, works as intended. But it doesn't. For example we know that the errors of classification made in NCLB are enormous, but tis gets overlooked. We can sense that something is wrong when the community loves its schools, and they seem successful by most standards, even winning awards at the state and national levels, and yet in NCLB they are declared by the tests to be failing. Something is really wrong with this scenario. Tests of all types, particularly standardized tests, can be very useful pieces of information. Its when too much is made of the scores that everything falls apart. Of course we need good test for medical baords and driving cars. But for kids and teachers we need formative testing (testing to inform instruction) a lot more than we need summative testing ( testing to give a grade or make AYP) (we say more on this distinction in our book.) The Europeans make the distinction between testing for instruction (to teach better) and testing of instruction (to see how the kids learned). The former is not well done in the US and rely too much on the latter. Good tests can really help us. Bad tests mislead us and have the illusion of precision that they often really are lacking. We trust too many tests to do jobs they are not psychometrically able to do well at all.

---

Question from James Calantjis,teacher,Grover Cleveland HS,NYC:
    Standardized testing has highlighted the educational crisis we are in that was hidden by social promotions.Isn't it true that there is no need to teach to the test as students should be able to pass a test if they have learned the curriculum? Shouldn't the focus be on the problems with students mastering the subject matter rather than the assessments(tests)?

David Berliner:
    You shouldnt confuse the social promotion issue with the testing issues. Actually there is a big and quite convincing literature that says that promoting kids and trying to help them is much much better than leaving them back. The trick is to help them, not abandon them, regardless of whether you leave them back or promote them. The testing issue is this: if you have good tests--rich, complex, constructed repsonse items, public performances, etc.-- They are can be taught to and we'd all be better off. But these are not the tests we have. Instead we have memory items, mostly in multiple choice format, and teaching to these kinds of test is not very educational. It sends students the message that knowledge is fractionated, that it exists as independent units that can be crammed into your head and regurgitated. Teaching to do well on those kinds of tests is not healthy. The better/richer the performances required by the test the more likely it is that we'd want to teach to it directly.

---

Question from Michael Fisher, Student, University:
    I agree with you on thoughts of formative versus summative assessments, buts since you brought up the comparison with Iraq; Would you agree that educators, whether administrative personnel or teachers, are not doing enough to help themselves in the education of students? Much like the Iraqis are not doing enough to help themselves. I dont believe the NCLB was designed to be a one way street. Do you? It seems to me if educators spent as much effort to work with-in the confines of the NCLB as they do complaining about and fighting it the general public would be much better off! Thanks in advance for your comment!

Sharon Nichols:
    Unfortunately, I do not think you are alone in holding a "blame the educator" attitude. My response is to simply ask, why would you want a teacher to simply work within a system they know is damaging and problematic? Wouldn't you want your teacher to stand up for the kids? to stand up for your child when the system they are asked to buy into is unfair and worse, harmful?

Teachers all around are not given the trust and respect most of them deserve to be able to make decisions for themselves. the "complaining" to which you refer is simply the voice of all teachers standing up and telling policymakers--many of whom have never stepped foot in a classroom--that they don't have it right. The policy of high-stakes testing is not right. But more importantly, it isn't working.

Were they simply complaining during the civil rights movement? Or, appropriately fighting for what is right. We truly need to support our teachers and listen to them. they know what is going on in teh classrooms and they know what is best for our students. It is amazing that we tend to listen more to policymakers than to those who are actually in the profession when it comes to education.

I think teachers are working very hard within the confines of NCLB-- they are doing their best. However, we rarely hear these stories. Our culture must value our teachers for what they do and what they know.

---

Question from Susan Garsoe, Fifth Grade Teacher, BV Elementary:
    Do you foresee that states will stop giving high-stakes tests? I see a whole body of businesses cropping up around the state tests, including selling schools practice tests, tutors in schools who prep kids for the tests, in-service trainings to help teachers get better scores. I'll bet there are lobbyists for the test publishers lurking around the halls of state legislatures.

Sharon Nichols:
    In order to answer your question, I need to clarify what you're asking. First, if you're asking if testing will end anytime soon, then I believe the answer is probably no. As you mention in your question, testing is a big business and there seems to be little doubt that part of what is fueling NCLB is somewhat about the profitable side of the business of testing.

Still, business aside, standardized tests should probably not go away. In fact, in our book, we argue that we are not anti testing. Standardized tests--when they are constructed well and have appropriate validity and reliability--are incredibly valuable for providing information about students and the nature of the curricula that is being taught. But, it is only when stakes are attached to them, that tests become questionable.

If you're asking if we think high-stakes testing--which is simply a form of testing where consequences are tied to test performance--will go away, our answer is "we certainly hope so!"

We hope to get more policymakers to read our book to understand more fully the damage that follows when stakes are attached to testing.

therefore, it isn't testing per se that is bad...but how they are used that is the demon. That is what we hope to stop.

---

Question from Larry -Ind. Observer:
    If we have students that cannot read and write at an eigth grade level, the government needs to hold teachers accountable for this. Bottom line.

David Berliner:
     This is a simple statement and quite wrong as it stands. You would be right if the eighth grade child were born under lucky stars--a two parent family that earns enough money to provide medical attention and cultural opportunities, a parent that has time to help with homework, parents that provided a high quality preschool, and so forth. But would you hold the teachers responsible for the kids being on "grade level" if it was the kid of a single parent working two jobs while the kid has some responsibility to help his/her sibs? Where no one ever has had health care in the family and so the kid was born a little under weight affecting his/her IQ a little and then misses 20 school days every year due to asthma? Or where the neighborhood is crime and drug ridden so the kid has to live behind locked doors in fear of a bullet? How about the million plus mostly low income kids that are lead poisoned and mercury affected? What about the kids that have immigrated by the millions in recent years and arent at grade level because they are never given the benefits of good bilingual or dual immersion programs? No, Larry--you are much to glib. Teachers are responsible and so is our nation for the kids we have. If many urban teachers (who may be the newest and the ones most likely to be teaching out of field) get kids who have trouble learning they need to do their best, but they are not the only ones responsible for achievement. Large numbers of our kids live impoverished lives and this is not easily fixed in 6 hours a day for 180 days. Sorry Larry, it takes a village that includes more than its teachers, and we have failed to take that proverb seriously.

---

Question from Dorothy Johnson, teacher, Sedgwick Middle School:
    How do we get out of the quagmire created by "No Child Left Behind?" I teach science in the middle school and, at times, math and science. Our state and system have always placed a high priority on testing, way before the NCLB. But, before I could go in depth in my subject area, provide lots of "hands on" lab experiences and time for thoughtful questioning. Although NCLB purports to provide for "thinking" and "problem solving", I find that I am forced to teach too much curriculum, too many facts and too much time spent on pre testing. Truthfully, I can't teach this way. It is not me. Throwing facts at children in middle school and expecting them to remember them 2-3-4 years later is ludicrous. I don't believe the public is aware of what the law is doing and the stress and pressure created on principals and teachers is resulting in low morale and burnout, of the the best.

David Berliner:
     I am so sorry for you. What you say is said by hundreds of others--and we have similar quotes in our book. If you get our book and believe that it illuminates the problems you state, as we think it does, please pass it on to a legilator or the staffers of the legislature. Make sure everyone knows what a failure this law is. There is only one way to make changes ina democracy--thats by the vote. organize with your fellow teachers. Be blunt--you'll not vote for anyone that supports this law that is bad for children, bad for teachers and bad for the future of this country. Please take some time out of your busy schedule and organize with others. We took time out to write this book to help you--we hope we can all amke a difference.

---

Question from Marcia Jensen, Project Director, Iowa Dept. of Education:
    The "highly qualified teacher" aspects of NCLB have far-reaching effects on future teachers. Can you comment on the increasing demand for a high-stakes test for teacher candidates now being mandated to states as part of licensure and certification?

Sharon Nichols:
    I think it raises many red flags. One problem we see over and over again from the impact of high-stakes testing in grades k-12 is teaching to the test and curriculum narrowing. Because there is so much importance placed on the tests, people who are feeling the pressure begin to make all sorts of adjustments to prepare students specifically for the test--which means many things that are not tested get left out. Ultimately, then, we don't know what we're testing---what kids know about math or reading What is problematic is that in a high-stakes testing situation, at any level or for any programs, it forces the curriculum to focus solely on those things that are tested and therefore, things that are not tested are devalued, and de emphasized. In teacher preparation programs, this is troubling. Teaching is so much more than what one can represent in a paper and pencil examination. For example, how will we know of a teacher's capacity to care? How to discipline students? How they will explain grades to parents? The problem is, if it is not on the test, it won't be emphasized.

It boils down to the question of "how are the test scores being used" If teacher exams have high-stakes attached to them, one must worry about the corruption and distortion of the type we share in our book.

---

Question from Roberto Serrato, Vice Principal, Madera Unified:
    High stakes testing was put in place because we were and are a nation at risk. What can and be done by the Federal and local institutions to make high stakes testing work? We are in this together. Do not start all over from scratch. A combination of past and present practices?

Sharon Nichols:
    well, I think we first need to question whether the risk is real or illusionary. I am not sure we are at risk--But maybe the better way to think about this is not whether we are or are not at risk, but what does it even mean to be at risk in the first place? My co-author in "manufactured crisis" authoritatively argued that we are not doing as bad as the fear mongers would lead us to believe, IN GENERAL. Still, we can always do better...it is just that different schools with different student populations require different solutions. and a significant piece of the puzzle is the economic disparities in which our youth live.

HIgh-stakes testing is a monolithic answer to a differentiated problem.

further, why do we need all students to perform at the same level anyway? Why not create a system that celebrates our students' diverse interests and talents and allows them to move forward on their own path instead of always defining it for them?

but, we cannot "Make high stakes testing work" because it does not work as our book describes. it leads to all types of corruption and distortion of the educational process. this is not acceptable. However, we can implement multi dimensional practices that minimize the importance of the tests. Im keeping tests, we can keep what we're already, but create an alternative system to evaluating effectiveness. we propose some ideas in our book.

but again, I do not think we can force high-stake testing to work. by its very definition, high-stakes testing creates situations that undermine test validity and sound educational decision making. there is no way to force that to work....

---

Question from Walter Carlson, Parent, Fairfax, Virginia:
    I agree there are adverse effects of the NCLB but what other choice does society have when after 3 generations the nation's schools have not been able to address the minority achievement gap -- regardless of how much money they've been given. Tests provide a means to identify were problems are and let focus schools on them. Even many educators are admittng that tests do help identify children who need help and schools that are failing. What other choice do we have besides tests?

David Berliner:
     I think this is a bogus strength of NCLB and the tests they demand. For at least 50 years i could have told you what schools were not doing well, based on norm referenced low stakes tests, and I could have told you which kids were not doing well. EVERYONE KNEW!!!!! Nobody had the will to do anything about it--they were not white middle class kids that were the ones who had the trouble, and so these chidren were abandoned. I see nothing new in NCLB except that the schools are now to fix all of societies ills. The real issue is do we have the desire to invest in the education of poor kids--after school programs, Saturday programs, summer programs, high quality pre schools, small class size for three years, moving high quality teachers to their schools etc. Identiying is half the battle to insure that poor and minority kids get a good education. We already knew how to do that. Its the other half --the will and money issues--that neeed addressing. And I've run studies of teachers identification of the kids who need help, and the tests ability to identify those kids. The correlations are really, really high. So in the elementary grades, where Ive done my research, teachers' professional judgment is every bit as good as the tests in identifying kids who need help. The tests add nothing if that is the goal.

---

Question from Jane Eberle, School Psychologist, Seattle Public Schools:
    Do you feel that the state assessments are discriminatory for particular students such as special education, ELL, racial minorities, economically disadvantaged?

David Berliner:
    as you know, but many politicians seems not to know, every test IN English is a test OF English. So of course we are often vastly underestimating the abilities of immigrant kids. In special ed we have tens of thousands of kids that dont get altertnative assessments or accomodations, but are special ed nevertheless. For those kids taking grade level tests, and for kids who havent yet mastered English (say in this country less then 3 years) the test are a embarrassing, they make the kids feel dumb, and that is hard ot overcome later. The problems for racial minorities and the economically disadvantaged is that the tests are not biased or discriminatory IF the kids had the chance to learn whats on the test. But if you've learned your math and science from people without a math and science major in college, or you've been taught by a lot of beginning teachers, or you've missed 20 school days due to no medical insurance, and asthma attacks, or your family has had to move you three times in four years to different schools and districts, then it is problematic if these kids have really had the same opportunity to learn. Unless we find ways to help our poor to stabilize thehir lives the tests will always be a barrier. Life circumstances are certainly discriminating against them, but that is not the tests fault. Its our slavish belief that the test is a fair assessment for all that is the problem. Its not.

---

Question from :
    Mary Mallory, Assistant Principal, King Middle School, DeKalb Public School System; What effect is high-stakes testing having on the morale and satisfaction of teachers teaching in so-called failing schools (schools that have failed to make academic ayp)?

Sharon Nichols:
    All of the anecdotal evidence we have from talking with hundreds of teachers is that it is having a negative impact. In low performing schools especially--which tend to be the poorer schools with fewer resources and therefore, with less infrastructure to help teachers be successfull--teachers are very stressed out. Depression seems to be common.

In our book we have lots of quotes from such teachers who feel not only lowered morale and a feeling of disatisfaction, but they feel demoralized and undermined in their ability to make decisions for themselves. In the literature, one variables that continuously pops up as important for teachers to feel satisfied is the perception that they have some control over their decision making. Additionally, more satisfied teachers are also those who feel they have support from their colleagues as well as from their administrators. When all of these factors are gone--as is common in a system of high-stakes testing--the result is a toxic environment where few teachers feel good about their profession.

However, I do want to stress that this is probably not the case in every "low" performing school. I suspect that in some cases, principals mediate a lot of the presure for their teachers. However, increasingly, principals will also be feeling the squeeze and therefore, will most likely be put in a position to be harder on their teachers.

so, the news is not good. If we continue on this path, it is likely that more and more teachers will not want to put up with such external pressures and control.

---

Question from Mark Covell, Doctoral Candidate, Youngstown State University:
    Do you think that we are moving in the right direction by adding a value-added measure in the analysis of achievement tests? I am referring to Dr. Sander’s work and the State of Tennessee’s use of the value-added analysis to measure and project student growth over a period of time. Student growth measures are then used to evaluate school effectiveness and ultimately teacher effectiveness. The State of Ohio is one state among many who are currently developing value-added accountability programs. Does the use of a longitudinal statistical measure of student growth bring more validity and reliability to achievement testing as opposed to the snapshot that we get from achievement test data only?

Sharon Nichols:
    I do think it is a step in the right direction--it at least acknowledges that a single snapshot does little to explain what a teacher and/or student is doing Still, we worry that similar problems will follow should stakes be attached to value-added performance. If we hold individuals accountable on the basis of improvement, we suspect similar types of corruption and distortion will follow and then, very little will have changed. Value-added is only useful if we do not also attached significant stakes to that type of performance.

---

Question from Brenda, Special Education Teacher:
    Is there any research on the results of high stakes testing and the impact they have for students of special education in terms of preparing them for college? In other words, is there any long term benefits to high takes testing?

Sharon Nichols:
    I am not aware of research that addresses this specific question. All of the information I have is that high-stakes tests more often than not becomesa barrier that is impossible for special education students to pass and therefore, it causes them to drop out or leave school before they finish. So for many special education students who otherwise are very successful in school, they often cannot pass the exit exam and are therefore denied a diploma. This causes many to drop out of school all together. so far, I have not come acoss any evidence of any long term benefits of high-stakes testing when it comes to individuals.

---

Question from Peter Lusa:
    In your research, how did you tackle the 50 individual state tests and did any of them come close to representing a nationally issued test like NAEP?

Sharon Nichols:
    This is sort of a difficult question to answer in this format. You can consult our EPAA article to see how we measured state level pressure among 25 states.

http://epaa.asu.edu/epaa/v14n1/

---

Question from April Gardner, Math Teacher, Lebanon Middle School,Graduate Student, University of Virginia:
    Are dropout rate increasing due to High Stakes Testing? So far I have found no research to support this, however, it seems as though this may be the case. Should there be research and some supporting evidence either way?

David Berliner:
     It seems pretty clear that in Massachusetts and in Texas there has been increased dropouts. In florida there are studies that go both ways. My own research suggests that the 9th grade to 12 th grade dropout rates are up in states with tough tests and high cut scores. Especially for minortities. This is also seen in the lower ages of those taking the GED after high-steakes tests have begun in a state. So id answer yes, and feel secure with that answer but there is soem disagreement with that in the literature. It turns out to be tough to answer becasue schools have always kept such lousy records on such things. D.

---

Question from Theresa A. Butori, Science/Math, BFASA:
    Does research back up the claim that NCLB testing changes schools for the better, does instruction improve ? Are there any recent case studies of schools and their education reform measures due to NCLB(success stories)?

Sharon Nichols:
    I think there are some "successes" that are identifiable. These successes turn up in location where special programs are instituted and where leaders have actively engaged in decision making that often times is independent from NCLB. (Check out Deborah Meier's work as one example)

But we have found no evidence that high-stakes testing under NCLB has had such a positive effect. Everything we have seen shows that instruction become restricted, narrowed, and undermined. Of course, the schools that are most likely to feel this type of impact are those that have a harder time passing. You won't see this kind of effect in the higher performing, mostly economically more advantaged school settings. These settings continue to do what they have always done--engage in creative, inquiry based type of learning.

Unfortunately, the pressures of testing under NCLB is putting a greater squeeze on low performing, mostly economically disadvantaged schools. So, while there may be a few cases of success, i don't think these successes can be directly tied to NCLB and are simply too rare to claim NCLB a success.

---

Question from Laurie Ogborn, parent and gifted education advocate, Sycamore Community Schools OH:
    My state will be adopting "Value Added" standards next year, through which all children are expected to show one year of progress for one year of instruction. Part of me rejoices - gifted children' progress (or lack thereof) may finally be addressed - while part of me thinks this just sounds like more testing. Do you have experience with Value Added, and can you comment on potential benefits that are being touted? Thanks very much.

David Berliner:
    I ahve no experience--and no one else does either--and it all sounds Ok. but if the stakes are high it will get screwed up like testing in the rest of the nation, as we document in our book. So the first issue is are the stakes to be high and if so id argue that VA models will fail too. Second, we ahve psychometric problems in determing growth and understanding what is proper growth. is everyomne held to a years growth? ELL, spec ed kids? kids who have moved, absent 20 or more days of school, are the schools where teacher turnover is huge and kids get lots of subs also expected to make a years growth?

There are lots of probelsm that VA models have to adress. Assessment always looks easy but turns out to be very hard. The more important and consequential the decisions to be made, th more the terst need to be psychometrically "pure." If we use test for information, nit to beat people up, they can be less psychometriclly pure and we can have them help us identify where help is neded for kds and schools.

---

Question from Cheryl Saliwanchik-Brown, Doctoral Candidate, University of Maine:
    Berliner and Nichols write, "We wonder, would the federal government treat members of the American Medical Association or the National Academy of Sciences with such disdain?" I agree, not only would these groups not accept sanctions and threats if they chose not to comply, they would also protest loudly and arduously. So why don't we hear more protest about high stakes testing from the National Education Association?

David Berliner:
     I dont know. they are too cautious, I think. They are probably concerned, as are many, that this bill might have some hope as a civil rights bill. It does shine a light on the inequities in our society--and that's good. But weve known about these inequities for decades and no one has done much about them, so i am not sure what is new here. I think NEA is wrong on this. They should be helping to get the civil rifghts part of the bill preserved and do away with the nonsense. We need more gutsy state chiefs like Nebraska's who sems to have that state on the right track and the teachers are all with him. We need to drop the nonsensical AYP until ALL are proficient--an unreachable goal if proficent is to mean anything. We need to make the assesmsnt useful information, not a cudjul, to whack teachers and schools with and we need mechanisms to monitor that the information is used to imporove schools. Not left to rot, as has been true. We need to dump NCLB and repllace it with one of a number of sytems we recommend in our book that can accomplish the accountability goals, without damaging the system as a whole. D.

---

Question from Sheila Radford-Hill, Executive Director, Luther College:
    Using student assessment to improve teaching and using assessment for accountability purposes are very different processes. How might an assessment system do the former while informing the latter?

Sharon Nichols:
    This is an important distinction that is completely lost to the NCLB constructors.

we could institute a system of formative assessments--those would be the form of tests that teachers use to help them make decisions. These tests would not be reported on to the public necessarily since they are only used to help teachers make decisions about curricula, pacing, and student achievement.

In terms of accountability, I am not tests of any sort are useful, because once you use a test, the test itself becmes a high-stakes test and then all the problems we identify in our book will follow. I think we need to re think accountablity all together. In our book we talk about using a system of the inspectorate. An independently trained group that assesses the local conditions of schools and reports to the community both the strenghs and weaknesses identified within the school culture as well as the community resources that are present or absent. This sort of accountability would be more multi dimensional and rounded.

---

Question from Linda Johnson, Music Teacher, Hambrick Elementary, DeKalb County, GA:
    Do you find that students become lax in their schoolwork as well as deportment once the high stakes testing is complete?

Sharon Nichols:
    absolutely. The exagerated emphasis on the test has made everyone in the school organize around the test. When it is over, what is left to do? I know teachers in a local middle school who have a month left now that the TAKS test is complete (the Texas test), and for a week straight, Teachers had to spend all their energy convincing students that it is still important to learn.

High-stakes testing has created a culture where everyone focuses on teh test, but when it is over, it is hard to know what to do. When you tell a child over and over again that you must learn something because it will be tested, you naturally diminish whatever intrinsic interest they may have. and we see this over and over again with kids....once the test is over, what more is there to do?

the opposite is also problematic. We know of students who come to class at the start of the year and basically say, "why do I need to be here. I know I will pass the test. It is easy for me." For them, learning is also irrelevant. I think the concern over student motivation is under emphasized. While the theory of testing is to spur motivation, it has the opposite effect for many of our students.

---

Question from Dr. Yvonne Siu-Runyan, Professor Emeritus; The University of Northern Colorado:
    What can those of us who are concerned about the horrid NCLB Act, and all the nonsense, such as high stakes testing, narrowing the curriculum, holding teachers and schools hostage, and the hiring of lame superintendents of schools do besides write letters to the editor, speak to Boards of Education, and talk with the general public about its folly and encourage them to follow the money. The NCLB Act needs to be repealed. So much nonsense has ensued since this law was passed. Like Drs. Nichols and Berliner I am concerned about this country and the direction in which it is heading - not a healthy place for the citizens and our future. We need to help parents understand the double talk of the NCLB Act, and empower teachers to speak out and then stand behind them.

David Berliner:
    You have answered ytour own question--Im with you! we need to speak out and threaten legislators with not voteing for them if they dont listen to us. We have to organize and make sure that we are a voting block at the district level for legislators to get the message that we will not support them if they support this bill. TYhank goodness we stilll live in a demoscracy and the 3 milliion teachers and hundreds of thousands of adninsitrators and the many millions of aprents that undertasnd what si going on need to be mobilized to speak and sign petitions and bhe as obnoxious as possibel, Prohabition was passed and removed! Woman did get suffrage! The viet nam war was ended earlier than it might have been after the waste of 55000 lives. Iraq will conclude sooner rather than later becuase the voters have turned. its all in a voting block. Keep writing those letetrs, keep informing folks of all that is wrong. i do it too. d

---

Question from Mary-Lynne Monroe, teacher, Portland Public Schools:
    Students in our district are tested yearly or every other year with the same testing format. Statistically, is repetitive testing like that truly valid? Isn't there some point at which it's more a measure of test-taking ability?

Sharon Nichols:
    Bingo! Absolutely. This sort of problem speaks to a type of validity we describe more in the book--construct-related validity. The extent to which the test is measuring some underlying skill, trait or ability it proports to measure. At some point, we have to worry that the test score is at least partly telling us about a student's ability to take tests. Excessive test preparation leads to this worry about the validity of the test as well.

In short, high-stakes testing for many reasons, including the one you cite above, leads to a sense of uncertainty, or questionable validity in what you're measuring in the first place. This is why the American Educational Research Association, the largest ed research association in the nation says that in light of these known validity problems associated with high-stakes tests, it is completely inappropriate, then, to USE test scores as decision making tools. If we don't know what we're measuring, how we can make a decision on the basis of that measure?

Only under special conditions would the validity of tests withhold under the strain of repetitive testing--including that test makers are spending the time and resources needed to properly construct their tests to ensure appropriate psychometric qualities. But we know this isn't happening. Stories abound about the errors of scoring, construction, and distribution from test companies. With over 50 million tests needed over the next few years, and only four major test companies in charge of constructing them, the likelihood of errors and validity problems are enormous.

---

Question from Judy Hundley, Principal at Bonham Middle School in Temple, TX.:
    Why is it when all of our kids are being taught for the first time in the 25+ years I have been in education, are there people who have to object because of the hard work? Tell me if I am wrong, please. Do you think that maybe the objection comes from the people who really don't want to work that hard (or teach all of our kids)? Finally, where were these same people when we were only teaching a few and ignoring the rest?

David Berliner:
    well its not me. I object to high-stakes test as the vehicle to get teachers and admisntrators to work harder. I object to stupid, low level tests as the goals of schooling sicne I have a broader education in mind. i object to narrowing the curriculum, forcing teachers and admisntrators to cheat, embarrassing special ed and ELL with tests they cannot pass. What i dont object to is hard work to master rigorous currculum objectives--i demand it! I think we need a rigorous curriculum and kids need to work hard to master it--but it needs to be in art and mucis and dance, it needs to be in history and social studies, in physical edcuationa dn health, in projects and simulatiosn and games on the web. If you think the TAKS has promoted that you are the only one Ive ever met to say so. Is hard work the only thing we want from the kdis or do we want hard work on a braod set of curricuum outcoems and in curriclum areas for which there are no single right answers. Its not hard work thats

---

Question from Liz Crane NBCT, Kindergarten, Ridgeview Elementary:
    Could you speak about standardized testing for primary aged students? In my district in NE Florida, we give the SESAT- to our kindergarteners. It is four days of testing.

Sharon Nichols:
    Liz, this makes me so sad. We are starting so early training our youngest students. I think this is problematic on so many grounds. I am not even sure where to start. to think we're pressuring our youngest students to perform on a test. To think we're starting at such a young age telling students that tests are why we are in school, to use tests that have validity issues to identify and track our students at such a young age is quite worrisome.

I think you need to fight this test. I cannot think of a reason to institute a standardized statewide test at such a young age. Teachers and schools could use the resources spent on these tests to conduct local assessments and evaluations of students' potentials and to evaluate needs. We already do this to some extent. I just don't see the point. write your legislators, tell them that losing four days on a test with kindergarteners who are supposed to be creating, exploring, and asking questions is too long. That Teachers know more about their students than what this test could possibly reveal. This is just awful.

---

Question from John Thacker, Education & Outreach Program Leader, LIGO Science Education Center:
    Can we drop high stakes testing and replace it with credits earned for hands-on, laboratory sessions? (or some similar authentic problems solving activities)

Sharon Nichols:
    I think this is a most viable idea. we know high-stakes tests don't work in intended ways. But we also know that it is causing teachers to have to drill and kill their students, thus diminishing the role and value of inquiry and creative thinking. And, this is a much more prevalent instructional method in low performing schools where the pressure to perform is great.

We have seen and heard time and time again that in those contexts, teachers under the strain of having to get their kids to pass the test, end up doing drill and kill rote activities over and over just to get their students through. And, they have to---else they will be judged as failures and their students will pay a consequence as well.

the practice of high-stakes testing causes everyone to worry excessively about what will be tested. The end result is that what is not tested gets ignored. And since we cannot measure well things like creativity, inquiry, question-asking, and problem solving, these things naturally get de emphasized.

however, I would go further than what you propose. I think we should expect problem-based learning to be part of the curriculum. Which means, if we keep testing, we must include different types of tests that honor these skills such as portfolios and performance assessments. Unfortunately, these are more costly and time consuming.

In the end, all students should be exposed to the best instruction and curricula possible.

---

Question from Jeffrey Swiatowicz, K-5 Math Coordinator, Ridgefield Public Schools:
    One of the results of high-stakes testing is that an enormous amount of data is now available to us. How do you feel about using the data which comes from this testing to shape decisions about the "How" and "What" we teach our children?

Sharon Nichols:
    Well, Under high-stakes testing, the biggest problem with this way we use all the data, is that we know the data cannot be trusted. So even though we have a lot of information, we can't be sure what the information is telling us.

In our work, we show how the pressure of testing leads to behaviors and activities that end up manipulating or affecting the data. For example, we see that the curriculum is narrowing and that teachers spend more time on test prep activities. The resultant test score then must be called into question---are we measuring performance under "ordinary conditions of teaching" or, are we measuring how well students take tests? So, all the data is really pointless if it isn't good data.

But I think once we remove the stakes from the tests, it may lead to greater validity and tehrefore greater confidence in what the test score represents. But ONLY if we have assurance that the test has proper validity and reliability can we even begin to think of using the test scores to make any sorts of decisions. We think the testing must be put back into the hands of teachers who can create formative assessments that are more informational--in the way tests are supposed to be. That is, teachers can take information from these tests to determine the efficacy of their curricula, instruction, and to diagnose students' knowledge strenghts and weaknesses.

An overabundance of data is not necessarily helpful in any way.

---

Question from Cheryl J., Radar Engineer, Math Teacher, Parent:
    I am good at taking tests. My math and analytical GRE scores were in the 700s (out of 800). My reading and writing scores on Praxis 1 were in the 180s (out of 190). My Praxis 1 math and Praxis Middle School Math scores were 190/190 and 200/200, respecitvely, and I achieved "recognition of excellence" on the Praxis Mathematics: Content knowledge test. I was in the public schools long before NCLB, but the public schools had nothing to do with my ability to take tests - in fact my SAT scores were profoundly average. The change came about in college, specifically when a classmate showed me that learning happens not by answering questions, but by ASKING them. This realization turned my own education around, which in turn improved my test scores. There were times when I asked questions while studying that showed up on the test! In teaching, I have found that the few children who are driven to ask questions, rather than dutifully answer them, are the best learners. Even Einstein's famous "thought experiment" that changed the way we understand the world was a question - and a simple one at that.

It is clear that NCLB takes the focus on answering questions to extreme hieghts. Do you believe that changing the nature of the tests is enough to acheive our society's educational goals? If not, what else can be done?

David Berliner:
    you discovered soemthing profoud--its been remarked upon by Jerome Bruner and others that asking questions not answeribng them is the skill we need to live rich personal lives and for society to prosper. In fact, a technique to help so called low ability readers was developed around your insight--its called reciprical teaching and it works well by getting kids to ask the questions, not just answer them. So you have a profound insight and as you can see our society is heading the exact wrong way. We are heading for a socity that always looks back and in convergent ways. the what if question cannot be scored and so it is dropped. This is really a mess! What can be done? ORGANIZE! go fight with a legislator. Vote only for those that understand your point. Buy our book and give it to the chief state school officer and demand he/she reads it. If we dont get political we will see our nation hurt. Thanks for speaking up. DCB

---

Question from Dr. Sharon Yates, Consultant, Nashville, Tennessee:
    Many organizations well-known to education such as Achieve, Ed Trust, and Fordham are encouraging high standards for college readiness. Does this emphasis conflict with your notion that high stakes testing is not appropriate for schools? How do you respond to the media that constantly reminds states that they are falling short of meeting NAEP standards?

David Berliner:
    lets not confuse high and rigorous standards with testing for acheiveemnt of those standards. I am for standards. Done right they guide schools and states and the nation. But the high-stakes testing is the culpret. it distorts learning of the standards and makes training to answer test questions the goal of education. Further 4 sepeate reports acknowledge that the NAEP standrads were set way too high. No nation has huge numbers of kids at Proficient if they used our system. And fe nations have schools as segregated as ours. we provid poor and some minority kids a losy education and wonder why they dont do well on assessments? thereal issue is whtehr advantaged kids do well. Do our schools work for the better off mememebrs of our society and thew answer is an overwhelming yes! Thats the kind of edcuation we need for all.

---

Question from Pat Buoncristiani, Consultant working in both Australia and the USA:
    How do you explain the poor performance of the USA in the PISA survey when compared with countries such as Finland, Australia, New Zealand - countries that do not have high stakes testing?

Sharon Nichols:
    Ok--this is David. I think you know the answer--obviously there are other routes to great student perforamnce other than tests. Finlnd has a much more equitabloe dsitribution of income and pays its teachers well and invests in their professional development--so teaching is a high-prestige job. That helps a lot!. In addition research suggets that we have too many teachers without a profound, deep understanding of mathematics. We dont pay to attract those and so we dont get many. In addition math is seen as hard. there is no reaosn for that other than a cultural norm and a teacher bias against mathematics. in addition we have a huge percent of high schoolers working over 20 hours a week and they dont. that makes a differecne. the real issue is wether we develop enough mathematics talent for a smart citizenry and the ecnomy. The answer appears to be yes to the latter and we are unsure about the former. We do not have many jobs that require solving quadratcis and knowing calculus or trig. If we have 10% such jobs it would eb surprising. So we are producing what we need for the economy and we simply havent geared ourselves as a culture to produce much more.

---

Question from Kimberly Dechant, MSA Graduate Student/NC Principal Fellow, Western Carolina University:
    Does high stakes testing penalize creative students?

David Berliner:
    Probably some, surely not all. Depends on the kinds of creativty they posses and the kinds of test they take. Constructed response ite=ms where ther isnt a single right answer but are scored based on a "good response, whtever it may be like" are not a penalty for creative kids. Multiple choice items migh penalize some.

We have to rememeber that IQ tests--highly reliable and so much a part of our culture--predict very little about life success. Other kinds of skills--sticktoitiveness for one--predicts just as well. About 90 percent of the variation in most of the outcoems we can measure in life is not predictable by out tests in school--IQ and anything else. The tests predict well how youll do soon after in things like the test measure. Thats it. Creative kids may or may not be assessed well and we ahve to watch for thatjust as we have to watch how ELL kids are asssessed and Special ed kids are assessed. No test can work well for all groups and all tests work best for the group that most resembles its norming group. So if youa re asking should we be cautiuous in interpreting the test scores of gifted kids the answer is yes. And it might be yes for all kids --caution is always needed and reliaance on a single measure of achievement for any kid is silly as well as a viol;ation of professional testing standards.

---

Question from Barbara Nadler, Consultant, NJ:
    In place of the tests you so disdain what do you propose to insert the accountability for student performance that is so lacking in public education?

Sharon Nichols:
    as we state in the book, we do not "disdain" the tests. In fact, we believe good tests are extremely valuable and play an important role in education. It is when we started using test scores to judge our teachers and our schools--when we attached such high-stakes to them--that is extremely problematic. So, tests in and of themselves are fine. Under the current system, so much emphasis is placed on a single test, it is undermining sound education. Also, I am not so sure the generalization that student performance is "lacking" makes sense. So many students are performing so well. The media blitz about average performing students is misleading because it is an average. Some students are succeeding very well under incredible circumstances. Still, there are many students who do under perform. These students are not afforded equitable opportunities to move forward in life. Students who come from poverty are more likely to attend schools with few resources. They are more likely to be taught by a teacher who is not an expert in the field in which they teach. They are more likely to suffer from health problems which impede ability to learn…and on and on. So I think we need to reframe what the problem is that we need accountability for. The current system of high-stakes testing assumes the “problem” is laziness. Students and teachers just need a reason to work harder. This “theory of action” is extremely problematic. In our book we propose such a solution in the form of an inspectorate—an independently trained group of individuals who visit schools and assess teachers and principals. We think this is a viable alternative to what is happening now. In short, we need to move away from using ONE test as the only indicator of how everyone is doing in a school where learning—an incredibly complex endeavor—takes place.

---

Question from Kathleen Mooney, Parent Engagement Specialist, Families In Schools:
    if high stakes testing is so bad for our schools, what evaluation tools do you believe should be used to evaluate the results of the work of our schools? We need evaluation tools that can be used across schools so that we can articulate our need for more resources to 'get the job done' and so that communities have a way to hold their schools accountable. Don't we need high stakes testing as one tool in a package of evaluation tools? Should we not be spending out energy to improve the eval system not tear it down? Solutions need to come with critiques. (I have not read the authors boook yet, I hope they address solutions.)

Sharon Nichols:
    Great question and we do address it in our book.

We agree that we need some sort of accountability mechanism. In fact, most teachers also agree. They want to be held accountable for what they do. But they want waht they do to be better understood at the same time. and they want to evaluated fairly. the use of a single test score is not fair, nor is it even informative.

There are many ways we can communicate to the community how schools are doing that do not rely on one test as the sole measure of school performance. Doctors are not held responsible for whether their patients take their medicine. Lawyers are not held responsible for clients who keep breaking the law. Why do we hold teahers responsible for students' performance, when much of students' performance is due to factors beyond a teacher's immeidate control?

In our book, one idea we propose is the idea of an inspectorate--something done in other countries. Here, the idea is to send an independently trained group of individuals into a school to evaluate and assess what is going on. After all, what better way of seeing whether a teacher is effective than by observing him/her teach?

An inspectorate could provide a report of sorts to the community, that describes to its members all the things the school is doing right! Imagine that...a report that includes some positive evaluations of how hard the school is working to over come certain challenges. Of course the report would also include the challenges faced by the school. does the school have computers? what sort of resources are available in the library? what sort of after school programs are there? how active is the community in providing volunteers? In short, the inspectorate could help to contextualize what is going on in the school.

I could not agree more, we should be spending our "energy to improve the eval system not tear it down"--except the current system does not work. it yields unreliable test scores on which so many important decisions are made. this is not right. Further, it is seriously damaging the lives of many of our youth and educators. We are obligated to tear down a system that is so harmful.

Instead, we need a system that makes sense. and they are out there, we just need the will and the imagination to invest in our youth and our educators.

---

Question from David Penberg, Coach, Eductors for Socila Responsibility:
    What about the social and emotional needs of students ? How they have been shortchanged and dismissed as a result of the obsessiveness with tests (prepaparation and taking)How many Columbine's and West Vrginia Techs, do we require to remind us that a sense of belonging and care and relationships are foundational to the well being of kids.

Sharon Nichols:
    David, the area of student belonging happens to be my other research area and therefore, I worry a great deal about the social and emotional needs of our students. I vehemently believe high-stakes testing is undermining teachers' capacities to care for their students. to care for their social, emotional needs. When tests trump everything else, and when the stakes are high, good teachers, caring teachers are put in a position to have to put aside their capacity to care. (Nel Noddings from Stanford has written about this very beautifully).

In our book we identified some extreme examples of this...one situation where a young student was in a car accident where he lost his brother and sister and had injuries that caused him to have to re-learn how to write....still, because the school needed 95% participation of students on the test, administrators forced him to participate. he may or may not have been ready, but the point is, the law's stringent requirements force educators to abandon compassion because the test is all that matters.

Encouragingly, many educators are standing up and refusing to do this---as in the case of one principal who refused to test her special education students because of the guaranteed damage to their self esteem because they would not be able to pass. So, many of our moral leaders are doing the right thing. But the climate of testing makes it exceedingly difficult. Teachers are forced to be complicit with acts of cruelty they otherwise would not engage in....this frustrates them and us.

But the end result is that our students lose. When teachers are forced to rush through curricula, to ensure all students know the same information in the same way, it makes it exceedingly more difficult to stop and attend to social or emotional needs that may come up. It is terribly problematic.

But as a culture at large, we need to do a better job socializing our youth. in an earlier book by myself and Tom Good (America's teenagers--myths and realities, 2004, Erlbaum) we argue that our youth are asked to do more and more, to hurry up and grow up, to be more active, but with less adult guidance and supervision. We need to come together as the adult community and help to take care of all of our kids. one theme in Columbine and Virginia Tech is that these kids were alienated, isolated.

I worry about this a great deal. thanks for your question.

---

Kevin Bushweller (Moderator):
    Thank you for joining us for a very interesting discussion about high-stakes testing. And a special thanks to our guests for taking the time to answer such a large number of questions. This chat is now over. A transcript of the discussion will be posted shortly on edweek.org

---