Performance-Based Assessment
July 1, 2008
Performance-Based Assessment
- Guests:
- Lewis Cohen is executive director of the Coalition of Essential Schools, the nation’s oldest and most influential non-profit school reform organization and a leader of the small schools movement.
- Raymond Pecheone is the co-executive director of the Stanford School Redesign Network and director of the Performance Assessment for California Teachers program. Formerly, Pecheone was the Connecticut bureau chief for curriculum and teacher assessment.
- Robert Littlefield has been principal of Portsmouth High School for almost ten years, and was recently named the Rhode Island High School Principal of the Year. During his tenure, Mr. Littlefield has worked to transform the culture, mission and focus of PHS into a high-performing, personalized, and supportive learning environment.
Scott J. Cech (Moderator):
Good afternoon, and welcome to Education Week’s Live Chat. Joining us live are Coalition of Essential Schools Executive Director Lewis Cohen, Stanford School Redesign Network Executive Co-Director Raymond Pecheone, and Robert Littlefield, the principal of Portsmouth High School in Portsmouth, R.I. In this Live Chat, they will discuss and answer your questions about performance-based assessments--exams that require students to actively participate, such as in dissertation-defense-style presentations, portfolios of selected student work, etc. Rhode Island this year became, by all accounts, the first state to make passing a performance-based assessment a graduation requirement for students, and several states are eyeing the Ocean State’s system, which also requires traditional standardized tests, as a potential model. I’m Scott Cech, an associate editor at Education Week, and I’ll be moderating this discussion with our three guests, each of whom has a unique background and perspective on Rhode Island’s performance-based assessment program, which is the subject of a story in the current issue of Education Week. We’re already getting a tremendous number of questions for this chat, so let’s get right to them.
Question from Pat Schettini, Superintendent, Reading Public Schools, Reading, MA:
What advice do you have for a school district who does traditional assessment well, but wants to (for the most part) move to a performance based model of assessment? What would be the first steps? What conditions are needed to make this move? How can obstacles for change be overcome?
Lewis Cohen:
First, the district needs to set the stage for this work by preparing its teachers, administrators and central office Curriculum and Instruction staff with professional development focused on assessment literacy. A good source for this work is Grant Wiggins’ “Understanding by Design” approach. Secondly, the district must build time into the regular schedule for teachers to collaborate vertically and horizontally in order to identify and scaffold the knowledge and skills that meet the district profile of a successful graduate so that your performance assessments are measuring the outcomes you most value. A key element of performance assessment is its public nature. In addition to the internal work it is vital to include stakeholders, especially school board members in an exploration of the power and utility of performance assessment. One effective way to do this is to incorporate these key stakeholders into your performance assessment process as trained evaluators so that they understand the purpose of the process and see first hand what students in their district know and can do. A first step in this regard might be to take key opinion leaders to visit schools that use performance assessment or for the district to join networks such as CES, for which this is a standard practice. Providing adequate support to build teacher competence along with developing stakeholder advocates, are in my opinion the most critical strategies for overcoming obstacles to change.
Question from John Thacker, physics teacher:
This is lengthy - it’s about differentiated instruction and assessment. If you attempt a differentiated instructional approach to reaching all the students in your class, does it follow that your performance based assessment would be differentiated as well... what would that look like in practice? If a lesser-capable student performs well on her differentiated assessment, does she get an “A” just as the most-capable student gets when they do well on their (more difficult) performance based assessment?
Lewis Cohen:
Performance based assessment is a standards based approach. All students are expected demonstrate mastery of the standards based on the same rubric. The beauty of the performance assessment is that it lends itself to differentiated instruction so that students who have used different material and received different levels of support still have the opportunity to demo0nstrate their mastery and meet the standards.
Question from Judy Lawson, Teacher, Hart County School System:
How many years had the students participated in performance-based assessments before graduating?
Robert Littlefield:
We have performance-based common tasks built in to all their courses beginning in the ninth grade. We have a common task committee who coordinates the effort to make sure the students have the opportunity to learn the skills prior to Senior Project.
Question from Barbara Parr, former Assessment Specialist, ETS:
How are you ensuring calibration among the scorers of performance-based assessments? Bias, especially with diverse cultures that are represented among our changing population, is very difficult to separate in scoring these instruments. What measures are you taking to ensure validity and reliability?
Robert Littlefield:
We have several faculty meetings to view and evaluate presentations. After, there is discussion about the results. We offer our “outside judges” a two-hour orientation to accomplish the same task. Also, when scoring presentations, we throw out the lowest and highest scores.
Question from Nancy Hammons, Library Media Specialist, Antioch High School in Tennessee:
When I think of performance assessments, I usually think of portfolios--I am assuming that all graduates had representative samples of work that showed mastery of certain skills or standards. What other criteria were required for graduation? Did students also have to pass certain standardized tests, or were these performance assessments used in lieu of more convential assessments?
Robert Littlefield:
In order to pass any course and receive credit, each student must: a. maintain a numerical average higher than a 65, and: b. achieve the standard of proficiency on at least one of the following: 1. the end-of-course exam (which includes an applied component comprising at least 50% of the test) 2. the common task built into the course (research paper, speech, presentation, essay, etc)
Question from Michael Rulon, Instructional Coach, Albuquerque NM:
I have worked in RI before, and am now in a state that is high stakes. Currently I am at a Charter School and we are adopting much of the direction of the Big Picture schools founded in RI. We are currently going to require our students to collect a body of evidence through exhibition and portfolio. Any advice in about roadblocks we are likely to encounter in this enviornment?
Raymond Pecheone:
Regarding demonstrations: make sure your students have ample opportunities to develop research and presentation skills in the curriculum leading up to the final presentations. Regarding portfolios: you must have the complete cooperation of your technology staff. They play a very big role in facilitating the compilation of electronic portfolios.
Question from Paul Graseck, High School Planner, Paul Cuffee School:
Will the complexity of the Rhode Island performance-based assessment system undermine its effectiveness or success?
Raymond Pecheone:
I don’t think so. The more we commmunicate our expectations and the more our teachers adopt common practices and nominclature, the simpler the system seems to everyone.
Question from Janice Robertson, Teacher Librarian, Peel District School Board:
If you were trying to sell the idea of performance based assessment to a school board, what are the top three reasons for using it?
Lewis Cohen:
• Assess What Matters We know that what gets measured is what gets taught, and that testing drives the curriculum. Students today need more than basic skills but close ended multiple choice questions measure student ability to recall discrete facts rather than their ability to engage in complex thinking and problem solving. Thus most of today’s state assessments are widening the gap between the skills students are acquiring in school and those they need for the globalized and technology infused society they must live in. By contrast performance assessments allow students to demonstrate 21st Century skills including Critical Thinking and Problem Solving, Communication, Creativity and Innovation, Collaboration, Information and Media Literacy, and Contextual Learning. • More Powerful Teaching and Learning Performance assessments require students to demonstrate what they know and can do. They engage the student in real world multi-disciplinary tasks and provide a richer, more sophisticated and more honest view of student learning. Deep learning is demonstrated when students can apply facts to solve novel problems. Further, performance assessments engage teachers in the development of tasks that reflect a range of content and skills and in developing scoring systems that provide invaluable feedback for improving instruction. Teachers report changes in both curriculum and instruction as a consequence of collaborating with colleagues in evaluating student performances. • Proven and Practical Performance assessments challenge students to engage in more intellectually rigorous work and provide teachers with rich diagnostic feedback about student learning and timely information for improving instruction. Many states including New Hampshire, Rhode Island, Nebraska and Oregon are making extensive use of performance assessments such as analytic essays, research papers, science experiments and advanced application of mathematical concepts. Most recently North Carolina added a Senior Project to its graduation requirements.
Question from Oscar Sisk, science teacher, Discovery High School, South Kitsap Schools:
Washington State class of 2008 students are in a similar situation. This year, many students were told they would not graduate because they did not meet the minimum on the WASL. The really hard thing to watch, as a teacher, is that some of those students were not told they would not graduate 8 days before the ceremony. Our programs had to be re-written at the last minute. Visiting grandparents went back home angry that it would cost them another set of cross-country tickets, and students were asked to “hang on to” their caps and gowns. My question: Could this sort of thing happen in RI?
Robert Littlefield:
Unfortunately, yes, I does happen in Rhode Island. But the same thing would happen if a student flunked his English final exam and failed to reach the required number of credits. The result would be the same. We are trying to balance two things: 1. Give ourselves plenty of time to have kids demonstrate proficiency, even if it takes several tries while leaving enough time to plan for graduation. 2. Not finish too early so that we are perpetuating the kind of stale enui that has traditionally been the end of senior year.
Question from Robert Baroz:
Given your experience of implementing performance-based assessments in RI, if you were ever in a position to develop and implement such a practice in another state, what would you do the same and what would you do differently than what you had done in developing and implementing a performance-based assessment system in RI? Why?
Robert Littlefield:
Things I would do the same: 1. Embed the Senior Project in the English class. It gives students regular contact and holds them accoutable. 2. Have an advisory program in place to increase communcation of standards and expectations. 3. Have a public relations campaign so that everyone in the community knows what we are doing and that we are in need of judges and mentors. Things I would do differently: 1. Begin the first steps of Senior Project in the junior year (topic choice, letter of intent) 2. Crosswalk common tasks from grades 9-11 so there is no doubt that students have had the opportunity to develop all the skills necessary for success in senior project (i.e. setting up interviews with strangers, building a timeline/action plan, using primary sources in all research papers.
Question from David Seiter, Assessment Team Northridge High School Layton, UT:
What was the largest graduating class to go through the performance assessment? Our graduating classes average around 600.
Robert Littlefield:
Our graduating class was 250, with 100 faculty. With about 40 outside judges volunteering over the course of two days, we were able to accomodate all presentations in a total of six hours.
Question from Joni Richardson, Teacher , Caumet City:
I definitely agree with performance-based assessment than ISAT testing for students. Would this test replace standard testing state wide if adopted by other states? if so, Who wouold score the test? Can this test be standaridized?
Lewis Cohen:
So far states have opted to supplement standardized tests with performance assessments. Cost is the primary barrier to basing statewide accountability systems on performance assessment and it is largely a psychological barrier. The focus of performance assessment on more important and relevant skills particularly those associated with college readiness, as well as gains in student achievement and teacher professional development make the whole educational enterprise more cost effective through the use of performance assessment even if the cost of the test per se is higher. Performance assessments are by their nature largely designed and scored locally but what can and should be standardized is the appropriate criteria for scoring a variety of products.
Question from Rick Lander, Principal, Shawnee Mission Horizons HS:
How are things handled when a student transfers in from out of state or a private school not requiring the performance-based assessments?
Robert Littlefield:
We get many students of military families (mostly Navy) who transfer at the start of senior year. We have them meet with guidance and senior project coordinators to identify any gaps in skills that might exist and, if necessary, modify the Project. We are clear: all graduates must complete the Project. However, we reserve the right to alter the standard for transfer students.
Question from Scott Shuler, Education Consultant, CT State Dept of Education:
Rhode Island seems to have taken balanced curriculum into account to a much greater extent than in many state testing systems. How has the inclusion of oft-neglected core subjects such as the arts in the assessment system affected curriculum decisions and curriculum balance in Rhode Island?
Robert Littlefield:
As part of our diploma system that was subject to approval by the Department of Education, we had to include a mechanism whereby students demonstrate proficiency in fine arts and technology. We accomplish this by requiring at least one-half credit in each. All technology courses and fine arts courses have built in performance-based demonstrations of proficiency.
Question from Peter Smyth, Principal, Charleston Charter School for Math and Science:
How do we get the testing companies, as an influential lobby, to let go of the standardized testing lock on the assssment?
Raymond Pecheone:
I don’t have much of an answer for this. I can only say what I say to everyone I speak to about performance-based assessments: once you see it up close, you will become immediately aware of the value. That is why we invited RI Governor Don Carcieri to our school, he is a proponent of testing and we wanted to raise his awareness of “the other way.” He took us up on the invitation and sat with me in a presentation room and helped evaluate five student presentations.
Question from Paul Graseck, High School Planner, Paul Cuffee School:
Will the cumbersome nature of Rhode Island’s performance-based assessment system, given the enormous number of standards to teach, lead to teacher burnout?
Lewis Cohen:
There is no question that teachers already face the threat of burnout from the many demands and mandates on their time. It is also true that the use of performance assessment places a new set of demands especially at the outset. However, effective assessment is integral to effective teaching. Performance assessments provide more meaningful and immediate information about what students are able to do with the material covered in class. This in turn can lead to effective teaching that insures better prepared students, greater student achievement and eliminates the need for the many hours teachers currently devote to test prep for standardized tests. Rhode Island educators that we have spoken with have said that rather than causing burnout the use of performance assessment has been energizing. They found it emotionally and professionally satisfying to see their students show an audience their ability do real things, well.
Question from Raymond Pecheone, Co-executive Director, Standford School Redesign Network, Director of Performance Assessment for CA Teachers:
I understand that performance-based assessment can be quite valuable for some English curriculum, but how does an English instructor incorporate the basics of language skills instruction into performance without sacrificing more essential practice with basic reading, writing, grammar? Students seem so deficient these days.
Robert Littlefield:
We are not completely devoid of standardized testing in RI; our students still participate in the New England Collaborative Assessment Program (NECAP). We are held accountable as a school to develop reading, writing and grammar skills and we do quite well. Our English teachers spend a lot of time collaborating on how to accomplish both tasks.
Question from Jeremy Spencer , EdS Science Teacher, Camden County Schools, Kingsland, GA:
When do you all see schools going true performance-based assessment? ...I feel as a classroom teacher that all I teach is how to take a test, not science...this is not good for 21st century America.
Lewis Cohen:
I am encouraged that we are moving in this direction. Policymakers are increasingly acknowledging the disconnect between what is being required to be successful on standardized tests and the skills our children will need to face an increasingly complex future. Last month Democratic Presidential nominee Barack Obama participated as an evaluator in a performance assessment at a new small high school in Mapleton Colorado. Afterwards he spoke about the need for more appropriate assessments. Congressman George Miller, an author of NCLB has also been openly grappling with this question. Now it is up to advocates to demonstrate the feasibility of statewide efforts like Rhode Island’s in order to convince policymakers that this work can be done at scale.
Question from Michelle unemployed Jamaican teacher:
Having done research on performance-based assessment I admire the intiative of the Rhode Island school. What three benefits could teachers who experience behavioural problems receive should they decide to implement a performance -based assesment system given it would incoporate evaluation at the end of their daily lessons.? Would you say teachers who explore using the Perfomance -based method build their technology skills and be better prepared to assist their pupils in developing skills needed in the 21st Century work place ?
Lewis Cohen:
Performance assessment can offer a means for keeping students engaged and on task common and proven approaches to dealing with discipline problems. In Rhode Island and other places using performance assessment we have seen an increased reliance on technology particularly with regard to multi-media approaches to presentation. But the real importance of performance assessment is it offers ways to assess 21st Century skills such as critical thinking and problem solving, communication, creativity and innovation, collaboration, information and media Literacy, and contextual learning that are not effectively measured using standardized multiple choice tests.
Question from Joyce Anderle, Assistant Principal, Coventry High School:
How much additional support/time, besides regular Capstone classes and time during advisories,should the school provide for students, paticularly when they are not motivated to complete these requirements? We provided after school classes, Saturday sessions, special days for Capstone presentations, etc. and many students still did not take it seriously. The onus for completion is on the administration and the faculty, not the student. Your thoughts?
Robert Littlefield:
I completely understand your viewpoint. It seems everyone has been talking about increasing rigor but no one told the kids! In one of my final meetings with the Class of ’08 prior to graduation I told them, “See, we weren’t kidding, were we?” I think the culture will change to meet our expectations. We just have to continue to communicate our expectations clearly. And yes, we do the summer ramp up program, the remedial after school classes and the hand-holding as well.
Question from Alex Harris, senior policy analyst, National Governors Association:
I have spent time with students, teachers, principals, and policymakers in Rhode Island discussing their assessment system. Even in this small state, they face considerable implementation challenges. Can this model work at scale in a larger state environment and what supports would need to be in place for this to occur?
Raymond Pecheone:
For many people in the East, their only experience with Rhode Island is driving through on Interstate 95 on the way to Massachusetts, or to vacation in northern New England. They are usually delayed by traffic jam only to find out it was caused by one car on the side of the highway with a flat tire. Why do Rhode Islanders slow down at every flat tire change? Because we all know there is a good chance we know the guy! I am the first to admit that Rhode Island is at an advantage because of our small geographic size. Over the last five years we have had several meetings a year attended by every high school principal, sometimes accompanied by the superintendent and the union president, and directed by the Commissioner of Education. As a result, we receive the same message. Also, there are monthly breakfast meetings sponsored by the RI Association of School Principals that enable HS principals to collaborate. Each meeting is attended by at least one representative of the the Dept. of Ed. I believe that we have built a model that can now be incorporated by larger states. You will just have to regionalize what we have done on a state-wide basis.
Question from Karen Hohman Almeida, Learning Manager, The Walt Disney Co.:
Was the impetus for this at all tied to work life beyond school, where performance-based assessments are becoming prevalent? Or, has there been any research around how this might better prepare our students for business world readiness? Thank you.
Raymond Pecheone:
The impetus for our plan cam from a Summit on High School Reform attended by principals, teachers, elected officials, the governor and many, many business leaders. The strong message coming from the business community was, “We want graduates who can gather information, problem-solve and communicate.” Our diploma plan is closely aligned with the applied learning standards that employers demand.
Question from michael burns, ap english teacher, minisink valley central schools, slate hill ny:
how do performance based assessments differ from state testing in other states?
Lewis Cohen:
Most state testing is focused on the “right” answers, generally involving the recall of discrete facts. This is the kind of thing easily measured with multiple choice questions. Performance assessment, by contrast requires students to apply their knowledge and to demonstrate their skills in performing authentic tasks. This type of assessment reveals more about process, such as problem solving, and gives a better sense of the student’s level of understanding.
Question from Caroline Ward, Intern, Learn and Serve America:
The EdWeek article featured focuses extensively on Barrington High School, a wealthy community that has been developing its Senior Project Program for years. What advice would you provide to more underprivileged schools that may not have the resources to deal with these requirements? How do you respond to the criticism that many rural communities such as mine (Exeter, RI) are facing increased dropouts due to the Performance-Based Assessment System?
Robert Littlefield:
I really have not seen anything in our Senior Project experience that would make it any less possible for students of underprivileged schools to meet the standard, provided the English teachers and the Project coordinators are prepared to offer supports. I think every school is going to experience some temporary push-back from kids because of PBGR’s but I know the only way to improve achievement is to have high standards, communicate them to students and maintain them. Things will get better.
Question from Dave Magnani, Massachusetts Nonprofit Network:
What do you think are the biggest obstacles to getting performance-based assessments to be part of our exit requirements, to balance to impact of paper and pencil tests, and what can be done about themm?
Raymond Pecheone:
Obstacles that are usually cited: 1. little or no reliability and validity evidence based on AERA/APA/NCME test standards 2. subjectivity--Whose work is it-- the students or others?? 3. Comparability within and across schools and districts--are tasks across schools and districts of comparable difficulty and rigor. 4. Scaleability-- to labor intensive and requires considerable capacity building to implement a crediable and defensible assessment system statewide or districtwide. 5. Policy makers in the U.S. don’t trust performance data and have little experience with it (an unknown and too risky). 6. Mixed messages--Runs counter to the federal accountability system that focuses on basic skills, core knowledge (procedural understandings). 7. Costs/benefits-- perception that these systems are much more costly and the benefits of these systems are unknown with respect to teaching and learning. 8. Unionism-often teachers are the heart of the system for training and scoring--which can bump into union contracts and other issues such as reasonable pay for time and service. 8. While I am a hard core proponent of performance assessment --these are real concerns that must be addressed if performance assessment is to become a legitimate assessment strategy for ALL_schools and districts.
Question from Lynda Clary-Burke 3rd grade teacher Henry County Schools (Georgia):
I have been moving into performance-based assessment for my little kids over the past 20+ years; this year all end of 6 weeks grades will be based on performance tasks presented to parents and children as each 6 weeks begins. I am alone in my efforts - our elementary mathematics coordinator is my only resource for face-to-face support in this move. My questions: how do your elementary school teachers, middle school teachers, use performance-based assessments/evaluations? Do your school systems use backward design starting with what you want your high school kids to be able to know/be able to do and then plan backward to kindergarten so that the kids have the most chance of actually getting there? I write rubrics in gradients to assess/evaluate. Do you publish your rubrics on the interent? I’d be very willing to share mine. thanks so much!!!!! Lynda Clary-Burke M
Robert Littlefield:
We are just beginning to use backward design to develop performance-based assessments in the middle school that are aligned with our standards. For years we have worked hard to get our own house in order. It is now time to move further back into the continuum. You can access our Senior Project handbook on our website: www.phspatriots.com
Question from Claudia B. Haas, Science Teacher Coach-Elementary, San Francisco Unified School District:
... As a policy fellow of the San Francisco Education fund, I’m concerned with EQUITY of resources, particularly when there’s a lot more teacher turnover in low-income schools. ... What kind of support do students at high-poverty, high minority schools have to come up with projects for their performance-based assessments?
Robert Littlefield:
Great question! This is the task of your Project coordinators to identify students who are at a disadvantage because, a.) they are on the other side of the digital divide and do not have access to technology at home, or b.) are not connected with the community enough to find a topic or a mentor. The beauty of Senior Project, however, is that it allows for INDIVIDUAL choice of a learning stretch. As long as a student can show it was a stretch for him to learn something substatially new, the topic is acceptable.
Question from Margaret Brown, Coordinator, NW Regional ESD:
How have you handled the varying abilities of special education students?
Robert Littlefield:
This is the constant challenge. One thing that is especially helpful is, when writing the IEP, you keep in mind the Graduation Plan. We develop goals and modifications that will enable them to be successful in demonstrating proficiency. Also, we are completely committed to the concept of full inclusion with the co-teaching model. All our special ed. students have access to the curriculum; they have the benefit of having a special educator in the classroom with the content teacher to help them along the way. There is no way we could hold our special-needs kids accountable without our extensive co-teaching program.
Question from Maria Hirst, Curriculum coordinator, American School of Guatemala:
I am interested in knowing the support that exists for the students in their senior year to prepare the assessment. Are they required to take a class? Is there a teacher who supervises their work throughout the year? etc.
Robert Littlefield:
We embed the Senior Project in the senior English course so there is a teacher who has regular contact with the student throughout the year. We certainly want to foster independence in the student, but we don’t want to cast them to the wolves all alone, either. We are in our summer school right now and one of our courses is “Senior Project Ramp-Up”, a class to give our kids the skills necessary to get a good start on the Project in September.
Question from Karen Schmulbach, ESL teacher, Memphis City Schoos:
How can we help ELLs pass performance-based assessments?
Robert Littlefield:
I’m not sure I can help you with this one. Our ELL attend a neighboring school. I can tell you, however, that they are held to the same standard.
Question from Regina Cockerill, Tech Coordinator, TASIS England:
This is on the front lines of education. Do you know of any academic studies and/or articles written about the effectiveness of Performance-Based Assessment?
Raymond Pecheone:
The Black and Williams studies are the most well known. Also the meta analysis of the classroom assessment literature as summarized by Richard Stiggins is a powerful argument for authentic assessment. Finally the international rankings (using PISA data) appear to offer some prima facia evidence of the potential power of performance assessment to transform teaching and learning (although other structures and curriculum systems are also contributing factors).
Question from Jana Beth Slibeck Francis, Director of Assessment, Research and Curriculum Development, Daviess County Public Schools, KY:
1. Did you work with a contractor to develop the rubrics? 2. Did teachers teach small units to help the students prepare for the performance assessment?
Robert Littlefield:
The rubrics are all ours, developed over a year and a half. We start with our five school-wide learning expectations and departments make modifications that result in more content-specific rubrics. Yes, we work on each phase of the final performance in small units embedded in courses along the way.
Question from Maria Santoso, Vice Principal, Charis National Academy Indonesia:
How do we make a balance between performance based assesment and traditional assessment?
Raymond Pecheone:
A balanced assessment system would have three essential components that would be weighted to be included in a making highs takes decisions--1. standardarized performance tasks that all students respond to within subject areas 2. state or national reference exams required of all students and 3. classroom based assessments that are teacher generated
Question from Nikita Ganatra, Former Middle School ELA Teacher:
How can you get more teachers to use performance based assessments, especially with the excessive assessments that students are already bombarded with?
Robert Littlefield:
First, the pragmatic way: we require that all end-of-course exams have a 50% applied component. Then, the theoretical way: we impress upon them the importance of addressing all learning styles; we convince them to rely on traditional assessments is actually discriminatory against some learners. We also give recognition to teachers who have the courage to try new assessment strategies that involve performance.
Question from Patrick Burk, Chief Policy Officer, Oregon Department of Education:
Did the state establish minimum criteria for meeting a state standard in content areas for awarding the diploma and, if so, how did you approach that problem?
Robert Littlefield:
We have standards in language arts, math and science developed in collaboration with Vermont and New Hampshire as part of our participation in the New England Collaborative Assessment Program. The standards are assessed in our NECAP testing program. The RI Board of Regents for Education is considering ways to incorporate the testing program into the Diploma Plan. It is safe to say the High School Principals in Rhode Island are unanimous in our opposition to testing and our support for the diploma plan.
Question from Bonnie Fortini, Director, Machias Adult & Community Education:
Do (or have) the students play (played) a role in the development of the assessment standards against which their performances are judged, and are these state-wide standards?
Robert Littlefield:
We have students as members of our School Improvement Team. The SIT team has made all final decisions at the local level regarding our Diploma Plan.
Question from Dan Fuchs, Education Reformer:
I was at a Coalition school during the early struggles to get a waiver for a number of schools in New York City, so that we could ask student to present and defend their learning, rather than take the Regents exams in order to graduate. One question we always came up against was the issue of “inter-rater reliability,” which basically calls for a suspension of subjectivity when evaluating portfolios and other non-standardized measurements. How did the state of Rhode Island address this question, if at all?
Raymond Pecheone:
the methods used traditionally are moderation processes that include: training of trainers, scorer training, calibration of scorers, read behinds to verify scores and audits (independently scored).
Question from Geetha Nehemia, Technology Careers, Broome Community College:
Do you think performance based assessments such as portfolios and portfolio defense will gain momentum on a national level?
Raymond Pecheone:
yes if we move from a cottage industry of development to common standards for techncal quality (as is the case w traditional testing) and follow them--we have to break the barrier that these assessments cannot be trusted as reliable and valid.
Question from Richard Ledington, Director of Research, Idaho Division of Professional-Technical Education:
As students leave school and enter the workforce (some with postsecondary education; some not), what will be the utility (benefit) of performance-based tests in communicating with employers the skills and abilities of the applicant or employee? For example, having a high school diploma does not really indicate if or what an applicant can do or is capable of. Thank you.
Raymond Pecheone:
You put your finger on it--you will have a portfolio of work (hopefully digitial) that you can share with employers (et.al.,) that represents authentic work products that that will enable empoyers to assess the relevance and rigor of your work in relationship to their job expectations.
Question from Jose S. Sanchez Assistant Superintendent for Educational Services Wallkill Central School District:
We are in the process of drafting commencement objectives for students, and creating a process for achieving these objectives grades k-12. The idea of a performance based approach to assessing students’ attainment of those objectives not assessed on the required NYS Regents Exams (for example character objectives) is appealing. My question concerns the cost for a performance based assessment process that is meaningful. A performance based assesssment would be in addition to the NYS Regents Exams, which as indicated above is required for graduation from high school in New York State. How much does it cost (time and money) a school district to manage this process?
Robert Littlefield:
We have two, .4 FTE Senior Project coordinators and one Proficiency-Based Graduation Plan coordinator who teaches full-time and receives a stipend. We have invested about 20 hours per teacher of compensated professional development time over the last three years to develop the plan. Admittedly, we have accomlished a great deal on a relatively short budget. Other comparable districts have invested twice as much. The resulting increased support from members of the community, however, makes it worthwhile.
Question from Anonymous, Student, University of Pennsylvania:
Given unequal funding of the various schools with the property taxes providing funding model, how practical or fair is it to judge teachers with different resources at their disposal? Even if performance is judged as a percentage measurement isn’t there only so much teachers can do with limited resources to begin with?
Robert Littlefield:
Do I have your question correct? Are you concerned about teachers of different districts with different socio-economic profiles being compare to each other? We are using a community-designed model to assess students for diploma status. We are not assessing teachers.
Question from Walt Gardner, education writer:
How does R.I. address the two major issues regarding performance assessment: its cost and its reliability?
Robert Littlefield:
The cost has been borne entirely by the local disticts. Given that there is a 5.5% cap on expenditures, we have had to incorporate creative thinking to accomplish this. We have accomplished this in a time of intense political conflict over school funding that included a special town meeting held under a tent in the dead of July to cut the budget and a lawsuit between the school board and town council to recover funds. Despite all this we have been able to implement the plan. Our reliability depends on the quality of our rubrics, our alignment with standards, and our commitment to continuous calibration with teachers looking at student work.
Question from Bill Hammond, Literacy Coordinator, DeKalb County Schools:
What does the research say about the value of performance-based assesment for children of color, especially boys? Who should we be reading to find out additional information?
Raymond Pecheone:
Performance assessment is more authentic, engaging and privileges various learning styles which in surveys and case studies appear that it is a more fair and equitable assessment of learning. Little hard research available to fully support this claim.
Question from Manuel G. Saldivar, PhD student, U. of Colorado at Boulder:
Due to economic and logistical concerns, many states and school districts are beginning to administer standardized tests via computer. Assuming this trend continues, how does computer-based testing help or hinder the spread of performance assessment?
Raymond Pecheone:
I believe computer based testing is in the beginning stages of creating a electronic platform that could be an invaluable policy lever to promote performance based work--As we speak their are a number of artificial intelligence systems being designed to assess written texts and in the future these systems can be applied to more complex performance based assessments and exhibitions.
Question from Mary Ann Hegenrother, Teacher, Amityville Public Schools:
As former principal of schools using a small school model in CT, I’d like to know what the expert thinking is on New Visions schools in NYC. How do they compare to models in the Coal of Essential Schls? What student assessment instruments are most valuable?
Raymond Pecheone:
While I I don’t have deep familarity with new Visions assessment work-- I believe the core structure of “small school’ assessment systems are similar in their focus on critical thinking, problem/project based curriculum, instruction and exhibitions --- these strategies are mission driven in the service of college and workplace readiness skills has as generally defined by the 21st century skills or David Conley’s college knowledge standards. One major difference between assessment models are in the design of the performance assessment, some systems use a generic framework for assessment (e.g.,CES habits of mind) and others use a more subject specific framework (NY performance consortium).
Question from Wan Park, Math Department Chair, Community School for Social Justice:
1. Since PBATs usually take a longer time for students to prepare/complete, the course with PBAT requirement focuses on less number of topics, but more in-depth. How do you satisfy the requirement (NYS, in this case) of covering so many topics in mathematics standards while having students work on PBAT? 2. Many colleges still assess their students via tests. How do I, as a high school math teacher, ensure that students are ready to succeed in college while having students work on PBAT? 3. How do we assess students with IEPs based on PBAT? 4. Consortium schools’ math rubric in NYC mainly assesses students on process strands of NCTM in order to be able to see how students progress through high school years. How do we assess students on content strands with equal amount of scrutiny?
Robert Littlefield:
I think one of the reasons we are successful is we have adopted an A/B Block Schedule with classes meeting every other day for 83 minutes. This model affords more quality instruction time while keeping “housekeeping time” to a minimum. Yes, we do have to sacrifice some breadth once in a while, but if the end result is increased depth of understanding we believe the trade is fair.
Question from Susan Westlund, Project Outreach Director:
1. How does performance based assessment drive instruction? 2. What are the benefits and barriers to scaling up the use of performance based assessment so that we could use the data to study the effects of district-wide, state and national initiatives?
Lewis Cohen:
Performance assessment allows for immediate feedback and unlike standardized tests allows for checking for deeper understanding. In addition performance assessment allows students with a wide variety of learning styles to interact with the material in an on-going way. Finally, these assessments measure important skills such as the ability to apply knowledge in novel contexts that teachers may not otherwise have the opportunity to emphasize. Performance assessments by their nature are generally locally designed and scored. The missing piece is the establishment of professional communities across districts and states that insure scoring criteria is standardized and that common standards are being assessed so that there is a basis for comparison.
Question from Jim Chaput, high school CTE and Photo teacher --Englewood High School Englewood Colorado:
... Is there a way to take effective “notes” on student performance for a certain classroom standard during the assessemnt in the classroom without a massive amount of teacher work/writing? Can a PA span a whole unit as a formative piece documenting student progress as opposed to a one time summative evaluation?
Raymond Pecheone:
I have an example of a prototype electronic system that enables the user to tag (highlight) relevant evidence in student documents--record marginal notes to prompt the scorer and with a ‘click” the data is stored as evidence supporting the scoring rubrics. This is doable and can be recorded over time.
Question from Stacy Brundage, Ed.D., Elementary school teacher:
How did the students perform [this year on the pencil-and-paper state assessments] in comparison to previous state assessments?
Robert Littlefield:
The short answer is they did terribly. We went from about 75% proficiency in math to 35%. HOWEVER, this year we adopted a different test, based on new standards and with new cut-off marks. It was like we switched from the long jump to a hurdle race to measure the physical fitness of our students. Our performance placed us fifth among high schools in Rhode Island. Of significance is the performance of our special needs children. They perform well after receiving instruction in a full-inclusion environment. We have every reason to expect adequate, if not extensive, yearly progress from this point foward.
Question from Dennis L. White, Research & Policy Analyst, George Washington University:
What is the additional or marginal cost for administering and evaluating RI’s performance-based assessment system in per pupil expenditure? What is the benefit-cost ratio?
Robert Littlefield:
Roughly, over the past four years we have invested about $50 K total in professional development activity and approximately $100K in FTE salary to implement our diploma plan. Since we just graduated our first class under the plan, it is probably premature to comment on the benefit-cost ratio.
Question from Jess, Teacher, JO Combs:
Our school (Middle School) is considering moving to a performance-based, rubric grading system this year. The rubric under consideration defines a “4" as “develops in-depth inferences and goes beyond what was taught.” Our question is, what does that look like in the different subject areas? Like, perhaps, math? I like the idea of performance-based assessments, but I’m struggling to figure out what the product might look like!
Robert Littlefield:
IN math we have identified four common tasks for each of our courses. The students are required to demonstrate proficiency on each task, one per marking period. If they do not achieve the standard, they do it again. For instance, one common task in geometry is a series of questions about a baseball diamond. An example is this: the right fielder catches a fly ball on the foul line, 30 ft. beyond first base and immediately throws to third base to pick off a runner who broke for home. How far is the throw? In order to achieve a 4 on the problem-solving rubric the student must: The student accurately makes and defends conjectures and accurately uses geometric properties to solve problems involving angles, lines and polygons. The student accurately solves problems off the coordinate plane involving distance and midpoint. The student effectively demonstrates proper techniques and strategies to solve all problems
Question from Scott Shuler, Education Consultant, CT State Dept of Education:
Rhode Island seems to have taken balanced curriculum into account to a much greater extent than in many state testing systems. How has the inclusion of oft-neglected core subjects such as the arts in the assessment system affected curriculum decisions and curriculum balance in Rhode Island?
Robert Littlefield:
In order to get our diploma plan approved by the RI DOE we had to include a component to allow students to demonstrate proficiency in fine arts and technology. We have built common tasks and common end-of-course exams into each of our art, music and computer courses in order to meet the standard. We now have proficiency requirements in math, language arts, science, technology and fine arts.
Question from Jennie Davidson, French teacher, Connecticut:
What evidence exists to indicate that performance assessment is successful - have graduation rates changed? College going and college persistence? What else has happened in Rhode Island and elsewhere?
Lewis Cohen:
A longitudinal study conducted by the New York Performance Standards Consortium of CES schools that used performance assessments in lieu of most of the State’s regents exams, found that these graduates, primarily children of color, enrolled in college at rates higher than their peers and were more likely to persist.
Question from Hope Torrents, School Programs Coordinator, Lowe Art Museum:
I have worked with Wiggins and McTighe’s resources and think it’s not altogether a ‘simple’ program. Was there a lot of professional development for the teachers using these resources?
Robert Littlefield:
When we committed to the Senior Project plan four years ago we began sending teams of 3-5 teachers to professional development institutes to become teacher-leaders. That cadre of 18 teachers forms a core of our trainers for the rest of the faculty. We have also dedicated about 6-8 hours of professional development time each year per teacher to bring the rest of the faculty along. You are right, it is not simple. However, it is far from being Herculean in difficulty.
Question from Fred Hutchinson, ELA Specialist, NY:
By what process do you guide the learner to develop their own thesis, motivating (especially younger adolescents) to devine, let alone pursue their own interests is quite an difficult task to manage.
Robert Littlefield:
Many of them call on their results of the Armed Services Vocational Aptitude Battery for direction. We also rely on advisors to begin discussions with students in the ninth grade to help them formulate ideas during advisory period. We will ask seniors to submit letters of intent in September, right after returning to school, to get the job started. Once accepted, a student must petition the SP Steering Committee to change their subject.
Question from Ryan McGrath, Teacher, Commack High School:
I think that performance based assessment is a critical piece of high school assessment, but opponents often target it’s reliability. Have you had a chance to analyze the success and reliability of the performance based assessment system in Rhode Island? If so, what data did you use and what were the results?
Raymond Pecheone:
I have not analyzed RI reliability data-- But performance systems that are state sponsored need to have strong (crediable and defensible) moderation process in place--to train scorers, examine interrater agreement, audit a sample of scores (independently score with expert scorers) as well as processes to adjudicate discrepant scores.
Question from Milvia Concas, Teacher, Currently, School Psychologist & Educational Leadership, Graduate Student:
Mr. Littlefield has been successful in working to tranform the culture, mission, and focus of PHS into a high-performing, personalized learning environment. I’d like to know what strategies have been effective in this process and how have parents and the community been involved in the transformation?
Robert Littlefield:
I think I would point to two strategies that have helped us chart a course of school reform. The first is an effort to personalize our school, to convey a message to our students and families that we recognize individual strengths, needs and interests. We adopted a school-wide advisory program four years ago. Each morning, every faculty member sits with a group of 10-14 students from the same grade in an effort to personalize their high school experience. The advisor plays a key role in making sure students are fully cognizent of our expectations and helps student chart a path through school, up to and including Senior Project. The second is to be committed to collaborative leadership. This was the toughest because it meant relinquishing individual control of a community I felt totally responsible for. However, what I got in return far outdistanced anything I could have accomplished myself. Empowering the School Improvement Team to develop the diploma plan and giving them complete control over professional development activities was a big step. Today, our SIT is a highly-respected institution in our school. When they make a decision regarding the daily timetable, professional development, graduation requirements, curriculum changes, etc. the decision is widely accepted because people know the group considered all points of view and chose the course of action best suited for our school. I turn similarly to our faculty advisory council for teacher-centered issues and to my department chairs’ council for advice and constructive criticism.
Question from Larry Hoffman, Ph.D., Educator, Milwaukee:
Assuming that performance assessments should begin as early as 2nd or 3rd grade, in what ways should they be similar to and different from performance assessments carried out in high school?
Raymond Pecheone:
Performance assessment at its base is engaging students in inquiry and exploration. Opportunities for engaging students in this work in the early grades needs to be both valued and be specifically included(high priority) in the school/district curriculum. In addition, district assessments both formative and summative must include performance elements that count and are evaluated and scored.
Question from Lisa Layera Brunkan, Co-founder, Fundourfuturewashington, state/national coalition seeking funding for libraries/21st c. skills:
Dear Panel: I’m hoping you can comment on how important the information resourcing (research!) proved to be for your model. Specifically, with declining funds for school libraries and replacement of certified teacher-librarians with paraprofessionals I’m wondering if the type of performance-base assessment championed in Rhode Island necessitates the kids being trained in information and digital literacy to execute it properly. Examples of how Rhode Island addressed this aspect of the project development would be highly appreciated. ... Great topic edweek, great vision Rhode Island! Lisa
Raymond Pecheone:
Lisa in the work I have done in the Bay area digital literacy is essential and I believe is a key competency that can make the work of performance assessment feasible, accessible and cost effective. Therefore our work with Envisionschools include student webpages, electronic scoring, training (on the drawing boards) and a project exchange where teachers can share practice.
Question from Bill Hammond, Literacy Coordinator, DeKalb County (GA):
What are the implications of PBA for schools who have not made AYP?
Raymond Pecheone:
PBA is essential to support student ‘s conceptual understandings and making the school work relevant and meaningful. With that said, schools and districts- given the unrelenting skill demands of NCLB --need to purposefully create space for PBA in their curriculum to attend to the thinking skills neeeded for college and life. They need both skill development and conceptual understandings and PBA should not be sacrificed for test prep.
Question from Phillip Person, Recent MA Ed Grad, Current Army Officer:
My question comes as a person that is the recipient of the product of the public school system, a taxpayer, and a person considering joining the education profession. (perhaps an unusual combination.) I am concerned that this assessment approach will result in less capable adults in society if the students are able to present a good picture but have not internalized the fundamental knowledge needed to function as adults. It sounds like it briefs well, but I am wary of its application.
Lewis Cohen:
I would argue precisely the contrary. The beauty of performance assessment is that it asks students to perform real tasks under real world conditions. What better measure of preparation than actual execution. Not only must students prepare a presentation, prepare a written document but they must also collaborate with others and answer questions from a live audience thereby demonstrating an ability to think on their feet. Compare this with the likelihood that any adult must be able to answer multiple choice questions to solve a work or other real life situation.
Question from Dennis L. White, Research & Policy Analyst, George Washington University:
It may be too soon to measure the effect(s), but have you observed - or do you expect to observe - changes in academic progress, high school completion, student receptiveness and behavior (generally), or teacher, family, and community support, that you can attribute to performance-based assessment?
Robert Littlefield:
We have a program analysis protocol in place to track trends and gather data. We are especially concerned with failures at the ninth grade level because of increased rigor associated with common tasks to demonstrate proficiency.
Question from Marinda Ferrell Teacher Pittsburg HS:
Will there be a coherent single comprehensive assesment system for all the states in the future?
Raymond Pecheone:
I believe state autonomy in assessment is firmly rooted and going to a federal system of testing is a big mountain to climb. But looking into the crystal ball their are some indication that standards and assessment could look very different nationally and at the state/district level. For example under discussion is: 1. national standards and/or curricular frameworks 2. NCLB overhauled and AYP is transformed to a growth scale 3. more balanced assessments are needed 4. multiple measures including performance assessments are needed 5. classroom asssessment and/or school based work should count and be included in any state accountability system. 6. need to create formative assessment systems to support teaching and learning
Question from Cliff Brush, Sr. Proj. Mgr., Office of Supt., Portland Public Schools (Oregon):
... What are examples of “do” and “don’t” lessons learned in small rural and large urban districts? ... Is there or will there be a cost-benefit study of the RI system? If yes, when and where is/will it be available? ... What is the Web link to information about RI’s performance-based assessment system?
Robert Littlefield:
Do: 1. Communicate your expectations to students from the day they enter your school as ninth graders. 2. Invite as many members of your community to participate in the program as you can. It opens your school up in unprecendented ways and results in new and energetic supporters. 3. Have many safeguards in place to combat plaigerism and forgery. Kids are very resourceful and the consequences can be devastating. 4. Devote a lot of time to teacher collaboration and looking at student work. Don’t: 1. Assume kids have retained all your expectations after hearing them once. 2. Allow kids to choose topics that are too broad and lack focus. 3. Underestimate what kids can accomplish. They will amaze you! Try this link to learn more information at the RI DOE website: http://www.ride.ri.gov/HighSchoolReform/default.aspx
Scott J. Cech (Moderator):
Thanks for all the great questions, and many thanks to Messrs Cohen, Pecheone, and Littlefield for their time and insights. Unfortunately, we have more questions than time, so we’ll have to leave the discussion there. A transcript of this chat will be available on Education Week’s Web site shortly: http://www.edweek-chat.org
The Fine Print
All questions are screened by an edweek.org editor and the guest speaker prior to posting. A question is not displayed until it is answered by the guest speaker. Due to the volume of questions received, we cannot guarantee that all questions will be answered, or answered in the order of submission. Guests and hosts may decline to answer any questions. Concise questions are strongly encouraged.
Please be sure to include your name and affiliation when posting your question.
Edweek.org’s Online Chat is an open forum where readers can participate in a give- and-take discussion with a variety of guests. Edweek.org reserves the right to condense or edit questions for clarity, but editing is kept to a minimum. Transcripts may also be reproduced in some form in our print edition. We do not correct errors in spelling, punctuation, etc. In addition, we remove statements that have the potential to be libelous or to slander someone. Please read our privacy policy and user agreement if you have questions.
---Chat Editors