Education Opinion

The Letter From... SBR is Really RB, and RB is Really SBR: Do You Want Me to Apply SBR to Your Program?

March 05, 2008 6 min read

Every little brother must know the big brother line: “Yes means no, and no means yes. Do you want me to hit you?”

No Child Left Behind was intended to encourage the private sector to develop a class of products and services with demonstrated efficacy. Providers and investors who believed this determined that if they could prove their offerings improved student performance, they could surmount the formidable barriers to entry that mainstream publishers built up over a century. This group welcomed the definition of “Scientifically Based Research” in NCLB that emphasized comparing the academic performance of similarly situated students who do and do not receive the program under study. Supplementary Educational Services providers, believing that they could not meet this standard, lobbied successfully for a lower, “Research Based” standard. That approach permitted evidence as limited as reference to bodies of related research that suggest a particular program might be effective.

The policy problem facing legislators and the Department of Education is how to balance three requirements.

First, to assure that the government does not waste its money, teachers’ time, or students’ education, on programs that simply don’t work. We’d like a standard high enough to at least weed out products and services benefiting solely from the placebo effect.

Second, understanding that the state of the art in k-12 program evaluation is nowhere near advanced enough to be confident that a very high bar will not screen out more useful programs than those lacking utility, to create a standard that leads to a market large enough to give educators real choice among offerings. We don’t want a standard no one can meet.

Third, recognizing that the growth of private sector offerings in school improvement depends on new investment in new providers, assuring that the standards are simple, clear and stable. Investors will not put money into something when they don’t understand the criteria of technical success or can’t rely on it to hold for the period during which they expect to see their financial return. We don’t want an arbitrary, capricious or ever-changing standard.
For roughly the past year, I have provided readers of School Improvement Industry Week (see explanation below) with the standards states are beginning to employ to assess the efficacy of new and current Supplementary Educational Service (SES) providers. Yesterday I excerpted the relevant Kansas regulations:

The Kansas State Department of Education (KSDE) will select providers of supplemental educational services who adequately meet the eligibility requirements described in the application. Applicants who meet the criteria will be included on an approved list maintained by KSDE and made available to local districts.... Providers will be removed from the list if they are unable to sustain the requirements or meet the achievement goals within two years of consecutive service....

Evidence of Effectiveness.... Demonstrate that the program is of high quality and that the applicant has been effective in raising the achievement levels of student who have received services in Kansas. If new to Kansas provide evidence in locations with similar demographics as Kansas. Use evidence from standardized tests, or student grades, teachers’ assessments, improved student attendance, retention/promotions rates, or other measures which can be correlated to Kansas data points. Strongest consideration will be given to evidence of positive impact on student achievement, particularly low-income underachieving students to include SPED and ESL students. Include details to explain whether this evidence was gathered from services that your organization provided or from another entity’s use of the program. Please note that priority will be given to third party independent research.

Some months earlier I offered the new rules promulgated by the Rhode Island Department of Education:

To be included on the approved list of supplemental educational services providers, applicants must meet the following criteria:
• Have a demonstrated record of effectiveness or have a high probability of increasing student academic achievement….
• Provide instruction that is of high quality, research-based, and specifically designed to increase academic achievement of eligible children on state assessments and attain proficiency in meeting the RI academic achievement standards. Reading instruction must be scientifically based and proven to be effective and include the National Reading Panel Report criteria

Reviewing the federal government’s implementation of NCLB’s SBR/RB provisions I think it’s fair to say that the states have been moving towards the legislation’s Scientifically-Based Research standard to define Research Based under Supplemental Educational Services. I am as adamant about the need for SES providers to demonstrate efficacy as anyone, not because it’s required by law – but because it is essential for the SES program’s future in a reauthorized NCLB. As Radio Moscow commentators used to say during the Cold War, “it is no coincidence” that states have adopted a very high standard of evidence for a program most don’t support.

But it is also fair to say that the Federal government has used the NCLB’s Research-Based standard to define SBR for most school improvement programs. The obvious example here is Reading First, where programs were judged on the basis of a third or fourth level employee’s view of their relationship to a body of knowledge on phonics, rather than specific evidence of program efficacy. It is also no coincidence that the big winners here were programs sold by the big publishers, and authored by the Department’s Reading First advisors.

If one goal of NCLB is to encourage new investment in new firms with new effective school improvement products, services and programs, the implementation of the law’s SBR and RB provisions could hardly be less helpful. No investor can look at this history and feel any confidence in the government’s interest in real choice or what works. The real message has to be “watch what we do, not what we write.”

Regarding School Improvement Industry Week

School Improvement Industry Week (SIIW) is one of the information services I provide to my clients. SIIW summarizes or excerpts five documents I consider important professional reading for leaders in this field. The five-page report generally includes something on federal or state regulation of No Child Left Behind, education research, marketing data, something specific to one state, and program evaluation. The document is delivered by email Tuesday as a web-enabled pdf, with every item hotlinked to its source.

I developed the service because it’s something I wished I had when I was New American Schools COO and President of its Education Entrepreneurs Fund. As important as this kind of information was to my responsibilities for organizational strategy, my in box was too full for me to take the extra time required to find and read the right materials. With SIIW I could have relied on someone who was doing this for a living, printed out the report, and put it in my briefcase or suit coat pocket to read when I was in a cab, on a plane or with a few minutes to spare. If I read one page a day for 52 weeks I’d have the gist of 260 documents covering our industry. That’s a lot better than what I actually was able to read.

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


School & District Management Webinar Examining the Evidence: Catching Kids Up at a Distance
As districts, schools, and families navigate a new normal following the abrupt end of in-person schooling this spring, students’ learning opportunities vary enormously across the nation. Access to devices and broadband internet and a secure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Branding your district matters. This webinar will provide you with practical tips and strategies to elevate your brand from three veteran professionals, each of whom has been directly responsible for building their own district’s brand.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Nearly two-thirds of U.S. school districts are using hybrid learning right now with varying degrees of success. Students and teachers are getting restless and frustrated with online learning, making curriculum engagement difficult and disjointed. While
Content provided by Samsung

EdWeek Top School Jobs

Special Education Teacher
Chicago, Illinois
JCFS Chicago
Communications Officer
Chattanooga, Tennessee
Hamilton County Department of Education
MS STEM Teacher
Woolwich Township, NJ, US
Kingsway Regional School District
High School Permanent Substitute Teacher
Woolwich Township, NJ, US
Kingsway Regional School District

Read Next

Education Briefly Stated Briefly Stated: January 13, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Obituary In Memory of Michele Molnar, EdWeek Market Brief Writer and Editor
EdWeek Market Brief Associate Editor Michele Molnar, who was instrumental in launching the publication, succumbed to cancer.
5 min read
Education Briefly Stated Briefly Stated: December 9, 2020
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of articles from the previous week that you may have missed.
8 min read