Leadership Symposium Early Bird Deadline Approaching | Join K-12 leaders nationwide for three days of empowering strategies, networking, and inspiration! Discounted pricing ends March 1. Register today.
Education Opinion

The Letter From... SBR is Really RB, and RB is Really SBR: Do You Want Me to Apply SBR to Your Program?

By Marc Dean Millot — March 05, 2008 6 min read
  • Save to favorites
  • Print

Every little brother must know the big brother line: “Yes means no, and no means yes. Do you want me to hit you?”

No Child Left Behind was intended to encourage the private sector to develop a class of products and services with demonstrated efficacy. Providers and investors who believed this determined that if they could prove their offerings improved student performance, they could surmount the formidable barriers to entry that mainstream publishers built up over a century. This group welcomed the definition of “Scientifically Based Research” in NCLB that emphasized comparing the academic performance of similarly situated students who do and do not receive the program under study. Supplementary Educational Services providers, believing that they could not meet this standard, lobbied successfully for a lower, “Research Based” standard. That approach permitted evidence as limited as reference to bodies of related research that suggest a particular program might be effective.

The policy problem facing legislators and the Department of Education is how to balance three requirements.

First, to assure that the government does not waste its money, teachers’ time, or students’ education, on programs that simply don’t work. We’d like a standard high enough to at least weed out products and services benefiting solely from the placebo effect.

Second, understanding that the state of the art in k-12 program evaluation is nowhere near advanced enough to be confident that a very high bar will not screen out more useful programs than those lacking utility, to create a standard that leads to a market large enough to give educators real choice among offerings. We don’t want a standard no one can meet.

Third, recognizing that the growth of private sector offerings in school improvement depends on new investment in new providers, assuring that the standards are simple, clear and stable. Investors will not put money into something when they don’t understand the criteria of technical success or can’t rely on it to hold for the period during which they expect to see their financial return. We don’t want an arbitrary, capricious or ever-changing standard.
For roughly the past year, I have provided readers of School Improvement Industry Week (see explanation below) with the standards states are beginning to employ to assess the efficacy of new and current Supplementary Educational Service (SES) providers. Yesterday I excerpted the relevant Kansas regulations:

The Kansas State Department of Education (KSDE) will select providers of supplemental educational services who adequately meet the eligibility requirements described in the application. Applicants who meet the criteria will be included on an approved list maintained by KSDE and made available to local districts.... Providers will be removed from the list if they are unable to sustain the requirements or meet the achievement goals within two years of consecutive service....

Evidence of Effectiveness.... Demonstrate that the program is of high quality and that the applicant has been effective in raising the achievement levels of student who have received services in Kansas. If new to Kansas provide evidence in locations with similar demographics as Kansas. Use evidence from standardized tests, or student grades, teachers’ assessments, improved student attendance, retention/promotions rates, or other measures which can be correlated to Kansas data points. Strongest consideration will be given to evidence of positive impact on student achievement, particularly low-income underachieving students to include SPED and ESL students. Include details to explain whether this evidence was gathered from services that your organization provided or from another entity’s use of the program. Please note that priority will be given to third party independent research.

Some months earlier I offered the new rules promulgated by the Rhode Island Department of Education:

To be included on the approved list of supplemental educational services providers, applicants must meet the following criteria:
• Have a demonstrated record of effectiveness or have a high probability of increasing student academic achievement….
• Provide instruction that is of high quality, research-based, and specifically designed to increase academic achievement of eligible children on state assessments and attain proficiency in meeting the RI academic achievement standards. Reading instruction must be scientifically based and proven to be effective and include the National Reading Panel Report criteria

Reviewing the federal government’s implementation of NCLB’s SBR/RB provisions I think it’s fair to say that the states have been moving towards the legislation’s Scientifically-Based Research standard to define Research Based under Supplemental Educational Services. I am as adamant about the need for SES providers to demonstrate efficacy as anyone, not because it’s required by law – but because it is essential for the SES program’s future in a reauthorized NCLB. As Radio Moscow commentators used to say during the Cold War, “it is no coincidence” that states have adopted a very high standard of evidence for a program most don’t support.

But it is also fair to say that the Federal government has used the NCLB’s Research-Based standard to define SBR for most school improvement programs. The obvious example here is Reading First, where programs were judged on the basis of a third or fourth level employee’s view of their relationship to a body of knowledge on phonics, rather than specific evidence of program efficacy. It is also no coincidence that the big winners here were programs sold by the big publishers, and authored by the Department’s Reading First advisors.

If one goal of NCLB is to encourage new investment in new firms with new effective school improvement products, services and programs, the implementation of the law’s SBR and RB provisions could hardly be less helpful. No investor can look at this history and feel any confidence in the government’s interest in real choice or what works. The real message has to be “watch what we do, not what we write.”

Regarding School Improvement Industry Week

School Improvement Industry Week (SIIW) is one of the information services I provide to my clients. SIIW summarizes or excerpts five documents I consider important professional reading for leaders in this field. The five-page report generally includes something on federal or state regulation of No Child Left Behind, education research, marketing data, something specific to one state, and program evaluation. The document is delivered by email Tuesday as a web-enabled pdf, with every item hotlinked to its source.

I developed the service because it’s something I wished I had when I was New American Schools COO and President of its Education Entrepreneurs Fund. As important as this kind of information was to my responsibilities for organizational strategy, my in box was too full for me to take the extra time required to find and read the right materials. With SIIW I could have relied on someone who was doing this for a living, printed out the report, and put it in my briefcase or suit coat pocket to read when I was in a cab, on a plane or with a few minutes to spare. If I read one page a day for 52 weeks I’d have the gist of 260 documents covering our industry. That’s a lot better than what I actually was able to read.

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Reading & Literacy Webinar
Science of Reading: Emphasis on Language Comprehension
Dive into language comprehension through a breakdown of the Science of Reading with an interactive demonstration.
Content provided by Be GLAD
English-Language Learners Webinar English Learners and the Science of Reading: What Works in the Classroom
ELs & emergent bilinguals deserve the best reading instruction! The Reading League & NCEL join forces on best practices. Learn more in our webinar with both organizations.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: February 7, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: January 31, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Briefly Stated: January 17, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education In Their Own Words The Stories That Stuck With Us, 2023 Edition
Our newsroom selected five stories as among the highlights of our work. Here's why.
4 min read
102523 IMSE Reading BS
Adria Malcolm for Education Week