Education Opinion

The Letter From: “What’s a CODiE and How Do I Get One” (and What Does it Tell Buyers)?

By Marc Dean Millot — November 21, 2007 5 min read
  • Save to favorites
  • Print

As I quoted last week, consumers don’t buy drills, they buy holes.

Likewise, educators don’t buy edtech software, they buy improvements in student performance.

That brings us to the Software and Information Industry Association’s (SIIA) CODiE award. (The name is derived from computer “code"; “i” is the symbol for “information.”) Does winning one tell educators what they need to know about edtech products? Can a prospective buyer rely on the award as an objective indicator that their purchase will improve student performance?

The short answer is “no.”

That is a problem for the whole school improvement industry.
(Download as a podcast here.)

How do consumers determine quality in a sea of offerings? They test drive, listen to buyers - and look for symbols. If a product is rated highly by Consumer Reports, it has credibility. If it wins an award from peers in its industry group, competitors must think it’s pretty good.

SIIA says that its CODiE is “the only-peer reviewed honor” in k-12 edtech products. Firms nominate themselves for various product categories and pay an entry fee of a few hundred dollars for each category. Nominations for the 2008 awards are open until December 3.

The Process

The first phase of the award competition is an informal ranking by representatives from the trade press, technology writers, analysts, consultants and subject-matter experts, etc. who serve as volunteer judges. Each product is reviewed by at least two judges; each judge reviews 10-15 products. Each judge provides their top five choices in each category they rank. Their votes determine the competition’s finalists.

In the second phase, SIIA member firms’ vote on the finalists.

In the third, member firms attending the association’s annual conference vote.

The informal review constitutes 50% of the total points a product may win, the sum of second and third phase voting counts for the rest. Products with the highest total score win their categories.

How the Award is Judged, Used and Perceived

(edbizbuzz.com readers may be interested in listening to my five-part podcast series “Educators as Informed Consumers” starting here. Each part runs under seven minutes.)

Finalists and winners have the right to display the CODiE label in association with their product and firm advertising.

If you are an educator reading an ad in a trade publication, walking past a booth in the vendors’ hall at a education conference, or leafing through a marketing brochure, and see the CODiE symbol associated with a product, you might be inclined to view it as evidence of efficacy. Look and feel are nice, but the taxpayer expects your “buy” decision to be about what works. So it would be reasonable to think that’s what SIIA and the firm in question are trying to tell you.

A CODiE probably does correlate with a provider’s interest in quality, broadly defined. Preparing for the competition gives companies some motivation to develop a good product. The first phase of the process involves feedback that helps refine programs. Given how judges seem to want to review products without outside assistance, the award is probably a good indicator of usability.

But if the CODiE is meant to denote the “quality” of its winners’ products; we have to ask how SIIA defines the term. What I find absolutely unfathomable as an outsider, that would certainly give me pause if I were an entrant, and floor me if I were a buyer of products with the label - is a complete lack of public information on the criteria that judges apply.

The nomination form itself suggests that reliable evidence that the product improves student academic performance is not relevant. Internal or third-party evaluation of effectiveness – scientifically based research, if you will - is not part of the competition. The CODiE reflects a bygone, input-oriented view of public schooling and SIIA does not employ the term “peer review” as it is used generally understood by professional educators.

If CODiEs were something companies kept to themselves as a means of internal professional recognition, that would be the end of it. The problem here is that an award that with no basis in evaluation is promoted by SIIA as a means for providers to increase sales in a market that views the “quality” of products as an outcome - a demonstrated contribution to student performance.

I doubt that any firm with a CODiE will correct any misimpression the educator reading the ad, walking past the booth, or leafing through the brochure might have on this point. No winner is going to append “this award says nothing about value-added to test scores” to the CODiE logo.

What is to be done?

“Buyer beware” is not the perception of industry quality we want to foster. Yet, to the extent that SIIA allows this kind of CODiE to continue, it can’t avoid complicity in misleading buyers - or allowing buyers to think something that isn’t correct if that’s what they want to think (choose your own semantics). However you describe it, this approach only contributes to the widespread perception that all the private sector cares about in k-12 education is profit.

That perception is a political problem for the whole industry, one that contributed mightily to the near-gutting of NCLB I, and one we have maybe two years to correct before we get back on that awful track to reauthorization.

• It’s time for SIIA to start making objective demonstrations of effectiveness part of the CODIE nomination process, especially in phase one - or drop the award.

• It’s time for providers whose products and services are based on evaluation to either move their trade groups in a direction where results matter, or to form one that places evaluation at the forefront of industry values.

• It’s time for the buyers’ associations – that is educators (teachers, principals, administrators, superintendents) to start 1) insisting that purchases be grounded in evidence of effectiveness based on program evaluation, and 2) recognizing providers that build evaluation into their programs with awards and, more important, contracts.

Evaluation science is far from perfect. It is helpful. It is getting better. And something beats nothing, which is where too many providers are today. We need to start expecting that providers - especially the ones with the brands we’ve always trusted to imply quality - are incorporating an advancing state of the art, instead of arguing about its faults, or ignoring it altogether. Awards are symbolic of industry values, and the school improvement industry’s values must include program evaluation.

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Reading & Literacy Webinar
Your Questions on the Science of Reading, Answered
Dive into the Science of Reading with K-12 leaders. Discover strategies, policy insights, and more in our webinar.
Content provided by Otus
Mathematics Live Online Discussion A Seat at the Table: Breaking the Cycle: How Districts are Turning around Dismal Math Scores
Math myth: Students just aren't good at it? Join us & learn how districts are boosting math scores.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Achievement Webinar
How To Tackle The Biggest Hurdles To Effective Tutoring
Learn how districts overcome the three biggest challenges to implementing high-impact tutoring with fidelity: time, talent, and funding.
Content provided by Saga Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: March 13, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Briefly Stated: February 21, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: February 7, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: January 31, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read