Early Childhood

Child-Care Rating Systems Earn Few Stars in Study

By Christina A. Samuels — September 10, 2013 4 min read
  • Save to favorites
  • Print

A new study on child-care rating systems appears to bolster concerns among some in the early-learning field that the ratings generated by those systems are only tenuously connected to learning outcomes.

The researchers, who were from several universities, found that children attending highly rated pre-K programs did not have significantly better results in math, prereading, language, and social skills when they finished the programs, compared with the children attending lower-rated programs.

The findings, published last month in the journal Science, could have implications for states as they work to tie their ratings to real-world outcomes.

Researchers were studying “quality rating and improvement systems.” As a result of federal Race to the Top Early Learning Challenge grants, funding from states, and foundation support, nearly every state has or is creating such a system, known by the shorthand QRIS. About 13,000 child-care programs in 20 states have been rated through a QRIS. Most of the systems use symbols such as stars to represent levels of quality. But those systems draw in so many elements that a center’s rank may end up with a distant connection to teacher-child interactions, which are known to be a strong predictor of how well children do in preschool and afterward, said Terri J. Sabol, a postdoctoral fellow at the Institute for Policy Research at Northwestern University and the study’s lead author.

“My biggest take-away is that states need to simplify their rating systems,” Ms. Sabol said in an interview. “There’s something really appealing about having these five-star systems, but that comes at a cost because those stars don’t mean a lot for child outcomes.”

Gladys Wilson, the president and CEO of Qualistar Colorado, an organization that rates child-care centers in that state, said the rating system has had the benefit of providing a clear path to continuous improvement for care providers. The “improvement” aspect of a QRIS is as important as the ratings themselves, she said.

Extensive Data

Qualistar has been intensively studied by the RAND Corp., which also found little connection between learning outcomes and ratings, though the study couldn’t draw strong conclusions because of difficulty in tracking children.The new study uses data collected in two studies that provided detailed information on prekindergarten teachers, children, and classrooms in 11 states, collected between 2001 and 2004.

Using information on more than 2,400 children in 673 preschools, the authors of the new report plugged those numbers into scoring algorithms that they created for nine states. Each of those states rates programs on staff qualifications, staff-child ratio and group size, family partnerships, and learning environment. Those measures and others are combined to produce a rating.

The researchers also created an additional measure, teacher-child interactions, which was evaluated through the Classroom Assessment Scoring System, or CLASS. The CLASS measure has been adopted in the past few years by Head Start as a method of evaluating preschool quality.

Teachers Crucial

After linking outcomes to the evaluation measures, the authors found that teacher interactions had the highest connection to student learning, followed by learning environment. Teacher qualifications, class size, and family partnerships had a weaker and sometimes inconsistent connection. Thus, rating systems that combined all those measures also had a weaker and less consistent connection to child outcomes, the study shows.

Study co-author Robert C. Pianta, the dean of the education school at the University of Virginia and the creator of the CLASS evaluation instrument, said that one way to simplify rating systems could be to make some elements non-negotiable.

“There shouldn’t be any variation in [teacher-child] ratio, or health or safety provisions, or whether the teacher has certain level of training,” he said. Once those elements are removed, the systems can focus more closely on the most powerful measures, he said."The people doing this work are terrific, they’re very knowledgeable about the field,” Mr. Pianta said, but he added that the desire to include many different measurements is a challenge. “We’re really rolling out a big policy without knowing what the consequences of that policy might or might not be,” he said.

The new findings are similar to the results of the RAND study of the 14-year-old Qualistar program, which included evaluations of 1,300 children served by more than 100 child-care centers and in-home day-care providers. Only 7 percent of the children remained in the study for its entire duration, but the findings suggested more research was needed before ratings programs were implemented at scale.

Gail L. Zellman, the principal investigator on the Qualistar study for RAND, said that child-care rating systems have had the beneficial effect of driving a conversation about what is most important in a good day-care center or preschool. But, she added, “the field has not sufficiently determined how to evaluate quality and how to assess it in a valid way.”

A version of this article appeared in the September 11, 2013 edition of Education Week as Child-Care Rating Systems Earn Few Stars in Study


English-Language Learners Webinar AI and English Learners: What Teachers Need to Know
Explore the role of AI in multilingual education and its potential limitations.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Pave the Path to Excellence in Math
Empower your students' math journey with Sue O'Connell, author of “Math in Practice” and “Navigating Numeracy.”
Content provided by hand2mind
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Recruitment & Retention Webinar
Combatting Teacher Shortages: Strategies for Classroom Balance and Learning Success
Learn from leaders in education as they share insights and strategies to support teachers and students.
Content provided by DreamBox Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Early Childhood What the Research Says 6 Challenges for Early Educators as Preschool Growth Halts
School enrollment for the nation’s youngest learners has nosedived—and could cause long-term problems.
4 min read
Close crop of the back of a pre-school girl's head showing her playing with foam puzzle pieces of shapes and numbers.
Early Childhood What the Research Says Starting School in Infancy Can Help Low-Income Children Keep Up With Peers in Elementary School
Research on a birth-to-4 initiative in Tulsa finds academic gains through 3rd grade.
4 min read
Teacher Silvia Castillo, center, reads a book about dinosaurs with Everett Fisher, left, and Jaz Endicott in a toddler classroom at Kids First on Jan. 30, 2019 in Lincoln, Neb.
Teacher Silvia Castillo, center, reads a book about dinosaurs with Everett Fisher, left, and Jaz Endicott in a toddler classroom at Kids First on Jan. 30, 2019, in Lincoln, Neb.
Gwyneth Roberts/Lincoln Journal Star via AP
Early Childhood Why Parents 'Redshirt' Their Kids in Kindergarten
Parents have a number of reasons why they decide to delay their children's school entry, but it's not always a good idea.
5 min read
Students participate in a pre-kindergarten class at Alice M. Harte Charter School in New Orleans on Dec. 18, 2018. Charter schools, which are publicly funded and privately operated, are often located in urban areas with large back populations, intended as alternatives to struggling city schools.
Students participate in a pre-kindergarten class at Alice M. Harte Charter School in New Orleans on Dec. 18, 2018.
Gerald Herbert/AP
Early Childhood Q&A An Investment in Early-Childhood Education Is Paying Off Big
Richard Tomko believes that expanding the early education pipeline buffers schools against enrollment loss and academic struggles.
2 min read
Dr. Richard Tomko, Superintendent of Belleville Public Schools in Belleville, N.J., visits science teacher Paul Aiello’s Medical Academy Field Experience class on Tuesday, January 10, 2023. The Medical Academy’s class uses Anatamoge tables, an anatomy visualization system that allows students to garner a deeper, comprehensive understanding of the human body and medical tools to prepare them for careers in the medical field.
Richard Tomko, superintendent of Belleville Public Schools in Belleville, N.J., has expanded academic programs while restoring trust in the school system.
Sam Mallon/Education Week