California State University, Long Beach
Inside CSULB Logo

Streamlining The Evaluation Process

Published: September 1, 2015

Helping to streamline a sometimes complicated and tedious process may be the best way to describe the main benefit of the newly formed Center for Evaluation and Educational Effectiveness (CEEE)—for those who choose to utilize it.

“We’re really bringing together initiatives that may have been independent in the past, such as external evaluations of federal or foundation-supported programs that faculty typically do on their own,” said Anna Ortiz, chair of the Educational Leadership Department and executive director of the center. “Now any faculty member can choose to run their evaluation or external contract through this center and use its resources and trained staff. It means they are not having to train a new staff every time they get a project.”

The center’s mission focuses on using scholarly and practical expertise to focus on four main areas—examining the effectiveness of practices, programs, and services for advancing equity, access and achievement in educational settings; supporting the application of data-based, high-impact practices, programs and services; encouraging innovation and effectiveness in organizational, instructional and programmatic practice; and working with partners to develop effective practices for urban education.

“Individual faculty, staff and administrators at CSULB submit approximately 300 grant proposals per year to federal, state and private funding agencies,” said Simon Kim, Interim Associate Vice President for Research and Sponsored Programs. “Most of these funding agencies now require some form of program evaluation that gathers data and then analyze them in such a way that the findings can be used to determine the extent to which the program is achieving its stated objectives and anticipated outcomes. With increased demand for evaluation, it has become quite a challenge to find a competent and skilled evaluator. I expect the Center for Evaluation and Educational Effectiveness to help faculty, staff and administrators with identifying funding opportunities, preparing grant proposals and carrying out evaluation services.”

When engaged, the center will take responsibility for developing a client’s evaluation plan. More often than not, evaluators are brought in to get involved with a project after it’s been funded. And sometimes a project may have even been going on for years before those overseeing it realize they need to bring in an evaluator.

“The best part about this is that you get to come in the front door and you get to work collaboratively with the group from the start,” said Don Haviland, an associate professor in the Educational Leadership Department and director of the center. “While they’re working on their project you, as the evaluator, are looking at, ‘How will we know if this thing is working and how can we give them information to help them make it better?’ It’s a nicer partnership and it’s a better way to develop a program.”

The idea for the center came about roughly six years ago when a handful of faculty were discussing how they do independent evaluation work in trying to help educational programs get better at what they do. The reoccurring concern was that evaluations were difficult without centralized support to provide some sort of structure.

“The interest really came when there were several offices and faculty within the college who were kind of all doing the same thing in terms of research,” said Ortiz, “so the idea was that if we came together and thought about it as a resource center then we could train grad assistants to do different parts of the research process so we weren’t all training our own grad assistants and repeating the work over and over.”

Standing in front of the center are (l-r) Alejandra Priede, Anna Ortiz and Don Haviland.
Standing in front of the center are (l-r) Alejandra Priede, Anna Ortiz and Don Haviland.

“We envision the center as a laboratory of practice,” said Haviland, “where we help clients understand and improve the effectiveness of their programs and services while facilitating student learning and preparing them for successful careers.”

Serving as the associate director of the center will be Alejandra Priede, a lecturer in the Educational Leadership Department in the College of Education. The staff will be made up of master’s, doctorial and undergraduate students and will constantly evolve depending on the workload, so in a sense it’s living organization which provides more flexibility.

The centralized approach will allow for the streamlining of some doctoral preparation in terms of a number of fellowship programs that in the past have been funded, for example, by the Irvine Foundation or Hewlett-Packard or Bechtel.

“When we receive those they are essentially doctoral students doing fellowships in evaluating programs that those foundations are funding and we’ve treated them all separately,” said Ortiz. “But now we can bring them all under this one umbrella and have a more streamlined approach.”

For those on campus who are putting together grants to fund programs it’s also very efficient for them, according to Ortiz.
“We’re working on a grant right now where the PIs (principal investigators) are writing the grant proposal and Don is working on the evaluation part of that proposal so they didn’t have to go find someone to be the internal evaluator for that grant because they came to this center,” said Ortiz. “It will make things easier for the folks on campus who are trying to get grants funded by the federal government or any number of other agencies. It makes their process easier, and that’s good because the process is hard.”

According to Haviland, in order to be successful, it’s helpful for an evaluator to know one simple thing—“What is this program trying to do?” That’s something he has learned over the years.

“You take these people and these resources and you have workshops and counseling and you have all these specific outcomes,” he said. “It might be retention, it might be student engagement or it might be student graduation. Whatever it is, having a really clear sense of what those goals are and how they go about achieving them is key for me as an evaluator.

“That’s why coming in on the front end is really important because I can ask the questions that everyone else may be taking for granted. Maybe they’ve been doing it for a year or even 10 years and as the evaluator I can come in and say, ‘Why are you doing this?’ That’s why coming in on the front end is so nice because then you get to ask those honest, but naïve questions right away. They get to rethink their design a little bit and you’ve got a better idea of the program you’re going to evaluate which in turn helps it become more successful which is really the outcome we all want.”