Using Focus Groups for Assessment, Evaluation, and Improvement

Dr. Yuerong Sweetland
Tuesday, December 12, 2017
People viewing a wall of photographs of a diversity of people.

A focus group, as Krueger (2015) defined, is “a carefully planned series of discussions designed to obtain perceptions on a defined area of interest in a permissive, non-threatening environment.” A focus group approach, as a qualitative method, can provide rich information on perceptions, feelings, and reactions through participant interactions resembling the dynamics of real life in a comfortable environment. In particular, it can be used to uncover mysteries behind vague or unexpected responses by probing into the “how” or “why” of participant behavior and thinking.

Used extensively in marketing research, focus groups have been gaining popularity among educational researchers and evaluation professionals. In higher education settings, focus groups can be used for assessing institutional level initiatives as well as course- and program-level experiences for students. For example, I recently collaborated with two Franklin University colleagues, Dean of Students Dr. Lynne Hull and Coordinator of Student Affairs and Enrollment Management Ms. Elizabeth Skursha, to conduct a focus group study. The goal of our study was to explore student experiences with the various initiatives at Franklin and how best to continually improve the services we offer to students.

Our team hosted five focus groups comprised of undergraduate, master’s, and doctoral students to gauge students’ perspectives on various campus initiatives — what they view as helpful and what improvements they would like to see. The focus group method is only one prong of a triangulated approach we use to examine the impact of University initiatives aimed at improving services and identifying areas for improvement. The other two prongs are a student survey and feedback collected from faculty and staff. Based on the data we have collected and analyzed, we have already seen some converging themes among the different prongs, which speaks to the reliability of our findings and which will add confidence to the decision making process.

To conduct focus groups for program evaluation and improvement in higher education settings, not only can students be invited to participate, but other stakeholders may also be involved, such as faculty, staff members, and individuals external to the University. For example, the Indiana University School of Education conducted focus groups comprised of a large range of stakeholders for their program evaluation, which included teachers, principals, higher education personnel, public, parents, and state organization leaders (Indiana University, n.d.).

Given its power to uncover mysteries behind unclear responses or data patterns, a focus group may be used to further assess and better understand student learning. In higher education settings, faculty and administrators need to assess student learning on the specific learning outcomes. Assessment artifacts such as course projects or tests provide direct evidence of student achievement. However, these approaches do not necessarily provide the “how” and “why” of student learning. To fill this gap, we may use a focus group to probe directly into course activities in light of designated learning outcomes. For example, student participants may be asked what activities or assignments best help them learn how to “use samples to construct confidence internals” (a statistics course outcome) and why. This kind of assessment activity engages students into reflections of their learning processes and the intended learning outcomes, thus truly integrating learning and assessment and yielding actionable information.

In spite of the various advantages of the focus group approach, decision makers are sometimes leery about acting on focus group findings, especially if resulting actions may have broad impacts. This explains why focus groups are often combined with other approaches, such as surveys, in program evaluation. There are different ways of pairing up focus groups and surveys for gathering information to answer research or evaluation questions. Sometimes, a survey may be conducted after the focus group stage to help confirm findings and establish generalizability, which is what the Indiana University School of Education did following their focus groups for their program evaluation (Indiana University, n.d.). In other cases, initial surveys may be conducted to uncover trends or themes and are then followed with focus groups to further explore the “how” and “why.” Ultimately, the approach to choose depends on the specific contexts of the evaluation or research.

Here is a story that illustrates why we may need focus groups to help interpret survey findings: An institution conducted a survey that indicated students were interested in taking certain foreign language classes. However, after the institution had expended the effort and resources to offer the classes in question, very few students registered, causing the program to fail. What might have been the reason for this seemingly inconsistent behavior? The survey procedure or instrument might have had flaws, leading to invalid findings. In addition to carefully re-examining the survey itself, the institution could have conducted focus groups following the survey to further explore student responses, before significant resources were spent to develop the program in question.

Blog Category: 
Assessment

Comments

8

A very insightful article, Dr. Sweetland! Focus group is indeed a great qualitative research method. As we have more and more doctoral students in our programs, we need to advise them to use this method to collect deeper and unspoken data.

Thank you for sharing your thoughts, Dr. Yang. I can't agree with you more. Yes, it can be a powerful yet inexpensive tool. I would strongly recommend it, especially for students who want to conduct action research. Researchers will just need to be cautious in interpreting the findings and making any generalizability statements. Yuerong

Good thinking, Yi. These kinds of focus groups can help us keep a finger on the pulse of student experiences and needs. I would say that this kind of focus group can also be used to monitor the experience of the faculty and staff in supporting programs in higher education, as well.

Great article, Yuerong. I was wondering - how often do you recommend conducting focus groups/surveys for program evaluation and improvement? Is it something that is beneficial to do annually, or should it be less often, like every 5 years?

Good question, Carolyn! The short answer is that it depends on the questions you want to answer. If it is a regular program evaluation that is conducted every three years, you might just conduct your focus groups every three years to correspond your program evaluation cycle. If you would like to examine whether participants' perceptions or experiences have changed throughout a program/initiative, you might conduct several rounds of focus groups throughout the initiative, at the beginning, middle, and end of the entire program. With surveys, we often have to be cautious about survey fatigue, since survey is very inexpensive and can generate numbers/patterns rather quickly and thus used extensively. It is less of an issue with focus groups, mainly because it involves smaller groups interacting in a comfortable environment. However, it can be time-consuming to analyze focus group data and requires caution in interpreting focus group data. I hope this helps!

Thanks, Yuerong! You've inspired me to consider a focus group as part of the research Dr. Constance Wanstreet and I will be doing about first year experience programs for adult learners.

Glad to hear that, NICHER! Focus groups will be a great tool to explore students' first-year experiences and how they transition into college for adult learners, as well as for traditional age students. Good luck. Let me know if you need any help. Yuerong

Here is a great one-sentence quote that highlight's Dr. Sweetland's thoughtfulness in her evaluation work: "Ultimately, the approach to choose depends on the specific contexts of the evaluation or research." I appreciate Dr. Sweetland's suggestions on how focus groups might be used effectively, especially in conjunction with other data collection methods.

About the Author