Campus / News / February 21, 2018

Surveys provide data but have issues

Graphic by Michelle Dudley.

Knox staff say surveys have their limitations and should only be applied to solve specific problems, but are frequently used as an essential tool for making decisions.

“Surveys are a little bit of an art and a little bit of a science,” said Charles Clark, the Chief Institutional Research Officer.

According to Clark, surveys are among the best tools for the college to ensure that a decision will benefit as many people as possible. Clark suggests that the best time to use surveys is when an issue is surrounded by many personal opinions.

“One thing I like to say about our office is that we are the antidote to the anecdote,” Clark said. “You never want to make a decision based on an anecdote. What you want is evidence.”

But there are many ways for a survey to fail, according to Clark, especially in the way questions are worded. It is important not to lead respondents into answering in any particular way. Questions should not have what Clark calls an “agenda.”

“Anyone can write a question, but will it . . . illicit a genuine response from the respondents?” Clark said. “It’s all about the structuring of the question. You want to leave out bias.”

Another way Clark mentioned how surveys fail is through respondents finding them tedious. This factor is related primarily to a survey’s length. As a result, repeatedly filling out demographic information has been eliminated in order to increase survey response rates.

“A lot of times when you get a survey from us we don’t ask about your class or anything,” Clark said. “We just ask you for your student ID, then we connect that with all of your information.”

This ability relies on what Clark views as an important distinction between anonymity and confidentiality. Every survey is confidential, meaning that the results are protected from outside individuals. However, unless the survey specifically states it is anonymous, the Office of Institutional Research is able to obtain identifying information about each respondent.

Marketing of the survey also plays a key role in participation.

“We typically run a response rate in the 30 to 40 percent range, which is a fair response range,” Clark said. “It’s about the same, maybe a little better, than other institutions like ours.”

Assistant Professor of Environmental Science Ben Farrer and the Student Life Committee (SLC) recently used a survey to find that 45.3 percent of students do not have “reliable access to affordable, nutritious food.” As a social scientist, Farrer believed that creating a survey was the ideal tool for finding the prevalence of food insecurity at Knox.

“Whenever you have a question that is a ‘how much?’ question . . . you want a quantitative method to answer that question,” Farrer said. “My approach was: this is a quantitative question, so let’s use a quantitative tool. Surveys are probably the best ones that we have.”

Farrer contrasted using surveys with other quantitative tools such as observing eating habits and comparing them to an objective measure of food insecurity, which he felt would be impractical. Farrer similarly decided to avoid tools that measure the severity of an issue, such as focus groups, because they are not quantitative.

Farrer also mentioned that the infrastructure for creating surveys at Knox makes using them simple.

“There were a few logistical reasons for using a survey along with the nature of the question,” Farrer said. “We used this software called Qualtrics which is an online platform for survey design. It is widely used in the field. All the ways you could ask a question, that’s coded into the program.”

Chief Information Officer Steve Hall frequently utilizes surveys to understand how the decisions he makes are viewed by the student population. After he decided to close Founders Lab from 2 to 7 a.m. – making it no longer open at all hours – Hall consulted with Sofia Tagkaloglou, president of Student Senate, who conducted a survey to gage student’s reactions.

“I’m a lightning-rod for issues,” Hall said. “So if I hear, all day, complaints about some particular problem . . . and I hear about it from 10 people for 20 minutes apiece, then I’ve spent all day and I think this is a huge problem Ñ but in reality I’ve only talked to 10 people. What I want to do is use a survey to see if it is a broadly held opinion or if it’s really just a minority of people.”

Hall made the decision to close Founders Lab at night because of statistical data showing that only three to five students used it during those hours. Around 350 of the approximately 1,350 enrolled students responded. As a result of the survey, three computers and a printer were placed inside of the always-open Sandburg Lounge.

“If you want to send surveys so you can really collect data, you should learn how to do it,” Hall said. “Thinking that just because you have access to a tool, you’re now ready to conduct a survey is a mistake . . . There’s a huge difference between good surveys and bad surveys.”

Tags:  campus climate survey social science Steve Hall Student Life Committee surveys

Bookmark and Share

Previous Post
Exec debates draw sparse student audience
Next Post
Thoughts from the Embers: Vote Monterey for Student Senate President!

You might also like


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.