A few nights ago during my lesson I had to explain the word "ratings" to my students, and then one of them mentioned that she used to work for a teleresearch company that also did the ratings for television channels here in Greece. Standing there in front of them, hearing her talk about it, I pictured what the workplace must be like, and remembered my years working for a company I'll call Polls R Us. (The only reason I won't reveal who they are is that, after having worked here in Greece for the past decade, they seem to have been pretty decent employers after all.)
I started working there in late 1989, and worked there on and off for about five or six years. I became a supervisor in the phone bank, but even then found time to work on the phones. The phone bank had 30 computers which dialled the numbers distributed by a central computer. We used the cati system, if anybody knows of it. We wore headsets so that we could type in the open-ended answers. Some interviewers were horrible one-finger typests. Once when I was supervising from the central computer I monitored a survey where the interviewer mumbled, "Jesus, where's that 'r'?" The respondent said, "You're a slow typist, aren't you?" and he said, "It's been a while since I was in high school."
Pollsters prefer to stick to closed, multiple-choice questions. One of the reasons is that open-ended responses need to be coded afterward, a tiresome job when what the respondent said or interviewer typed up doesn't make much sense. They prefer the answers to fit into the pre-existing codes, even to the point of telling interviewers not to volunteer the information that there is an "Other: specify" option. I was never sure if the purpose of the survey was to provide the client (usually a politician) with what they wanted to be told or to publish the results and use them to help shape the public's perception of an issue. At any rate, it didn't seem they were very interested in what the people actually thought.
And really, I can't say I blamed them. If you want to get an idea of how stupid the average person is, all you need to do is call them up at home and ask them their opinions about political issues.
Here's a typical exchange:
Thomas: On a scale of one to ten, where one is not at all impressed and ten is very impressed, how impressed are you with the Politician A?
Thomas: From one to ten?
Respondent: Yeah, one to ten.
Thomas: Sorry, sir, could you pick one number from on to ten, where one is not at all impressed and ten is very impressed?
Respondent: Yeah, uh, six.
Thomas: All right, and on a scale of one to ten, where one is not at all impressed and ten is very impressed, how impressed are you with the Politician B?
Respondent: Pretty impressed, I'd say.
Thomas: From one to ten?
Respondent: Yeah, one to ten.
Another common type of question used to limit the possible responses to four.
Thomas: Would you say that over the past 12 months the state of the economy has improved significantly, improved somewhat, worsened somewhat, or worsened significantly?
Respondent: Sen – senificant.
Thomas: Significantly improved or significantly worsened?
Thomas: All right, and would you say that over the past 12 months health care has improved significantly, improved somewhat, worsened somewhat, or worsened significantly?
After a while the respondent got tired of trying to pronounce "significantly" and realised life would be a great deal easier for him if he chose "somewhat" instead, all the while answe ing with a seriousness that suggested that every response had been arrived at with scientific precision.
(I often wondered how many terrible politicians in Canada breathed a sigh of relief to discover that people only somewhat disapproved of their record, and all because the majority of them could not pronounce the word "significantly".)
Having to repeat each question two or three times meant that the survey I'd assured them would not take more than ten minutes was now going on for half an hour. Invariable the respondent would rouse himself from his question-answering stupor and ask, "How much longer?
It's all very fine, he was saying, to be able to take part in a democratic process — especially considering that so many governments throughout the world spend more time suppressing what people have to say — and to help shape government policy, but I'm trying to watch Wheel of Fortune over here.
Another thing I found particularly frustrating was that we were not allowed to explain anything to the respondent. The official response as, "I'm sorry, but I'm not allowed to interpret the question for you." Theoretically, this makes sense. I may interpret incorrectly, either through ignorance or through bias, and that particular question will no longer be the same one everyone else is answering. Plus, I don't know how many people answered it without understanding it. I only know which ones asked for clarification. But the alternative is pseudoscience. I tell them, "Answer the question according to what it means to you," and then we interpret your answer in whatever way is convenient for us.
We did a survey once about biogenetic engineering and I don't know how many people told me they were against it because they didn't want anyone putting chemicals in their food. And these people's responses carried as much weight as a biologist's.
In the early 90s, Canada was going through a tough time with its sense of national identity. The premiers of the provinces met at a resort on Meech Lake and tried to come up with a constitutional agreement that would satisfy Quebec's demands for "equal but separate" status. The debate divided the country and created a lot of bad feeling about Quebec, which most people in the rest of the country viewed as a spoiled brat. Meech Lake was in the news so much that people were absolutely sick of hearing about it. Eventually, the Meech Lake Accord, as it was called, collapsed, largely through the efforts of Elijah Harper, who came to be seen as a national hero by opponents of the Accord. One evening I asked a young gentleman, most of whose attention as on a hockey game, about the Meech Lake Accord, and he asked me:
I doubt if I knew then where exactly Meech Lake was. I don't remember now. That evening, however, I decided to break the rules and do a bit of interpreting, first pressing the key for "Don't know/No answer".
"Oh, you mean like something on paper," he said. ""I don't know anything about that. I live near Lake Huron, eh?"
One of the funniest memories I have of my years at Polls R Us is of one of the very young high school students who worked on the phones. We were doing a survey for some organisation involved in raising awareness about psoriasis. Knowing that interviewers would have problems pronouncing it, the company wrote it out phonetically for them. One kid, about 15 or 16 years old, didn't find it helpful. I heard him ask:
"Do you or does anyone else in your family have sorry asses?"
One of my favourite TV shows when I was a young child was Mr Dressup. Ernie Coombs, who played Mr Dressup, had two puppets on the show, a boy named Casey, and a dog named Finnegan. A child once wrote a letter to Coombs and asked him, "Does Casey know he's only a puppet?"
If you try to get your head around the logic of that question, you'll get an idea of the sort of confusion I felt one evening when a respondent interrupted my introduction to ask, "Are you a computer?"
In fact, I was stunned. It took me a moment to answer.
"Oh, good! I hate talking to those things!"
Years later, the same thing happened again, but this time I was prepared. The respondent asked me:
"Is this one of those recordings?"
"Would you ask a recording a question, sir?"
"Yes I would!" he said, angry and embarrassed, and hung up.