Friday, September 16, 2016

Some participant research for the campaign season

In this miserable election season, I have chosen to add myself to an online panel that generates polls. It's a British outfit with international reach called YouGov that presumably makes its money doing market research, but uses its panel of volunteer survey participants for election polling as well.

In conjunction with The Economist, YouGov's latest presidential poll finds Clinton maintains a narrow lead over Trump, putting their findings pretty much in line with the average of national polls this week.

I joined up because I am curious about the process of online polling. Online polling panels are designed to overcome the increasing difficulties of conducting traditional phone surveys. Hardly anyone answers their phones these days and even among those who do, 90 percent won't talk with a pollster. Pollsters are legally barred from making automated calls to cell phones so reaching the 50 percent who only use mobile phones is very expensive.

Online pollsters like YouGov try to attract large numbers of volunteers who will share their demographic and opinion data via internet quizzes. They offer prizes and some dubious privacy guarantees to participants who then make up their "panel". They use what they know about panel members to create samples of people who, when aggregated and weighted for various characteristics, will combine to form an accurate representation of the electorate. There's a lot of statistical mumbo-jumbo going on here.

YouGov's U.S. website is not very forthcoming about how this works, but the United Kingdom site explains their method a little.

YouGov conducts its public opinion surveys online using something called Active Sampling ...When using Active Sampling, restrictions are put in place to ensure that only the people contacted are allowed to participate. This means that all the respondents who complete YouGov surveys will have been selected by YouGov, from our panel of registered users, and only those who are selected from this panel are allowed to take part in the survey.

... When a new panel member is recruited, a host of socio-demographic information is recorded. For nationally representative samples, YouGov draws a sub-sample of the panel that is representative of British adults in terms of age, gender, social class and type of newspaper (upmarket, mid-market, red-top, no newspaper), and invites this sub-sample to complete a survey.

... Once the survey is complete, the final data are then statistically weighted to the national profile of all adults aged 18+ (including people without internet access).  All reputable research agencies weight data as a fine-tuning measure and at YouGov we weight by age, gender, social class, region, level of education, how respondents voted at the previous election and level of political interest.

Presumably they are doing something similar in the U.S. I wish they'd share what markers they use for "social class" here as well as how they segment our news sources. Nate Silver's 538.com gives YouGov's product a B grade and comments "we’re awaiting more evidence about the reliability of online polls."
***
So what's it like being part of a panel? If I hadn't decided to do this as a research exercise, I'd have probably unsubscribed out of boredom after the first couple of surveys I received. Mostly I am asked about media and products in which I have no interest. I do get some political questions, like one today about whether I'd approve of the Prez pardoning Edward Snowden.

When I do get such questions, I have to wonder: for what sort of sample would a company doing broad surveys need the views of an old, white, economically comfortable, liberal San Franciscan? Aren't I an open book demographically? But apparently sometimes they need such ones in the pool.
I am interested that they use the locution "Caucasian" to describe my race. Whoever writes this stuff needs to read The History of White People. That historical term is both nonsense and racist. The federal Census uses "white" and I think "white" is the name most people in the U.S. would understand. Maybe the anachronistic label is a British thing?

1 comment:

Hattie said...

The media and the pollsters seem to me to be missing something. They impose their simplistic patterns on the public in an effort to quantify our complexities.
It's a can of worms, as you know who said.