Some important analysis

Having done this kind of work professionally for years, I'd been wondering about this particular problem. Good to know the Pew folks are paying attention to it. The article does a great job of owning up to the vulnerabilities of this kind of research, and once more proves that just having facts based on a large sample doesn't guarantee their accuracy unless the sample is truly representative of the population you wish to model.

And in the age of folks who no longer use landlines, or barely use landlines, phone surveys are starting to miss key groups:

As implied by these results, in each of the three polls, the cell-only respondents were significantly more supportive of Obama (by 10-to-15 percentage points) than respondents in the landline sample. For example, in the September survey Obama led McCain by a 55%-to-36% margin among cell only voters, but the candidates were tied at 45% in the landline sample.

In large part, this reflects the fact that a substantial minority of the cell-only sample is younger than 30 - a demographic group that has consistently backed Obama this year. Traditional landline surveys are typically weighted to compensate for age and other demographic differences, but the process depends on the assumption that the people reached over landlines are similar politically to their cell-only counterparts. These surveys suggest that this assumption is increasingly questionable, particularly among younger people.

So a critical question for any survey these days is: How'd you deal with the problem of young people who are cell only? Younger folks are hard enough to get on the phone, let alone when they don't use land lines. Merely counting the land line folks more heavily doesn't make up for it. If the dimension of bias in your sample is correlated with the outcome you wish to measure, you're pretty screwed. Can't do a land-line survey to ask people about why they gave up land lines. Can't do a web survey asking why people aren't online. Would be similarly bad to do web survey about anything having to do with old people. Or users of Video Professor. "I'm not answerin' yer questions! You'll steal my identity! I just want to see pictures of my grandkids on the email."

What's the solution? Probably working to build a panel of younger people. Probably start by doing random intercepts on the street, in various locations (places of work, common commerce, schools) and do a survey about communication habits preferences. Be sure these results have the right demo balance. Then figure out what proportion of the population is cell only/ etc. Then do some follow up work to figure out differential response rates for the various groups. Contact them in their preferred mode: Text them? Web-survey? Call on cell/ land? Finally, do studies weighting not only for demographics but preferred mode of communication, to be sure you've got the right proportion of folks.

It's my impression that most of the political polling being done and publicized is largely based on old RDD methods with some demo weighting on the back end. These are going to miss the Obama vote in an important way.