Things To Keep In Mind About Polling

I wanted to pass along a couple more articles about polling before we collectively begin to follow polls on an hourly basis.  First, from the IBD/TIPP poll (remember, this is the most accurate poll from 2004):

The race tightened again Sunday as independents who'd been leaning to Obama shifted to McCain to leave that key group a toss-up. McCain also pulled even in the Midwest, moved back into the lead with men, padded his gains among Protestants and Catholics, and is favored for the first time by high school graduates.

The margin is around 2%, but almost 9% are still undecided.  The momentum and the current breaking trends show most of those going toward McCain.  Newsmax has some more analysis:

In terms of age group, McCain still is virtually tied with Obama with respondents in the categories between 25 years of age and 64. Some 9 percent are still undecided. He leads among voters 65 and over by 2 points, 45 percent to 43 percent. Obama has a commanding lead only among the young respondents, those 18 to 24. But that group's reliability on Election Day varies tremendously.

Among party faithful, the poll shows that McCain is holding onto Republicans by an overwhelming margin — he has 89 percent locked up — and is winning now among self-described independents, 45 to 43 percent.

McCain also has a 15-point lead over Obama among voters who earn at least $75,000 a year, and now holds a 54 percent to 40 percent edge among male voters, up from a 4-point lead just several weeks ago.

The good thing about this is that Obama's socialist message appears to have gotten out, and the successful high earners in this country are abandoning Obama in droves.

Again: don't...listen...to...the...media...hype...!!!  This is going to come down to the wire, and to exactly how those undecided voters break.

Next, let's discuss the issue of exit polls.  They are hideously inaccurate because they oversample Democrats.  Unfortunately, they are also used by the media to trumpet their candidates (this year, it's the Obamessiah) winning long before the polls even close.  We have seen it in every major election for years.  Hot Air breaks it down for us:

Per Geraghty, 77 percent of Obama's supporters say they're willing to spend 10 minutes answering an exit poll questionnaire versus just 64 percent of McCain's. No matter what's happening on the ground, then, expect The One to have a large-ish lead when the data finally drops at 5 p.m. tomorrow.

Fortunately, it appears that some minimal steps are being taken to prevent the frenzy of 2000 and 2004:

[T]he networks now follow strict rules that govern projections, examining not only exit poll data but actual vote tabulation and turnout information. NBC — which keeps its decision desk isolated from the calls made by competing networks — will only call a winner once its statisticians conclude that the chance of an error is less than 1 in 200. And no calls will be made until all the polls have closed in a state…

[E]xit polls are designed to provide a demographic portrait of voters, not to predict the winner of a close race. The early waves of data can be especially misleading because they do not necessarily reflect an accurate sample of the electorate.

This year, there's another factor to take into account: the huge surge in early voting. To get a measurement of those voters, Edison/Mitofsky has been doing an extensive telephone survey in the last week, focusing on the 18 states with the highest concentration of absentee and early voting.

For all the precautions, the networks' decision desks will still be analyzing the exit poll with some wariness. In both 2004 and 2006, the polls overstated the Democratic vote in many states. Because of that, Fox News stopped relying on the data halfway through election night in 2006, declaring it unreliable.

A similar problem cropped up during this year's primaries, when exit polls often showed a greater lead for Sen. Barack Obama than the actual vote, decision desk directors said.

So, honestly, how accurate are those exit polls?  Iowahawk has the goods:

Statisticians love balls and urns. A typical Stats 101 midterm, for example, usually includes a question along these lines:

"You take a simple random sample of 1000 balls from an urn containing 120,000,000 red and blue balls, and your sample shows 450 red balls and 550 blue balls. Construct a 95% confidence interval for the true proportion of blue balls in the urn."

...whip out your calculator and text your frat brother who has a copy of last semester's midterm. He instantly recognizes the correct formula is

95% confidence interval for P = p +/- 1.96 * sqrt( p*(1-p) / n) * FPC

That second part, after the "+/-", is what you know as the "margin of error." Your frat brother texts you back and reminds you that since the population is very large, the FPC is very close to 1 and can be dropped. He also reminds you to uses the conservative estimate of p = 0.5 in the margin of error calculation, since you don't know the true value of p, only the sample estimate. So the whole formula simplifies to

p +/- 1.96 * sqrt( .25 / n)

=p +/-  0.98  / sqrt( n)

Assuming you still have juice in your calculator batteries and you're not hungover from the Sig Eps kegger last night, you should get

0.55 +/- 0.031

To cut to the chase, he goes through some contortions to play out the math.  Here's where it gets interesting:

Works pretty well if you're interested in hypothetical colored balls in hypothetical giant urns, or survival rates of plants in a controlled experiment, or defects in a batch of factory products. It may even work well if you're interested in blind cola taste tests. But what if the thing you are studying doesn't quite fit the balls & urns template?

• What if 40% of the balls have personally chosen to live in an urn that you legally can't stick your hand into?
• What if 50% of the balls who live in the legal urn explicitly refuse to let you select them?
• What if the balls inside the urn are constantly interacting and talking and arguing with each other, and can decide to change their color on a whim?
• What if you have to rely on the balls to report their own color, and some unknown number are probably lying to you?
• What if you've been hired to count balls by a company who has endorsed blue as their favorite color? 
• What if you have outsourced the urn-ball counting to part-time temp balls, most of whom happen to be blue?
• What if the balls inside the urn are listening to you counting out there, and it affects whether they want to be counted, and/or which color they want to be?

If one or more of the above statements are true, then the formula for margin of error simplifies to

Margin of Error = Who the hell knows?

Exactly.  It would be wise to ignore exit polls completely.  Still, that's what all the reporting will be based on, so what can we expect to see?  We will certainly see leads for Obama, precisely because of the reasons stated above.  If we don't (which is very, very unlikely), it probably means a big McCain win.  For a schedule of when the first exit poll results are allowed to be released, go here.  Some analysis of the must-wins that you can be watching for combined with the timeframes of exit polls:

Play around with CNN's electoral map and you'll see what I mean. Give [Obama] every last state from 9 p.m. on, then try to figure out how many pre-9 p.m. states he can lose and still reach 270. Unless I missed something, it boils down to this:

1. He can afford to lose any single state above the line, no matter which it is, so long as he wins all the others.

2. He can afford to lose any combination of Indiana, Virginia, and Missouri (but not all three), so long as he wins every other state above the line.

3. Under no circumstances can he afford to lose a combination of Pennsylvania, Ohio, and Virginia. Two out of three is a must, plus every other state above the line. McCain currently trails in all three by anywhere from four to seven points, per the RCP state averages. (Fun fact: Given my assumptions, if Maverick wins Pennsylvania and Virginia, Obama wins Ohio, and they split Indiana and Missouri, it's 269-269.)

What this means, given The One's lead, is that each hour on Tuesday night operates almost like an elimination game. If McCain wins Indiana and Virginia, he survives and advances to the next round against Ohio and North Carolina. If he wins both of those, he moves on to the eastern regional championship in Pennsylvania and Florida. And if he wins both of those, he heads west for the Final Four Six. As I say, he can technically afford to lose one or two states early (depending upon which ones they are), but the bellwether omens will be grim if he's dropped any battlegrounds before 8 p.m. And again, to win the election he'd almost certainly then have to win all six of those western states. If you believe PPP, not only does The One lead big in Colorado and New Mexico, but a majority of voters have already cast their ballots. Assuming that's true, McCain's margin of error above the line is virtually zero.

Needless to say, if both Indiana and Virginia are called for Obama, commence drinking immediately.

Interesting analysis.  I like how they put the timelines on it, too.  Should make for some interesting viewing Tuesday night.  But don't stress.  Don't let anything keep you from casting your vote, even if you hear about exit polls indicating the state is done.  JUST DON'T LISTEN!!!  More Democrats take those exit polls than Republicans, and the variables involved in calculating those exit polls are so vast that it is impossible to truly estimate with any reasonable accuracy.

If nothing else, remember what has happened in both 2000 and 2004 - exit polls showed Bush losing to both Gore and Kerry (and don't give me any lip about 2000 being stolen - that is a myth perpetuated by the Democrat Left, and is full of crap).  Listen to Scott Rasmussen of Rasmussen Reports:

The bottom line is that in every state we polled--Colorado, Florida, Missouri, North Carolina, Ohio and Virginia-- Democrats are a lot more eager to take exit polls than Republicans.

In five of the six states, a majority of Democrats say they would be Very Likely to participate in the exit polling process. At the same time, in five of the six states, fewer than 40% of Republicans would be willing to do the same.

In every state, Republicans are at least twice as likely as Democrats to say that they are not at all willing to take an exit poll.

Unaffiliated voters tend to align more closely to Republicans in all six states in both willingness and unwillingness to participate in exit polls.

Exit polls simply don't reflect actual votes.  But that's not all we have to deal with this time around:

Then there's the issue of early voting, which is expected to reach unprecedented levels this year. Nationwide, Rasmussen Reports polling suggests as many as 37% of voters may cast their ballot before Tuesday.

In four of the battleground states, among those who had voted by last Sunday, Obama voters outnumber McCain voters by 20-plus points. In Missouri, early voters are evenly divided between the two candidates, while in Colorado Obama voters best McCain voters by "only" 15 points.

In Florida, which is typical of the others, 62% of those who have already voted cast their ballots for the Democrat, while 38% voted for McCain. In Virginia, 35% of early voters opted for the Republican, but 65% voted for Obama.

In part to compensate for this, exit polling has already begun in the 18 states where early voting is allowed. On the basis of our data from the battleground states, however, the responses appear likely to be weighted heavily in Obama's favor since his voters far outnumber McCain's. And, remember, Democrats are more willing to play the exit poll game than Republicans.

So, the exit polls will certainly be favorable toward Obama.  Not only are Democrats much more likely to participate in the exit polls, but more Obama supporters have already cast their vote.

From the McCain campaign:

Its bottom line: 

Based on the previous [years'] exit poll results, we should expect once again that Tuesday's exit poll data could overstate the Obama vote and under represent the McCain vote. It is important that the campaign make sure the media realizes this, so that when the exit polls do leak, people do not overreact to the early exit poll data. 

And the memo's five "key points":

1. Historically, exit polls have tended to overstate the Democratic vote.

2. The exit polls are likely to overstate the Obama vote because Obama voters are more likely to participate in the exit poll.

3. The exit polls have tended to skew most Democratic in years where there is high turnout and high vote interest like in 1992 and 2004.

4. It is not just the national exit poll that skews Democratic, but each of the state exit polls also suffers from the same Democratic leanings.

5. The results of the exit polls are also influenced by the demographics of the voters who conduct the exit polls.

According to the memo, the exit polls in 2004 had John Kerry "performing 5.5 net points better than the actual results showed him to have done."

Bottom line: ignore the exit polls altogether.  Keep your chin up, and be optimistic.  We'll find out Wednesday morning how things turned out (except for all the inevitable lawsuits, of course).

There's my two cents.