Baseball Crank
Covering the Front and Back Pages of the Newspaper
November 14, 2012
POLITICS: Sometimes, It Really Is Different This Time - A Polling Post-Mortem (Part I of III)

As promised, a mea culpa on my pre-election poll analysis: why I was wrong, why the state poll averages were right - and why I'd say most of the same things if I had to do it over. I suppose I have lost a good deal of credibility with a number of people by making the kind of out-on-a-limb prediction I don't usually make, and being wrong. But my assumptions have always been out in the open. Let's examine why they led to the wrong answer, and which of those assumptions should be re-evaluated in the future.

I. More Evidence Is Better Than Less

Discussions of polling often lend themselves to more heat than light. A lot of the post-election poll commentary is even dumber than the pre-election poll commentary, as victorious liberals spin a narrative that conservatives were all "poll deniers" or "poll truthers" ignoring the polls. Now, it's true that there were more than a few people on the Right who made intelligent discussion of the polls harder rather than easier. It's also true that some of the efforts at "unskewing" the polls were unhelpfully ham-handed; in this pre-election essay I explained why Dean Chambers' unskewedpolls.com was more alchemy than science. Chambers was projecting a strong Romney lead in the polls back in September, when neither I nor almost anybody I knew believed that Romney was actually ahead. My mantra on Twitter in August and September, like that of many conservatives I respect, was simply that it was a close race, that Obama had problems with independent voters and hadn't closed the deal yet, and that there was still time for Romney to catch up to him. While a lot of Romney's problems were baked-in by mid-summer, I don't think that was an unreasonable view to take at the time; as it turned out, there would be twists in the race throughout October. But those of us who attempted to take the polls seriously, and drew conclusions from the polling evidence itself as well as external evidence, were not denying anything; we were just looking under the hood.

A. Believe The Polls, But Don't Believe Only The Polls

Let me start by restating my philosophy of polling, and indeed my philosophy of examining most any question. Polls are not reality. They are a tool for measuring reality. They are traditionally the best single tool, and polling averages make them a better tool by evening out the outliers. But they are not perfect tools: public polls have called the outcome of races wrong before, or been off on the margins of victory by a significant amount. There are reasons why campaigns spend a lot of money on their own polls, which - other than for purposes like push-polling or testing messages - they would not do if an average of public polls was as flawless a guide to the electorate as a thermometer. If you think public poll averages are an infallible predictor, then the uniform practice of actual campaigns is totally irrational. Indeed, listen to Jim Messina, the nuts-and-bolts guru of the Obama campaign, on his view of public polling:

Every night, Obama's analytics team would run the campaign 66,000 times on a computer simulation. "And every morning we would come in and spend our money based on those simulations," said Messina.

Their models ultimately predicted Florida results within 0.2%, and 0.4% in Ohio. The only state they got wrong, noted Messina, was Colorado, "where we got one more point than we thought we would."

The Obama campaign was able to do that, he said, because they turned away from mainstream polling from shops like Gallup, which he called "wrong the entire election," in their prediction that fewer minorities and fewer young people would turn out to vote.

"We spent a whole bunch of time figuring out that American polling is broken," said Messina. "We never did a national poll. We only did local and state polls."

Is Messina a "poll truther" or "poll denier" for saying that "American polling is broken" or rejecting "traditional" polls? Was he wasting Obama's money by running his own state and local polls instead of just reading 538? Or is he reflecting the fact that public polls involve a certain amount of guesswork about voter turnout that can only be definitively tested each new election cycle by the final vote tallies?

The most recent set of publicly available polls are also not the only tool for measuring reality. There are all sorts of metrics - some more hard and quantifiable than others - that have traditionally been useful in assessing the state of play: voter-registration numbers, early voting data and absentee ballot data, trendlines in the polls, what the pollsters themselves are reporting about voter enthusiasm, and hazier indicators like small-dollar donations and the size of crowds on the campaign trail. It's hardly anti-empirical to examine these additional facts, and allow them to affect your conclusions. The people who correctly predicted that Harry Reid would hang on to beat Sharron Angle when the final polls said otherwise were looking at this kind of information. If all you did was read the RCP average or 538, you would have missed that.

None of these are reasons to disregard the polls; they should still be the primary item of evidence. But neither does it make sense to treat them with blind reverence and ignore all the external evidence - especially when (1) different sets of polls are in conflict with each other and (2) the internal breakdowns of the polls, which can act as a sanity check on the plausibility of the polls' assumptions, are telling you that the polls in question are predicting something that is historically unlikely to happen.

To go back to one more example from baseball analysis (sabermetrics), this is sort of like the "statheads vs scouts" debate: whether you can better project a player's future accomplishments by looking at his statistics or by listening to experts analyze his "tools." I'm in the stathead camp: I believe strongly in quantifiable data, and I place it at the center of just about any baseball analysis I do. In fact, I probably err more often than not on overemphasizing the numbers. But as Bill James, the father of sabermetrics, has often said, sabermetrics (or any other field of hard science, soft science or would-be science) is not the search for statistics, it's the search for truth, and in searching for the truth, you don't blind yourself to some of the available evidence simply because it doesn't fit in a column of your model. Most statheads today will tell you that no matter how good your statistical measurements and how highly you prioritize them, you can learn things from also listening to the scouts that you might not learn anywhere else. The scouts can't tell you a guy with a terrible stat line is actually a major star - but in a close case, their view can make the difference.

B. Regrets, I Have A Few

If I had it to do over, maybe I'd be less definitive in pronouncing Obama "toast," but for the most part I'd look at the same evidence, see the same things and draw the same conclusions I drew at the time. I do, however, have two main regrets.

First, which is an unfortunate side effect of doing punditry as a part-time unpaid hobby rather than a full time job, is that I should probably have spent more time breaking down the internals of all the battleground-state polls to see exactly how geographically consistent the national trends were: I had seen those trends confirmed in Ohio and a couple of the other key battleground states, and didn't do the deep, time-consuming dive into the internals of every single battleground state. As it turned out, Romney did very well with independent voters in a number of the battleground states, but critically underachieved with them in others and lost those states. Romney beat Obama with independents by 5 points or more in Ohio (+10), Virginia (+10), North Carolina (+15), Nevada (+8), and Pennsylvania (+5). In more lightly-contested states, he won them in Missouri (+26), Indiana (+11), New Mexico (+8) and Arizona (+6). But in two other battleground states, his margin with independents was narrow: Colorado (+4), and Michigan (+1). And he actually lost independents in Iowa (-14), New Hampshire (-7), Florida (-3), Wisconsin (-1), and Minnesota (-4). (It says something about Romney that his biggest failure with independent voters, especially in states with lily-white electorates that didn't present the demographic challenges apparent elsewhere, came in the places where the voters had been exposed at length to his scorched-earth 2008 and 2012 primary campaigns). In non-battleground states where exit polls were taken, independents mostly trended pro-Romney in red states - Alabama (+52), Mississippi (+34), Montana (+7) - and pro-Obama in blue states - California (-12), New York (-4), Illinois (-5), Vermont (-40), Maine (-28), Maryland (-12), Massachusetts (-7), Oregon (-7), Washington (-3), Connecticut (-3). In just two totally non-competitive states did independents buck the state's partisan tilt: Romney won independents in New Jersey (+1), and Obama tied him in Kansas. If Romney had won all the states where he carried independents, he would have won, but not in all the states I thought he would win:

map.indys.JPG

I might have caught some of this with more time to commit to each state, although in some cases, that weakness came out only in the very late polls. Or not; for example, PPP had Romney up by 2 with independents in New Hampshire on October 28, and up by 7 with independents in Florida the same day.

Second, and far more importantly, I didn't do enough to re-evaluate my conclusion after declaring it on Twitter on October 19 and laying it out in detail on October 26, eleven days before Election Day - and that week turned out to be Romney's high water-mark in the national and state polls. Nationally, Romney led in the RCP average all but two days between October 9 and October 31, and Obama was below 48% in the average every day from October 8 to November 5. In four key states, Romney had seized the lead in the RCP average: in Colorado, he led from October 9-29; in New Hampshire, he led from October 19-21; in Virginia, from October 19 to November 2; in Florida, from October 8 to the end of the race. Those leads gave me confidence that Romney's momentum in the national polls was real and would leave him needing to pick off just one more state to win. Instead, he lost all four, and the state poll averages were predicting three of those four losses by Election Day. When I wrote my initial post, I assumed (given the lateness in the cycle and the evident deterioration of Obama's position in the national polls and with independent voters by October 26), that there was not time for any new game-changing events. I was wrong.

I try hard to avoid confirmation bias in evaluating the evidence before reaching a conclusion - but it can be a lot harder to avoid it after publicly committing to a conclusion, which of course is the exact same "painted in a corner" problem I suspected Nate Silver of having. I spent the week after my "toast" post without electricity, internet, heat or hot water and only sporadically able to get to my office (I had written most of this essay when the lights went out on October 29), and probably did not spend enough of my remaining time going back over the subsequent polling to reconsider whether the conditions that seemed to show Obama in an untenable position were abating. (By Election Day, for example, the national poll averages were running in Obama's favor as well). I remain doubtful that Hurricane Sandy had enough impact to swing the election and particularly doubtful that it did much to affect turnout, which decided the election; but its interruption of the dynamic of the endgame probably did do what the final week state and national polls showed happening: cut into Romney's lead with independents and possibly cost him his chance to win New Hampshire, Virginia, and/or Florida. Exit polls showed that late-deciding voters broke in Obama's favor, which I had not expected to happen for an incumbent whose approval ratings and favorability with independent voters had been underwater for so much of his term.

II. Poll vs. Poll

The polls, particularly the national and swing-state polls in the third and fourth weeks of October, were telling consistent stories about the opinions of different groups of voters, but contradictory stories about the numbers in which those groups would show up to vote. I looked at internal evidence, and saw that Romney was winning independents handily and Obama was drawing nearly no crossover support from Republicans, which meant Obama needed to win entirely by having enough extra Democrats vote to overcome Romney's independent advantage. The first premise was borne out by Romney's win with independents and slight edge with crossovers, although his 5-point margin with independents was at least 3 points smaller than the margin most polls had been showing when I first made my "toast" call. Had Romney carried independents by 8 or 9, as the polls were showing at the time, and not lost them in a few key states, Obama would have needed at minimum a D+4 electorate to win nationally. As it turned out, Obama won by about 2 points with a D+6 electorate compared to 2008's D+7; Democratic turnout was down a point from 2008, but Republican turnout matched the 32% of 2008. So, my analysis correctly judged what Obama needed to do. The historic turnout conditions of 2008, which I believed were necessary for Obama to win, were effectively repeated.

There were all sorts of reasons, based in history and observable fact, to believe that those turnout conditions were unlikely to repeat themselves. Republican turnout had been unusually low in 2008 - the lowest since before Ronald Reagan started converting a lot of people to the party - following the historic financial crisis and 8 years of Bush, and had bounced back in 2010. While 2010 was a midterm election and involved a distinct electorate, its results underscored that there seemed no obvious reason to believe it would approach that nadir again. History tells us to treat with caution the assumption that elections held in the aftermath of a catastrophic event like the 2008 financial crisis are representative. The harder metrics, which I will discuss in Part III, also suggested GOP turnout was still doing well, if not as well as in the off-year elections of 2010. And the vaunted OFA turnout machine could only count for so much: certainly, no matter how sophisticated Obama's turnout operation, low GOP turnout was something he could not manufacture through operational efficiency. Obama had polled poorly in areas like job approval for most of his tenure, and campaigns that lose independent voters tend to be losing at least some of the enthusiasm of their base as well. As I will discuss below, virtually all the features of Obama's profile in the polls - mirrored in the post-election exit polls - were traditionally characteristic of losing candidates, as were the nature of the "this time it's different, really!" arguments made by his boosters. It had to be different for him to win. The customary laws of political gravity had to be defied. And they were.

As I'll get to more in Part II, with regard to turnout, what the exit polls show is that Obama really did do something that was very, very historically unusual - and on top of that, and perhaps even more importantly, Romney's GOTV operation (both the mechanics and his ability to inspire marginal Right-leaning voters to show up) turned out to be far less than projected. This time, it really was different. And the challenging question going forward is whether it will stay that way.

III. State vs. Nation

For the third consecutive presidential election, the RealClearPolitics averages of state polls correctly called nearly all the states correctly - 49 of 50 states (all but Wisconsin) in 2004, 48 of 50 (all but Indiana and North Carolina) in 2008, 49 of 50 (all but Florida) in 2012. (The four missed calls all involved states that broke late and narrowly to the Democrats). There was a lot of hoopla about Nate Silver's 538 model, and in my poll-reading pieces I noted - as have others - some quibbles with his model, with which I still have issues. But it's not clear that the methodological quirks of the 538 model matter much anyway; they've yet to deliver a different answer than the RCP average in a presidential contest or in any significant number of statewide elections. Ditto for other state poll averages, all of which were basically on the mark. The state poll averages implicitly projected an electorate with a large Democratic turnout advantage, and that's what happened.

That does not mean "the polls" were all correct. The national poll averages at both RCP and TPM had Romney ahead for much of October and ahead at the time I wrote my original "toast" post. This race was often compared to 2004 and ended up much the same, but in 2004, John Kerry never led in the RCP average in October. There were a variety of similar precedential indicators that Obama was in worse shape in the national polls than anybody who had gotten re-elected in the past. And traditionally, we have tended to see movements in the national polls followed by movements in the swing state polls - exactly what was reflected in the RCP average in Colorado, New Hampshire, Virginia and Florida. By October 29, Rasmussen's state-by-state polls were showing Romney leading in enough states to win the Electoral College. Of course, Rasmussen was wrong.

It is somewhat Orwellian for Obama supporters to paint as "poll deniers" those of us who looked at a conflict between two sets of polls containing two divergent sets of assumptions, and conclude that one of those sets of polls was more likely to be correct because external evidence rendered those assumptions more reasonable.

There was great debate over whether it mattered at all to look at the national polls, since presidential elections are determined on a state-by-state basis. This was really two questions in one: how likely was it that there would be a popular vote/electoral college split, and was it more likely that you could predict the national popular vote from state polls, or predict the battleground state vote from national polls. It is worth noting, on the first score, that the defenders of the state poll averages were only half right. On the one hand, they spent months arguing that Ohio in particular was a firewall that would remain favorable to Obama even if the national vote went slightly for Romney - and if Romney had hung onto his leads in Colorado, New Hampshire, Virginia and Florida, Ohio would have been enough to put him over the top. I thought the theory that Ohio would vote more Democratic than the nation was unlikely: Ohio has had an exceptionally consistent record from 1860 through 2008 of being just barely more Republican than the country as a whole nearly every election. I was right about that: Obama won Ohio by less than his national margin. As The New Republic's Nate Cohn notes:

[T]he exit polls show that Obama did worse among Ohio's white voters than John Kerry.... if there was anywhere that the president should have excelled due to the auto bailout, it would have been northeast Ohio. But the president lost northeastern Ohio's two classic white middle class bellwethers: Lake County, home to the overwhelmingly white suburbs and exurbs east of Cleveland, and Stark County, home to Canton. The president also lost additional ground in traditionally Democratic stretches of eastern Ohio, where Obama performed worse than any Democrat since McGovern in a stretch of "coal country" along the Ohio River. And Obama’s problems weren't limited to eastern Ohio. The president performed poorly in southwestern Ohio, including one deeply conservative and culturally southern county where Obama's performance was the worst by a Democrat since at least 1868.

But score one as well for the firewall theory as a whole: Obama could have lost Ohio and Florida (both of which voted more Republican than the nation) and Virginia (which voted more Democratic than the nation) and still won 272 electoral votes.

In theory, you should always look only at the state polls, since that's where the election is; if the state pollsters are as good at what they do and poll as regularly as the national pollsters, there would never be a reason to look at national polling. While we all look at national polls, especially early in the race when states are being polled less regularly, I mainly looked at state poll averages in 2004 and 2008. Which gets to the second point: how likely state polls were to offer a better picture of the national environment than national polls. I agreed with Sean Trende that the potential flaw in this theory was that many of the state pollsters at issue (other than pollsters like Rasmussen or PPP that poll both nationally and in the states) were relatively new to the game or had individually sketchy track records that couldn't be automatically substituted for the record of "state polls" as a whole.

As it turned out, though, the state pollsters were closer to the mark, and should be the main focus of our attention in future races, at least in the closing months. But why were they right when the national pollsters were wrong? For the answer to that, you need to look at how the pollsters decided what the electorate would look like. I will cover that topic tomorrow in Part II.

Posted by Baseball Crank at 11:00 AM | Politics 2012 • | Poll Analysis | Comments (21) | TrackBack (0)
Comments

I do not think Messina is a poll truther--but the fact is that Gallup was an outlier among the average of polls and could've been treated with some outside perspective skepticism in that regard. The problem, though, wasn't "Something is going on with state/national polls" (which is (a) true and (b) what people like Sean Trende said) but rather "Romney-Land-Slide! Unskew the troll polls! LOL Democrats!! Rasmussen and Gallup ONLY4EVAR!"--that was the problem.

Posted by: The Political Omnivore at November 14, 2012 11:22 AM

IMO, you are WAAAAAAAY overthinking this. The toplines of the polls were on average correct - both the state and national final polls had Obama winning. You were mistaken in looking at the internals - which are subgroups subject to greater degrees of error - and adjusting the toplines. In the future, look at the average of the toplines and don't place much weight on the internals. THIS is you major takeaway - everything else is minor.

Posted by: A.S. at November 14, 2012 12:19 PM

BTW - folks on the right really, REALLY need to stop just looking at Rasmussen. I know it feels warm and fuzzy to say "By October 29, Rasmussen's state-by-state polls were showing Romney leading in enough states to win the Electoral College." But the state polling *averages* never showed Romney leading in enough states. Rasmussen is just one pollster - who happened to be skewed right - he will not on balance be more accurate than the poll averages. Every post I saw at Power Line or The Corner saying "Rasmussen is showing Romney up by X in State Y" just made me want to scream. And it happened A LOT.

Posted by: A.S. at November 14, 2012 12:24 PM

I'm sorry to see that independents were -3 in CT. I was hoping independents in CT would have sent a message even if Obama won the state.

Posted by: MVH at November 14, 2012 12:48 PM

One lesson from this is throw away preconceived notions on how to apply polling data. Apply the data to the results not the outcome you hoped for, looking for flaws to validate your theories.
Even after so many professional and novice pollsters were wrong people continue to improperly look at the election results and maps. Things we have learned and are still learning is a few pollsters and pundits pretty much decided to live within a information bubble of comfort. As a result they provided false polling data using outdated and flawed methodologies to forecast the election. A few lessons, national polls don't matter in presidential elections. Once a candidate creates their road map to 272 that is what matters. To prove this point look at the state maps how the votes were cast. The trick going forward will be engaging the average voter to win them over, not the base or the super voter. also, this independent voter label fewer people are aligning with a party (the world is complex) you have to attack them on a case by case basis not as a block of independent voters. This is possible thanks to data mining to find voters where they were previously unknown. lastly, the GOP is way behind on injecting themselves with new voters, the last 8 years the Dems have owned the 18-22 new voter pool. next election cycle they will have have a majority of voters 22-32 pretty much locked up. Then if they get the next batch of new voters in 2016 that is 18-32.

Posted by: javaman at November 14, 2012 1:44 PM

MVH - I was surprised at how uniformly independents in the solid red & solid blue states tracked the state vote. I assume Kansas is different because the GOP has such a strong hold in that state (Kansas was essentially founded by an armed wing of the GOP) that most right-leaning indys are registered Rs.

AS - I get to more of this in Part II, but saying "trust the poll toplines" and "don't trust Rasmussen" are in some obvious tension. Rasmussen & Gallup were projecting different electorates - the mere fact that more pollsters were taking the opposite view did not necessarily make them more likely to be right. You had to look at how they were coming to those conclusions.

Rasmussen really made his name in 2004. His track record since then has been pretty good, not uniformly but he was predicting the GOP wave before most anybody else in 2010. He obviously missed the target this time. Anyway, my point in citing his state polls in that line was mainly to note that the favorable news for Romney at that juncture was not solely from nationwide polls.

Posted by: Crank at November 14, 2012 1:47 PM

Crank will probably cover this, but the real story here is that some of the polls that were predicting an Obama victory (PPP for example) were in fact using a "how do we get the result we want" approach. PPP has expressly admitted this. They just kept changing the electorate mix in order to get Obama's numbers high enough.

NOW: it turns out that these polls' electorate mixes ended up happening (because D turnout was higher than most objective people thought it would be and R turnout was lower), but that appears to be a coincidence. It wasn't because the pollsters had some some special scientific insight into what the mix would be. Had the mix been what Crank (wrongly but I think reasonably) predicted it would be, then PPP would have been wrong and Crank and other would have been right. (And yes, I realize that last statement kind of sounds like saying, "Well if the square had 3 sides, it would have been a triangle.")

It will probably takes months of study to sort out, but the lesson from all this might just very be, "just because you get the result your process predicted doesn't mean your process was right."

Posted by: mcgovern republican at November 14, 2012 2:04 PM

Yes, that point re PPP is definitely in Part II.

Posted by: Crank at November 14, 2012 2:07 PM

I hope that you will further examine the correlation between independents skewing Republican, and the R/D ratio...

I examined the final Ohio polls before the election, and it seemed to me that Obama either lost independents by 10%+ but still prevailed because there were fewer Republicans than in 2008. Other Ohio pollsters such as Rasmussen assumed an even D/R split but in that case Obama often narrowly won the independent vote! The one scenario that I did not find in any poll was Romney winning independents *and* high Republican turnout.

I agree with A.S.: trying to dismantle state polls because you do not like the bottom line is a recipe for disaster. The internal definition of "independent voter" varies from poll to poll, and it does seem as if a significant number of disgruntled former Republicans now call themselves independents.

To me, Obama's victory was largely a foregone conclusion. His job approval was at 49%+ from September onward according to RCP. Democrats approved of his job by 90%+ according to Gallup. So it made sense to assume that the hundreds of millions spent on GOTV plus early attack ads to "define" the challenger would work every bit as well as they did for Bush in 2004.

MARCU$

Posted by: mlindroo at November 14, 2012 2:14 PM

I don't agree that Crank was wrong to challenge the underly assumptions of those polls - it was a decent argument that the Dem turnout would not be as big as 2008.

I'm also not surprised to see a three-part essay. When you don't have cable tv or internet access, what else is there to do?

Posted by: MVH at November 14, 2012 3:07 PM

Crank - perhaps I was not clear, but the point is: trust the *average* of the poll toplines, not that we should trust any particular individual poll or pollster.

With respect to mcgovern republican's point about PPP, I don't think he accurately described what PPP has discussed. Regardless, the point is that all of the pollsters have their own "special sauce". It is impossible to know from one election to the next whose special sauce is right (perhaps none of them). But we should be reasonably confident that if we average together all the pollsters' special sauces, we get a more reasonable approximation of the truth than if we look at any of them individually.

BTW, I'll add that I think one of the (many) areas of politics in which the right is lagging significantly is polling technical expertise and analysis. The Left has Nate Silver, HuffPo Pollster, Josh Marshall's TPM polling reviews, academics like Alan Abramowitz and others. The right has... what? The only thing is probably RCP - which is simplistic and, I think, not as insightful as it should be. There are many, many technical areas that the right must improve on, but it is hard to believe that polling expertise and analysis is really something that we should be lagging in.

Posted by: A.S. at November 14, 2012 3:19 PM

I do not think TPM is really better than RCP, but of course all the people you list are full-time paid professionals. Besides Sean Trende at RCP, the rest of us are amateurs with day jobs. Therein lies the story of the Right's chronic problem in these kinds of online battles: a huge asymmetry in funding.

Posted by: Crank at November 14, 2012 3:51 PM

I think, perhaps, the biggest thing you overlooked in trying to reanalyze the polls is, as Nate Silver pointed out, for Romney to win virtually all the polling outfits out there would have had to be wrong.

The averages were right because the great majority of the polling outfits were closer to D+6 in their polling instead of D+2. And Rasmussen and PPP pretty much cancelled each other out.

So the thing which really perplexed me (as someone who looks at data all day) was how one could look at a data set in its entirety (with a pretty large n) and decide that the majority of those data points were simply wrong. Especially when those data sets (the averages) had historically been pretty good (at the state level).

Looking forward to the rest of this, but don't take too long - I need my R.A. Dickey celebratory column. :)

Posted by: Jay at November 14, 2012 10:16 PM

A.S. - you could be right. I'll try to find the story about PPP. My recollection is that Jensen basically said they just decided to assume that turnout would be the same in 12 as in 08 without any real reason. He called it "art" not "science." (?) My understanding is that Rasmussen basically did the opposite as PPP, but he at least seemed to have an objective reason to do it. But he was certainty wrong about prediction.

Posted by: mcgovern republican at November 15, 2012 8:47 AM

This is why I will continue coming here, even as a Phillies fan. Thanks for all the unpaid work, Crank.

Posted by: CJ at November 15, 2012 11:10 AM

MVH, the independents in CT did send a message, just not the one you were looking for.

Crank, do you ever actually sleep? Seriously, as deeply as I disagree with your political views and as critical as I am about your logic, I am constantly amazed at the depth and breadth of the analyses you undertake.

Posted by: Magrooder at November 15, 2012 11:21 AM

Appreciated. I have not slept much, or well, the past 2-3 months.

Posted by: Crank at November 15, 2012 11:31 AM

mcgovern - I think that's what pollsters are supposed to do - figure out how their polling approximates the beliefs of the public at large. I don't assume PPP made their polling choices because they produced polling showing Ds to be ahead, but rather what they saw in the polls they did caused them to believe what they did about the turnout. Seems to me to be an iterative process - do some polls and see what people are saying about turnout, then adjust your later assumptions as needed. But I'm not a pollster - what do I know?

Posted by: A.S. at November 15, 2012 12:24 PM

Crank - I agree that there is an asymmetry. I don't know enough to know if it is in funding, or that the right simply allocates funds to other things, or what. The campaign and politics infrastructure on the right certain is inferior to the left, and has been for a long time. The left surely has advantages - it controls the MSM and academia, but I would think there should be somesetting advantges for the right somewhere else.

That said, I think that these are the kind of things that we on the right ought to be focussing on now -- not the policy changes (e.g., immigration) that seem to be dominating the conversation about what the right need to do to improve election results. There have been many, many conversations on the right about what policies to advance (and which candidates are needed to advance them) - I've had a few of those conversations with you in your comments section. But I hear virtually nothing about how the right will improve its infrastructure. And it's not just on things like polling. We need to be competitive in the MSM. How? I don't know - maybe Karl Rove's billionaire friends should be buying the newspaper that owns Politifact, instead of giving money to Rove's failed advertising effort. Whatever it is, it needs to be different than the "more of the same" we're getting.

Oh, and as someone who is also a lawyer in Manhattan, same age as you, and also married with kids - I don't know how you do it either. I am similarly amazed at what you do on your blog.

Posted by: A.S. at November 15, 2012 12:44 PM

man, you spent a lot of words there saying "I know polls aren't perfect, so I decided to not bother doing the work necessary to analyze them properly, and instead let my personal biases pick and choose what numbers to look at instead".

Messina put the necessary work in to allow him to judge a specific poll as a clear outlier, you did not. Messina, in fact, put more work in to getting his numbers than that one poll did in the first place - he knew his numbers were better because his research was larger and more rigorous than Gallup's was. Yours was not.

Posted by: everdiso at November 15, 2012 4:30 PM

For the record, in your self-analysis you're still refusing to admit the two central mistakes you made.

1) You treated Party ID as a "hard" demographic instead of as "attitudinal" data. That was a fatal mistake, and one that most every poll skeptic made. You should NEVER have been looking at Party ID - you might as well have looked at the demographics of "conservatives/liberals/moderates" or similar attitudinal data. (actually, that one might have been better to look at, really).

What you should have been doing in an attempt to analyze turnout models - as Messina was - is looking at the HARD demographics - ethnicity, age, region, etc. THAT's where you look to analyze (and weight for) voter turnout.

This whole 'D+X' stuff was wrong from the very beginning, and any analysis which started from that point was bound to be wrong.

Looking at attitudinal data to weight or adjust polls is pseudo-science, and no better than what Chambers did. In baseball terms, using that data was like judging a prospect by his girlfriend's hotness, instead of his BB% and K%.


2) I know you try hard to dismiss Silver's work in the "State v. Nation" portion of your argument - but your entire argument rests on the crux of there being a significant disparity between the national and state polls - and even a lead for romney in the national polls.

Since you rely so much on that apparent "disparity", between the polls, it seems self-fulfilling to dismiss Silver's model.....when that model correctly showed that there was no significant disparity between the state and national polls, and moreover that Romney was NEVER leading at any point.

You can't argue based upon there being a significant disparity in state v. nat polls (or that romney took the lead at any point) and then while you're arguing it dismiss the fact that the most accurate prediction models did not show this disparity, or ever show Mitt in the lead.

Especially when you attempt to dismiss those more accurate models by referencing less accurate models like the RCP average.

3) You decided to favor national polls for NO GOOD REASON. Just because you wanted to. There was plenty of data to show you that state polls were far more accurate than national polls, but you chose to go with the national polls anyways.

Models such as 538 (or Messina's) didn't just randomly decide to favor state polls over national polls - they made that decision based on the fact that state polls had proven to be much more accurate than national polls. Messina, as you quoted, didn't even bother with national polls.

If you're going to go through all this effort to analyze your own mistakes, at least be honest with yourself in where you went wrong - you weren't wrong on the details, your mistakes started at the very beginning with your basic assumptions.

YOUR BASIC ASSUMPTION that Party ID was a good data set to analyze voter turnout was simply incorrect, and not based in any type of science or valid polling practice.

YOUR BASIC ASSUMPTION that State Polls v. National Polls was effectively a 50/50 proposition was simply incorrect, and ignored the fact that evidence clearly showed that national polls were not even comparable to state polls in terms of accuracy.

These were just bad assumptions from the very beginning, not based in data or science. This is where you went wrong.

Posted by: everdiso at November 15, 2012 5:06 PM
Post a comment









Remember personal info?






Site Meter 250wde_2004WeblogAwards_BestSports.jpg