The Roller Coaster Electorate - A Guide to PPP's South Carolina Special Election Polling

Photo courtesy of rawstory.com

Last night, reputable polling firm Public Policy Polling (D) had one of their worst misses in months, while newcomer Red Racing Horses did little to advance their stature.

They were the only two polling firms to release public surveys on the South Carolina 1st Congressional District special election between Mark Sanford (R) and Elizabeth Colbert-Busch (D). And neither was able to see that the infamous "Appalachain Trail" Governor was about to make a triumphant landslide return to Congress.

To be fair, it wasn't an easy outcome to predict given Sanford's luck over the last three weeks.  He was abandoned by the national Republican Party, he was competing against a pseudo-celebrity challenger, he was being outspent 5-1, and embarrassing details about his divorce to Jenny Sanford were hitting the press.

Nonetheless, The Guardian's Harry J. Enten has a great post-mortem on the pollster failings in this race, specifically PPP's 2-week old finding of Colbert-Busch leading Sanford by 9 points, which as Enten notes, has the dubious distinction of being one of the worst polls taken in any special election since 2004.

Enten's piece, as well as a twitter debate between The New Republic's Nate Cohn and a Daily Kos tweeter, prompted me to take a closer look at the shifting electorate reported by PPP over the three polls and six weeks they surveyed this special election. 

As the chart below indicates, as brief as it was, the SC-1 special election was somewhat of a roller-coaster ride according to PPP, in terms of both who led the race and who PPP expected to comprise the electorate:

The race started in late-March with the Democrat at a very, very small advantage, which was unusual considering the partisan leanings of the 1st District, as well as the fact that PPP poll respondents looked awfully similar to the 2012 presidential electorate; they supported Romney over Obama by 56-40% (similar to the actual 58-40% margin), conservatives greatly outnumbered moderates and liberals by a 2-1 margin, Republicans outnumbered Democrats by 15%, and African Americans only made up 12% of the electorate.

Either way you looked at it, Sanford was in trouble in a district he had never won before with less than 66% of the vote.

Then came April 16, and things got worse. The A.P. reported that Sanford was facing trespassing charges that violated the terms of his divorce with Jenny Sanford. By the 19th, PPP was back in the field, and two days later they had the scoop: Colbert-Busch was out to a LARGE lead against Sanford, 50-41%. And my how the electorate had changed over the course of four weeks.

Colbert-Busch's margin over Sanford had grown 7 points while the share of blacks in the electorate had grown 6% (from 12% to 18%). This was a big driving factor in Colbert-Busch's overall advantage over Sanford, being that she led among this group 75-19%. But it wasn't the only factor.

In fact, every demographic you could list became more favorable for Colbert-Busch in PPP's post-trespassing scandal poll. Women, who supported the Democrat 51-41%, jumped from holding an 8 point advantage over men to a 12 point advantage. The Republicans partisan I.D. advantage of R+15 was cut to R+9. Conservatives saw a 10 point drop in their representation among survey respondents, while liberals jumped 5%. And the margin of voters who said they voted for Romney over Obama in 2012 had plummeted from +16 to just +5. All in 4 weeks!

Sanford looked finish, with only two weeks until election day:

18 special off-cycle house elections in past 10 years with polling in final 2 weeks. Avg error 3.6 pts. Only 1 over 9 pts. Sanford down by 9
— Harry Enten (@ForecasterEnten) April 24, 2013

From the time of the PPP survey showing a Sanford train-wreck, until election day where he pulled out a miraculous mini-landslide, there was no real "external" game-changer. There was a debate that was largely considered a draw, there was the national Republican Party withdrawing funding for Sanford (but that certainly wouldn't HURT Colbert-Busch). But no definable external event that  could explain the sudden surge in support for Sanford that was coming.

Just 13 days after showing Colbert-Busch running away with it, and three days after Red Racing Horses first survey of the race showing a 46-46% tie, PPP was out with some new substantially different findings. Sanford now held his first lead of the race, 47-46%. Black voters were dropping back out of the electorate like flies (down to 13% from 18% 2 weeks prior), the advantage females held over males as a percentage of the electorate was back to pre-Trespass-Gate levels, as was the Republican party I.D. advantage over Democrats, and Conservatives jumped up 9% as a share of the electorate, while liberals dropped 6%.

But making the April 19-21st PPP poll look even more incredulous were last night's actual election results. Whether by actual finding or fortunate assumption, PPP and RRH caught on to a Sanford surge, but they both missed the extent of that surge by a fair amount.

Sanford didn't tie Colbert-Busch as late polls predicted, he beat her 54-45%.

Photo courtesy of Huffington Post's live election results, which had the best graphics, and quickest results of all the agencies reporting that night (AP, Pollster, SC Election Commission), by far.

Either the constituents of South Carolina's 1st Congressional District were experiencing some serious mood swings over a relatively short period of time regarding their pick of who to send to Washington, or pollsters relied on faulty assumptions at some point during the course of their surveying.

For a more detailed look at the methodological critique behind PPP's April SC-1 poll, see this twitter debate between Nate Cohn and DKElections, from the tweet that started it.

That PPP poll (ECB+9) from two weeks ago really looks great, doesn't it...
— Nate Cohn (@electionate) May 8, 2013