Written by Anita Sharma, Vice President.

The Scottish referendum on independence landed at the intersection of personal and professional for me. I have fond memories of studying at the University of Edinburgh during college. From the moment my plane landed in Edinburgh, I was struck by the country’s beauty (could the grass be any greener?), the kindness of Scots (could they be any more fun or interesting), and the richness of its history (just Google Mary Queen of Scots or William Wallace). While I couldn’t have predicted where the Scottish National Party was headed, there was an undeniable spirit of solidarity in the air, energized, at the time, by the construction of the Scottish Parliament.

Fast forward to the present. I still feel a strong connection to the country but have also eagerly watched a civics and history lesson unfold. Many articles have been written about the amazingness of voter turnout and how, even while Scotland remains part of a centuries old union, changes have been promised and are inevitable. The majority spoke, and Scotland will not break off from the United Kingdom. Still, the will of the 45 percent who voted in favor of independence cannot and will not be taken for granted.

Now, about that 45 percent. As an American – with midterm elections on the horizon – and as a pollster, I’ve been particularly attentive to the vicissitudes of what surveys showed in the lead up to September 18th, including one poll showing voters pretty much split on the future of Scotland. If you read the papers today, it’s increasingly difficult to know what surveys to trust, if you trust any at all. Everyone cares about the numbers in the horse race but not the methodology behind those numbers.

With questions about the accuracy of polling raised in Eric Cantor’s primary loss and again with the vote in Scotland, there’s an opportunity to recap some of the key factors affecting pollsters’ success in predicting the outcome of elections:

  • Does the respondent universe – the people who actually are willing to take a survey – represent a probability sample of the electorate? In the US, getting a probability sample is increasingly difficult because of low response rates for surveys and the costs associated with reaching people on cell phones, with both of these variables being critical in the representativeness of the sample. There’s also the added complication of predicting just who is going to turn out on Election Day.
  • Does what people say they are going to do when taking a survey correspond with what they actually do? In Scotland, the social desirability of a vote in favor of independence may have been among the factors showing the outcome too close to call with one of the most recent polls on the referendum.
  • How do undecided split their vote? The number of undecided voters varies by election, but it can be pretty hard to predict the distribution of this group.
  • Do opinions shift in the last 24-72 hours of a campaign and, if so, how many? The last polls are often conducted before this window, so they can completely miss the shift as well as the magnitude of the shift.
  • Once they show up to their polling places, do people change their minds at the very last minute? Again, it’s difficult to quantify this in political polling, whether it be last minute changes for or against a certain candidate or a specific issue on the ballot.

The next six weeks will be interesting! There are a lot of state races to follow and, in turn, a lot of polling results to be judged. And let’s not even begin to wonder about 2016…