Some recent polls…

Posted by padams under Media Commentary, Political Strategy

Paul Adams

Sometimes when a journalist wants to make a point, he or she refers to “the polls” as if they were a single entity. Of course, there are often times when a number of polls — even with different survey dates, and different questions — show similar trends. Indeed, when this happens, the polls do reinforce one another, and give us greater confidence that the trend they express is real.

But that isn’t always the case. Every once in a while, a poll comes along that tells a very different story than others conducted in a similar time-period. This is what happened last week when a poll by Ipsos Reid showed the Conservatives with an 11-point lead over the Liberals. Other polls by Harris-Decima, Nanos, and EKOS Research (with which I am associated), in contrast, showed a close race, as they have through most of the summer.

I’ve seen this phenomenon of the off-trend poll from the inside as both a journalist and later as a pollster (now with EKOS Research), and it isn’t always easy to know what to do, when you have one sitting in your hands. A poll like that can be the herald of a new trend — exactly what pollsters and journalists are looking for in their polling — or it could be the notorious “twentieth out of twenty” polls: the one that falls outside the margin of error, usually described as plus or minus a certain figure nineteen times out of twenty. Of course, polls may also be wrong because of non-statistical error, which is all that the margin-of-error concept captures.

In the 1993 election campaign, many observers were surprised that Kim Campbell’s Progressive Conservatives held up so well early in the campaign — retaining a lead over Jean Chrétien’s Liberals — despite what seemed like a terrible campaign. Then, a Toronto Star poll came along saying the Liberals had vaulted into the lead. That made intuitive sense, but no one was quite sure until a CBC poll that I was involved with came out a few days later confirming the trend. The Star had the better story because it was first with the news, but believe you me, we had more confidence in what our poll said because it confirmed what the Star’s had already reported.

In the 2006 election campaign, at EKOS, we had a surprisingly high number for the Liberals in a smallish (under a 1000) sample taken on a weekend. And weekends, for whatever reason, often produce off-trend results. It would have made a great story — if it were true. If not, it would all turn into a embarrassment within days. We decided to sit on the result to see what Monday’s numbers brought. We and the Star were criticized by some, and even accused of manipulation, but the next night’s results settled back on-trend, and we were glad we had made the decision we had.

At the same time, I am not sure I would argue for that same decision today if I were confronted with it again. People sometimes complain about all the polls being taken nowadays, but the fact that we get so much data nowadays helps us weed out what might be misleading results. In 2004 and 2006, only Nanos had a daily tracking number throughout the election campaign. In last year’s election, Nanos was joined by Harris Decima and EKOS. Because Nanos had increased its sample sizes from the early years, and because EKOS was using a new methodology called IVR which enables much larger sample sizes, the number of Canadians being sampled each night by major national pollsters had increased by many multiples.

The result is that an off-track results gets identified quite quickly. With so many polls in the field, and In the internet age, with information circulating so quickly, I am inclined to think that pollsters should put their polls out, and take their lumps (as they surely will within a very short time) when their poll sounds an off-key note.

In fact, just a matter of hours after the Ipsos Reid poll showed an 11-point lead for the Tories last week, Harris Decima showed the same close race everyone else had been seeing all summer, and a few days lateran EKOS poll said something similar.

There was no fault in putting the Ipsos poll out, I am now inclined to think. The mistake was in trumpeting it as strongly as some newspapers did, and Ipsos did in its own release.

Unfortunately, this poll, unsupported by any other has become something of a “factoid”; witness a line in this morning’s Ottawa Citizen which states that, “some recent polls show a summer swoon for Ignatieff’s Liberals”.

Not some, but one; and all the others tell a different story.

By the way, here’s a seat projection based on the latest EKOS poll, illustrating the close race that most pollsters are seeing at the moment: it suggests that the Tories would win 119 seats, the Liberals 111, the Bloc 49 and the NDP 29.

Paul Adams teaches journalism at Carleton