Call it the 'Calgary effect'

A polling firm is calling it the "Calgary effect."

Overlooked cell phone-wielding young voters and people hanging up their phones are among the factors that contributed to the "major polling failure" that wrongly projected Calgary Mayor Naheed Nenshi's resounding defeat in October, says an internal Mainstreet Research report.

"It's really the whole perfect storm. ... There's just so many factors involved here, it's almost impossible to blame any one on its own, that's what makes it so unique. It's just a tough place to poll," Mainstreet Research president and chief executive officer Quito Maggi told National Observer Monday, following the release of two reports on the polling firm's handling of the Calgary municipal election.

Leading up to the Oct. 16 vote, Mainstreet consistently pegged incumbent mayor Nenshi as many as 17 points behind competitor Bill Smith, former president of the Progressive Conservative Association of Alberta. On voting day, Nenshi won with 51 per cent of the vote, compared to 44 per cent for Smith.

A key factor in misreading intentions was missing Calgary's young voters, according to an analysis conducted by Mainstreet's vice-president of analytics, Joseph Angolano.

"Calgary is one of the youngest cities in Canada," Angolano wrote, noting 33 per cent of voters were in the 18-34 age group. "When a population skews that young, it stands to reason that a methodology that relies rather heavily on landlines might have trouble mirroring the actual election results if a solid amount of that younger demographic decided to come out and vote."

Angolano concluded the Calgary results are an anomaly — hence, the "Calgary effect" — although he did note Mainstreet experienced problems in 2016 in Saskatoon, another city with lots of young people. Though mayoral election polls for Saskatoon were not as off as those in Calgary in 2017, the experiences in both cities inform Angolano's recommendation the firm take up "random digit dialing" in addition to directory dialing in order to reach more people on their cell phones.

Angolano also recommended the company poll people for more days and at more times. Traditionally Mainstreet calls people between 6 p.m. and 9 p.m., potentially missing those who work untraditional hours. Angolano recommended calling people between noon and 4 p.m., too.

He suggested Mainstreet incorporate more questions in its polling to get a better grasp of voters' positions on policy, implement American Association for Public Opinion Research standards, house its data with a university, and investigate online polling options.

Maggi added that when polling in Calgary, Mainstreet should add more language options as it calls out to would-be voters, starting with making Punjabi, Cantonese and Mandarin options available. In the future he expects those options to extend to Arabic and Tagalog.

Maggi says Mainstreet spoke to "lots and lots and lots" of voters aged 18-34, but, "we just couldn't reach the Nenshi voters."

Where Angolano called it the "Calgary effect," Maggi said he might have called it the "Nenshi effect."

"It always relates to turnout. We know that turnout in Calgary increased, but where else have we seen something like this? The only other example is Calgary in 2010," he said, when Nenshi first ran and won the city's mayoral election.

Maggi added, "When the race is not that competitive (as in 2013), when it's a snoozer of an election, you can poll and be very representative, in any mode in Calgary. But when it's a hotly contested election, in the two that Nenshi has been in 2010 and now again in 2017, something weird happens with the turnout."

Report on the Calgary Election by Mainstreet on Scribd

Polling criticized

Before and after the final results of the 2017 mayoral election, the Mainstreet polls and the Postmedia newspapers that partnered with the firm and published its results — the Calgary Herald and Calgary Sun — were publicly criticized.

A report on Mainstreet's media activity said the polling firm, "broadcast absolute, unwavering confidence that their numbers were correct before the vote, in some cases ignoring the obvious warning signs that should have given them pause. On social media, Mainstreet bickered and argued with (opponents on a) variety of the issues, even after election day. In their polling reports, Mainstreet prognosticated on Nenshi’s apparent political failure, using their data as the backdrop to explain the current political situation in Calgary. It was that front-facing communication strategy —or, really, lack thereof — that turned a failure into a disaster."

The second report, written by former Vice journalist Justin Ling, offered a series of recommendations, including: hire a media relations expert; keep media interviews on topic; create a clearer media relations structure; stop responding to every critical tweet; admit when you're wrong; stop making enemies; trust readers; share more information on the firm's website; and stop trying to win.

"This is going to make me a better pollster, it's going to make me a better person, it's going to help us communicate our results in a different way," Maggi said of the two reports, and the suite of recommendations he fully accepts.

"Sometimes bravado gets the better of us. In the Calgary election, confidence turned into over-confidence, things were said ... I've apologized for all those things. Reading Justin's report was like looking back at your life story and having somebody retell all the worst, most embarrassing things about you."

The Only Poll That Matters... by Mainstreet on Scribd

Further study expected in the new year

Following the Calgary election results, Canada's self-regulating polling industry group, the Marketing Research and Intelligence Association, announced it would conduct its own investigation into Calgary's "wonky" polling results. Other public polls that took place during the municipal election were also far off, though they suggested Nenshi would win.

The association's chief executive, Kara Mitchelmore, told National Observer on Monday that terms of that inquiry are still being determined and an academic is expected to lead the investigation in the new year.

Maggi said he looks forward to the broader inquiry.

"I actually can't wait to participate in the MRIA (investigation) because I'm really fascinated to see what happened in the other polls," he said. "Especially having unpacked what we just went through, I want to see the other side."