With all eyes on the general election on 30 May, the best indicator for both the man in the street and those at the heart of the political campaigns is the political survey.
Vince Marmara is one of Malta's foremost statisticians, and has run surveys for over two decades. The run-up to this general election has been no different, with Marmarà publishing results from a survey every weekend during this campaign.
But what is the science behind the survey? How does one put together a sample? Does the sample size make a difference? What happens when a respondent does not answer? How does one budget for people's emotions? And why might a survey go wrong?
These are some of the questions put to Marmarà by The Malta Independent on Sunday to understand the process behind a political survey.
"The starting point is that a survey must be scientific," Marmarà said. This means, he continued, that any person has to have an equal chance of being part of the survey's sample. "You aren't going to phone everyone, but everyone has to have the same chance to be a part of the sample," he said.
Online polls, Marmarà said, are not scientific - particularly those done on social media. "In the academic world, they will treat it as rubbish because it's not scientific," Marmara said.
"I always say that, whatever happens, I can defend every number, because I know I applied science with absolute loyalty. There are no emotions, no agenda - I have none of that," Marmara said.
How is a survey sample put together?
The reliability of a survey, as Marmarà said, relies intrinsically on the reliability of the survey's sample.
For the survey to reflect the sentiments of the Maltese people, it must first and foremost be made up in such a manner that it represents the demographics of the Maltese people.
Asked what factors he takes into consideration when putting together a survey, Marmarà said that the biggest factors for political surveys are having each electoral district represented, the age of the respondents, their sex, and how they voted in the previous election.
It's an enormous undertaking. Marmarà's surveys so far during this electoral campaign have had a sample of 1,200 people. In order to achieve that, Marmara said, his team has to phone over 8,000 people. This includes phone numbers which no longer exist, or ring dead immediately, and also people who simply do not pick up the phone when called.
The data collection is then carried out on a quota basis: with a sample of 1,200 respondents spread across 13 electoral districts of roughly equal population, this translates into about 90 to 95 participants per district.
Whether a respondent makes it into the survey is then on a first-come first-served basis, Marmara explained.
So for instance, if the survey gatherers have reached the necessary amount of respondents from the 1st district, any individual that they call from that point on who happens to also live in the 1st district is not included in the survey. Data gatherers will keep calling individuals until they have reached every necessary quota to make up a representative sample.
The sample size must also be representative vis-a-vis how people voted in the previous election - in this case, 2022. This adds to the equation even further, as the sample must not only be representative of the island's demographic district population, but must also reflect how people voted in the last election as well.
"It is a first-come first-served basis, and that ensures that everyone has the chance to be a part of the survey," Marmarà said.
One will see varying survey sample sizes used, and Marmarà said that this is an interesting concept.
"If we are speaking about a survey done for a business sector, a sample of 600 people is one of the best. It's only a 4% margin of error, even based on a population of 500,000 people. If you have a population of 5 million people, a sample of 600 people is still enough," he said.
This is because statistical theory is such that if the data collector has access to everyone in the same way - via mobile phone, in this case - then such a sample size can work even for such a big population.
So why are Marmarà's sample sizes usually 1,200 people? "Because there are so many different factors in an election, and there is the defect that people can choose not to answer or say that they are undecided," Marmara answered.
Marmarà continued that due to this, he tries to use a bigger sample so there are bigger pockets of people who say that they have voted for the PN or the PL. This then allows one to properly interpret certain things like voter retention for main parties and shifts between parties.
The Multiple Imputation Technique and how an undecided voter is classified
Each survey contains a portion of respondents who will state that they are undecided. Marmarà's survey, issued on 16 May, has 10.7% of respondents who said that they have not decided how they will vote. That is equivalent to just over 30,000 voters.
That is where the Multiple Imputation Technique comes into its own.
The technique is used to try and predict what an undecided voter or a respondent who refuses to indicate a preference will do come Election Day. The technique essentially builds a profile for that respondent, taking into consideration the information that he or she would have given, such as previous voting behaviour, gender, age, district, and so on.
"Then you try to find a solution for that person multiple times to try and come up with what they would say if they had answered. The model then gives a probability score, and the person is assigned to a party accordingly," Marmara explained.
There are different techniques and ways of going about handling undecided voters, but Marmarà says that he has applied the Multiple Imputation Technique since 2005 and was always very close to the end result, with the 2024 European Parliament elections being the only exception.
'Against all odds'
There is such a phenomenon in surveys known as "against all odds" - a concept which was the subject of one of the educational episodes that Marmarà has been uploading to social media in the run-up to the election in an effort to help people understand every element of a survey.
"We speak about a 5% sampling error - which is over and above the margin of error of 3%, worth around 9,000 votes - which caters for events that may happen and for people's emotions," Marmarà said.
This sampling error - or Confidence Level - is a technical way of saying how likely the survey is of being correct. In Marmara's case, his surveys had a Confidence Level of 95% - and therefore a sampling error of 5%.
In simple terms, this means that 95 times out of 100, the survey - within its margin of error, which itself is calculated using a mathematical equation - will be correct. When it's not, though, that's when the "against all odds" phenomenon comes in.
"If the sampling error didn't exist, then we don't need an election. We can just say that we are certain, and be done," Marmara said.
"Think about a football game: if Germany were to play against Malta; that Germany will win is almost certain - but only almost. There might be a one-off, and against all odds Malta would win. It's a small probability, but it can happen," he said.
This is what happened in the 2024 European Parliament elections - but it's not a one-off. Marmarà noted that in 2016, American pollsters placed the likelihood of Hillary Clinton winning the US President Election at 99%.
"Donald Trump had a 1% chance. What happened? He won. This is the concept that I've been speaking about. Will it happen in Malta? The chances are less - but it can happen," he said.
How can a survey get it wrong?
But the question remains, how - after all the science and sampling - can a survey get it wrong, as happened in 2024?
In the run-up to the 2024 European Parliament elections, all three of Malta's pollsters got the end result wrong: Marmarà's survey predicted that the PL would win by 27,000 votes, MaltaToday's survey predicted that the PL would win by 24,000 votes, and Times of Malta's survey predicted a PL win by 30,500 votes.
In the end, the PL's margin of victory was of just 8,454 votes, shocking political observers and the parties alike. So why was each survey off the mark?
"That's where emotions - the qualitative - became superior to the quantitative. We had extraordinary events happening, and those have an effect on people's emotions and on their decisions," Marmara said.
Those extraordinary events, of course, were the Vitals Hospitals' scandal, which led to charges being brought against Joseph Muscat, Keith Schembri, Konrad Mizzi, Chris Fearne, and Edward Scicluna.
Marmarà said that there will always be undecided voters right up until Election Day, but in this case it was evident that the number of undecided voters was not decreasing.
On most occasions, the indecisive will ultimately opt for the party they voted for before, but extraordinary events can create the possibility of them not voting or shifting their allegiance, Marmarà said.
"Labourites remained indecisive up to the end owing to big events across the whole campaign. The doubt was clear, and it was even clear in the surveys that the non-voting contingent for PL was much higher, and the number of undecided voters was increasing instead of decreasing," Marmarà said.
He recalled that academic colleagues had suggested to him that, given the situation, he could consider not publishing any other surveys during the campaign, but he decided to persist with publishing his work.
"Do I regret it? I don't because I cannot abandon people's expectations midway through, and because my position is tranquil: I know what happened and why it happened," he said, as he highlighted that the same thing happened in the Brexit referendum and with Trump's election.
"That's why I decided to focus on people's emotions in my explanations this year," he said.
"I do surveys for both sides - for candidates and districts on both sides. It's my profession and I will never betray that or the people," he said, adding that the emotions seen in an election are unheard of in any other study.
Asked whether he changed anything in how he runs his surveys after the result, Marmara said he did not. "You cannot change anything in the method: mathematics is a language which you cannot change."
Asked whether, therefore, one can even take into account these emotions or other types of human behaviours - like people who say that they purposely lie about their voting intentions when called up by pollsters - Marmarà said that "emotion cannot come into the process at all".
"When the number comes out, that's the number. The number is emotionless: it is maths, science... there is backing for it," he said.
Emotions are catered for and quantified through both the sampling error and the margin of error - other than that, "the mathematical process is worked in a precise manner and with complete loyalty to science", he said.
But might we see a repeat in the general election? "A general election is a vote for you - not one to give a warning shot to one party or another," Marmara said.
"You're voting on how you want to live in the next five years, which is why you cannot compare a European Parliament election with a General Election because the sentiment is completely different," he concluded.
This story has been edited slightly to clarify why several thousand calls must be made to put together a representative sample for the survey.