A convicted drug trafficker, who migrated to Australia as a toddler in 1956, could be deported to Malta after losing a High Court appeal to have his visa reinstated, Australian media report.

Sixty-six-year-old John Falzon migrated with his parents and never became an Australian citizen.

In 2008, he pleaded guilty to cannabis trafficking and served eight years in jail.

At the end of his sentence, his visa was cancelled on character grounds and he was placed in immigration detention, SBS reports.

He is currently being held on Christmas Island.

Falzon argued in the High Court that he'd served his time for his crime and ejecting him from Australia was an attempt to punish him further. 

But in a unanimous decision the High Court dismissed that argument.

The court found the immigration minister must cancel a visa held by a person if the minister was satisfied that they failed the character test because of a substantial criminal record.

Falzon's family in Australia includes two sisters, four brothers, four adult children and ten grandchildren, Greg Dyett reported on SBS News..

One of his sisters, Ann Falzon told SBS News the family is devastated.

"All comes down to he's got to go back to Malta. He came here when he was three-years-old, like he doesn't even know anybody over there. They're just going to dump him in Malta and leave him and what's he going to do, he's got no money, he's got nobody over there, we're all here," she said.

She said her brother has paid a high price for his drug trafficking.

"We’ve been waiting already how many years and we just get let down after let down. All this is over his marijuana. He grew a bit of marijuana and they locked him up for 8 years and now he’s been in detention for three years. He just wanted to come home.”