OPINION: When technology goes rogue

THOSE of us who worry about the malevolent potential of artificial intelligence, commonly called AI, were given a salutary reminder by way of the recently publicised UK Post Office scandal.
That shameful affair shattered the lives of up to seven hundred sub-postmasters and postmistresses, wrongly accused of fraud, victims of a technology gone rogue. Many innocent victims were prosecuted, jailed, bankrupted, impoverished and, in a number of cases, driven to suicide.
However, the Prime Minister’s description of its being ‘one of the greatest miscarriages of justice in our nation’s history’, although well meant, will have been taken with a large helping of salt by those mindful of the British justice meted out to so many Irish people in the late 1990s.
Although the scandal had been ongoing for some twenty years, it has only now created the stage where the wrongs can be righted and where the good names of those unjustly punished can be restored. And all of this because of the storm created by a remarkable ITV programme which finally forced the British Government to quash all the convictions and make compensation payments.
It all started with the introduction of the ‘foolproof’ Horizon accounting system by the British Post Office to supervise and validate the cash transactions of hundreds of small sub post offices across the UK. Built by Fujitsu, it did not take long for the glitches to appear. In the remote Scottish village of Dalmellington, the postmistress was puzzled when her computer screen froze as she recorded an £8,000 cash lodgement at her post office. She pressed ‘enter’ again, and then again, to no effect. What she did not know was that by then the system had recorded her as personally liable for an amount of £24,000. In short time, the Post Office had started a fraud investigation; she was found guilty of embezzling what was an entirely fictional amount.
What nobody, except those at the top of the management ladder, knew was that the ‘Dalmellington bug’ would strike again and again until, in the end, seven hundred sub post office owners had been convicted of theft and false accounting.
In the remote Tyrone village of Killeter, two investigators from London arrived to accuse the sub postmistress, Deirdre Connolly, of embezzling £15,000, the amount which had gone unaccounted for in the Horizon system. Was she providing the money to the paramilitaries, they wanted to know, before handing her over for prosecution.
As the number of cases piled up, it became clear to Post Office management that something had to be amiss. But the cover up continued, regardless of the injustices being inflicted on loyal employees. The system cannot be wrong, the Post Office insisted. The computer was foolproof. Up against the expertise of computer specialists, there was no way that isolated sub post masters, without any technical competence, could challenge the workings of the Horizon system.
It was only when a whistleblower at Fujitsu came forward that the truth came out. Despite the denials, it turned out that defects in the system had been identified early on and that officials in the Post Office were fully aware of the fact. The condemnation came loud and fast. The current Minister described the Post Office as being not only incompetent, but malevolent in its actions. The former chief executive, Paula Vennells, gave back her CBE.
The Post Office scandal is a cautionary alert of what can happen when too much authority is delegated to the gods of the machine. Artificial intelligence – now just one step away – can be as malevolent as it can be beneficial. Humanity would do well to be wary of what advanced technology can bring.