Four hundred years ago, in July 1610, the painter known as Caravaggio set out on his last journey. He was travelling from Naples towards Rome in a boat loaded with three of his own paintings – gifts intended for the papal nephew, Scipione Borghese, who had helped to arrange the artist’s pardon for killing a Roman pimp named Ranuccio Tomassoni in the summer of 1606. Caravaggio had been a fugitive for more than four years, living under a banda capitale, a papal sentence of death that had been issued soon after the murder.

Now, finally, redemption was in sight. But it was not to be. Caravaggio was in a bad way even as he embarked on the felucca for Rome. Some nine months earlier he had been waylaid outside a notorious Neapolitan tavern called the Osteria del Cerriglio. Four men had attacked him in the dark alleyway next to the tavern’s side entrance. It had been a vendetta attack, payback for another of Caravaggio’s many crimes, an assault perpetrated on a nobleman on the fortress island of Malta during his long and hectically zig-zagging flight from papal justice. While three of his assailants held him down, his aggrieved Maltese enemy cut a message of revenge into Caravaggio’s face. At first, the painter had been reported dead. Later it was simply said that he had been so badly disfigured as to be almost unrecognisable. On the evidence of the two pictures that he was able to paint in the aftermath of the attack, he had been so badly injured that both his eyesight and his ability to hold a paintbrush steady had been critically compromised.

The sick man’s long-awaited journey to Rome in the summer of 1610  went badly wrong. When he got to the small coastal garrison at Palo, where he hoped to disembark and load his pictures on to a coach bound for the Eternal City, his papers seem not to have been in order. He was put in jail, his luggage left on board ship, and the boatman carrying his paintings went on to the small settlement of Porto Ercole. Those pictures were his passport to forgiveness, so Caravaggio bribed his way out of prison and hurried to Porto Ercole himself by land in an attempt to intercept his prized possessions. But the effort of the journey was too much for him. The heat of July, coupled with his own exertions, brought on either a stroke or a heart attack. Almost as soon as he arrived in Porto Ercole, he died.

Caravaggio was just thirty-seven years old, and he had lived the most dramatically disturbed and interrupted life of any great painter. Yet in that short span of time he had managed to create a truly compelling and extraordinary body of work: The Calling of St Matthew and The Martyrdom of Matthew, in the Contarelli Chapel in San Luigi de’ Francesi; The Conversion of St Paul and The Crucifixion of St Peter, in the Cerasi Chapel of Santa Maria del Popolo; The Death of the Virgin, now in the Louvre; The Seven Acts of Mercy, in the church of Pio Monte della Misericordia; The Beheading of St John the Baptist, for the Oratory of St John, attached to the co-Cathedral of Valletta, in Malta. Using light and shadow to unparalleled dramatic effect, setting scenes and stories from the Bible as if taking place in his own time, focussing with relentless immediacy on the poverty of Christ and his disciples, Caravaggio was like a lightning strike, transforming not just the world of Italian painting but the entire landscape of western European art. He shaped painting in Spain like no other artist before or since, exerted a huge influence on the Dutch artistic tradition and powerfully formed the development of painting in France: without Caravaggio, the work of Ribera and Zurbaran, Rembrandt and Georges de la Tour, are all unimaginable. His influence continued into the nineteenth century – Theodore Gericault’s Raft of the Medusa, the founding masterpiece of French Romantic painting, is among other things a monumental homage to the dark and brooding art of Caravaggio – and on into the twentieth, with Picasso’s despairing confession to Salvador Dali that he had tried but failed to give the agonised, dying horse in Guernica the same sense of heavy animal reality as the horse attending St Paul in Caravaggio’s great Cerasi Chapel picture.

Caravaggio’s influence continues to this day. A few years ago I interviewed the American film director, Martin Scorsese, and discovered that Caravaggio was one of his own personal touchstones. Caravaggio’s sense of light and shade and his dramatic sense of composition had shaped numerous frames and scenes in Scorsese’s early movies, Taxi Driver and Mean Streets. Many years later, when he set The Last Temptation of Christ on the streets of New York, Scorsese was once more cleaving to Caravaggio’s example: “the idea was that Jesus was going to be Jesus Christ on Eighth Avenue and 49th Street ...  quite a place, especially at three and four in the morning. This is where Jesus would go. He wouldn’t be hanging out on Park Avenue in New York. He’d be in the street with the crack addicts and the prostitutes. The idea was to do Jesus like Caravaggio.”

I should confess to a strong personal interest in the subject. During the past ten years and more, in between writing my columns for this newspaper and making television programmes about art and culture for the BBC, I have been working on a lengthy, highly detailed critical and archival biography of Caravaggio. As a result of new research and new finds, unavailable to previous generations of biographers, I believe I have been able to shed light on many aspects of his life that have until now remained shrouded in mystery – including his much debated sexuality, the circumstances that led him to commit the murder of 1606, the events surrounding his imprisonment on (and escape from) the island of Malta, and the cause of the vendetta that led to his terrible scarring in 1609 in Naples. I believe I have also been able to offer a convincing solution to the riddle of how Caravaggio met his death in the summer of 1610.

In Rome, the quatercentenary of Caravaggio’s death is being marked by an exhibition of almost thirty of his paintings at the Scuderie del Quirinale. The show was pulled together at rather late notice and suffers, inevitably perhaps, from certain weaknesses and omissions. The paintings of Caravaggio’s Maltese period are sadly under-represented; his last two pictures have not been procured; and the magnificently morbid David with Goliath, which was certainly painted in 1606, straight after the murder of Tomassoni, has once again been romantically mis-dated to the end of Caravaggio’s life. Those quibbles aside, this is a brilliant and thrilling exhibition, worth seeing alone for the bleak and almost unbearably moving pictures that the artist painted in Sicily, just before the last year of his life. With all due respect to Martin Scorsese, the truth is that no one has ever quite managed to do Jesus like Caravaggio.

Caravaggio: A Life Sacred and Profane, by Andrew Graham-Dixon, is published by Allen Lane in July.