In February 2015, the exhibition A Strong Sweet Smell of Incense: A Portrait of Robert Fraser, curated by Brian Clarke, was presented by Pace Gallery at the Royal Academy of Arts in London.

The Robert Fraser Gallery became a focal point for modern art in Britain, and through his exhibitions he helped to launch and promote the work of many important new British and American artists including Peter Blake, Clive Barker, Bridget Riley, Jann Haworth, Richard Hamilton, Gilbert and George, Eduardo Paolozzi, Andy Warhol, Harold Cohen, Jim Dine and Ed Ruscha.

The works were removed from the gallery by the Metropolitan Police, and Fraser was charged under a 19th-century vagrancy law that applied to street beggars.

His London flat and his gallery were the foci of a "jet-set" salon of top pop stars, artists, writers and other celebrities, including members of the Beatles and the Rolling Stones, photographer Michael Cooper, designer Christopher Gibbs, Marianne Faithfull, Dennis Hopper (who introduced Fraser to satirist Terry Southern), William Burroughs and Kenneth Anger.

[4] His flat at 23 Mount Street, on the third floor above Scott's restaurant, was described by Barry Miles as one of the "coolest sixties pads in London".

Although Jagger and Richards were acquitted on appeal, Fraser pleaded guilty to charges of possession of heroin, and was sentenced to six months in prison.

Keith Richards from the Rolling Stones was living with Fraser at the time, and it was here, sitting by the window in the lounge room, that Richards had the inspiration for the song Gimme Shelter "I had been sitting by the window of my friend Robert Fraser's apartment on Mount Street in London with an acoustic guitar when suddenly the sky went completely black and an incredible monsoon came down.

He returned to London in the early 1980s and opened a second gallery in 1983, with a show of paintings by the stained glass and architectural artist Brian Clarke,[11] but by this time he was suffering from chronic drug and alcohol problems and the gallery never replicated the success of its predecessor, although Fraser was again influential in promoting the work of Clarke, Jean-Michel Basquiat and Keith Haring.