In their book Bob Dylan All the Songs: The Story Behind Every Track, authors Philippe Margotin and Jean-Michel Guesdon describe the song as one in which Dylan and co-writer Robert Hunter have "created a beautiful love story set as a film noir, with abandoned cars lining the boulevards and light from only a few stars and the moon".

They also claim that the song bears a resemblance to Otis Rush's 1958 song "All Your Love (I Miss Loving)" with a lead guitar part that is similar to Fleetwood Mac's 1968 single "Black Magic Woman".

[5] Jon Dolan, writing in Rolling Stone, which placed the song 16th on a list of "The 25 Best Bob Dylan Songs of the 21st Century", praised its atmospheric musicianship: "With Mike Campbell’s guitar lashing against rumbling drums, and the forlorn feel of Donnie Herron’s trumpet and David Hidalgo’s accordion, the song has a mysterious noir feel".

In an article accompanying the list, critic Tyler Dunston noted that the lyrics seemed to be a deliberate reversal of the Ovid exile poem that gave Dylan's song its title: While Ovid was exiled by Augustus, alone in a land far from home where no one speaks his language, Dylan is at home with 'the only love' he’s ever known.

Ovid’s vision of exile is in many ways the same as the one Dylan hints at in lines like, 'Don’t know what I’d do without it/ Without this love that we call ours'.

[9] Ultimate Classic Rock critic Matthew Wilkening ratedit as the 4th best song Dylan recorded between 1992 and 2011.

Blue-Tongue Films created a music video for the song directed by Nash Edgerton, a filmmaker and stuntman, that features Joel Stoffer and Amanda Aardsma in an extreme marital spat.