As a young man, Allan Jr was initially a valet to William Williamson and a gamekeeper for the Marquess of Londonderry.
After becoming infatuated with and working at the quarries near his property, and on the North Dock in Monkwearmouth, he decided to turn a limestone cliff in Marsden Bay into an actual house and inn.
In doing so, he followed the example of an old man, Jack Bates, who had previously (1780s) attracted custom to a cliff cave about a hundred yards to the south.
Between 1846 and 1849, the new owners of the copyhold, Andrew Stoddart and John Clay, with the financial backing of Cuthbert Ellison, attempted to eject him, having bought out all the claimants to the land above the Grotto.
Alan Robinson wrote a book in the 1970s denying the notion that Allan was a hermit, and this is confirmed by Bill Greenwell's research in 2021.