He was born in Naples while his parents were travelling, 13 February 1841, the son of John George Musters of Wiverton Hall, Nottinghamshire, formerly of the 10th Royal Hussars, and his wife Emily, daughter of Philip Hamond of Westacre, Norfolk.
His paternal grandparents were John Musters of Colwick Hall, Nottinghamshire, "the king of gentlemen huntsmen", who had married in 1805 Mary Anne Chaworth, heiress of Chaworth of Annesley Hall, Nottinghamshire, the "Mary" of Lord Byron's poem, The Dream.
He went to school at Saxby's in the Isle of Wight, and Green's at Sandgate, Kent, and then to Burney's academy at Gosport, to prepare for a naval career.
In 1861 he passed in the first class in his examination; was posted to the royal yacht HMY Victoria and Albert; promoted to lieutenant 4 September 1861, and appointed to the sloop HMS Stromboli, Captain Philips, serving in her on the coast of South America from December 1861 until she was paid off in June 1866.
When at Rio de Janeiro in 1862 he and a midshipman of the Stromboli, as a prank, climbed Sugarloaf Mountain, and planted the British ensign on the summit.
Musters lived on good terms with the indigenous Tehuelche people, travelling with one group from the Magellan Straits to the Río Negro, and then traversing the northern part of Patagonia from east to west, a distance of 1400 miles.