He was the son of Gregory Torre, remembered as a defiant royalist by his descendants, and of his wife Anne, daughter and heir of John Farr of Epworth.
Around 1650 Gregory's fortunes were partly retrieved when he inherited the substantial estate of his elder brother Thomas Torre, which included the small manor of Temple Belwood in Belton where James was to grow up.
[3] Torre had succeeded as an eleven-year-old to his father's not insubstantial estate in North Lincolnshire in 1660, and was brought up in the Elizabethan manor house that his predecessor Sir John Ferne had built at Temple Belwood on the outskirts of Belton.
Torre's surviving day book for this period illustrates his busy activities in his twenties and thirties as a Lincolnshire landowner with an interest in cattle breeding, and visiting amongst a wide social circle in the parish gentry.
[4] The death of his father-in-law Dr Lincolne in 1681 led to a dispute amongst the heirs, two of them alleging irregularities in disposing of the doctor's property in his last years to Betty and her husband.
Betty Torre's closest family link was with her next eldest sister Dorothea, who had married as his second wife John Wyvell of Osgodby Hall in the North Riding of Yorkshire.
There is also good evidence that in the late 1670s he was already seeking the acquaintance of the circle of Yorkshire antiquarians around Dr Nathaniel Johnston of Pontefract as a way of furthering his genealogical work, which by 1685 had generated a substantial stack of notebooks.
His day book, listing his daily expenses and receipts from 1672 to 1690. as well as the building accounts of 1672 for his renovations of his manor house of Temple Belwood, is held in the Leeds Library, MS Box IV 5.
Drake knew James's son Nicholas, but never met the man himself, and he interpreted Torre's career through his later work on the ecclesiastical province of York, not acknowledging his longstanding earlier enthusiasm for genealogy and heraldry.
[8][12] A volume titled The Antiquities of York City was published by Francis Hildyard in 1719 under Torre's name – quite spuriously, and apparently simply to take advantage of his antiquarian reputation.