His father died when Carr was less than two years old, and for a time his mother attempted to continue raising her children at Spring Forest, but by 1781 the family had moved to Monticello, Jefferson's plantation.
During the interval before he remarried, Carr may have fathered at least one child, and perhaps two children, with a free woman of color named Judath Barnett who lived near Dunlora.
Over the next decade, he rebuilt the plantation house at Dunlora, in part using brick leftover from the construction of the main buildings at the University of Virginia.
During the War of 1812, he again served a Captain of Cavalry, this time with the Albemarle Volunteers, First Elite Corps, under Colonel Thomas Mann Randolph, his second cousin once removed.
These claims were given credence due to several factors such as Jefferson's presence at Monticello during the time periods that the children were conceived and the lack of pregnancies when he was not present.