[1] He lost his parents at a very early age, but, under the guardianship of future governor of New York and vice president of the United States Daniel D. Tompkins, entered the United States Military Academy at West Point, New York, two months before his 12th birthday – making him the youngest cadet in the history of West Point.

[1][2] After a brief tour at Rouses Point helping to survey and establish the 45th parallel along the border between the United States and Canada, Brewerton taught engineering at the academy.

Apparently, his survey of Rouse's Point was inaccurate as it was later discovered that Fort Montgomery had been built on land that really belonged to Canada.

After a year at West Point, Brewerton had a number of temporary assignments building fortifications in Delaware, Georgia and New York Harbor.

[1][3] At the outset of the conflict in April 1861, Brewerton was on an extended tour of Western Europe, Greece, Palestine, and Egypt, escorted by Zealous B.

Promoted to lieutenant colonel, on August 6, 1861, Brewerton was thereafter superintending engineer of the fortifications and improvements of Baltimore Harbor, Fort Monroe, and Hampton Roads.