In March 2008, Prime Minister Lee Hsien Loong reshuffled his cabinet, from which time Balaji retained only his portfolio at the Foreign Affairs Ministry until his death in 2010.

He trained at the Henry Ford Hospital in Detroit, Michigan, between 1985 and 1990, obtaining a diploma from the American Board of Neurology Surgery and becoming a fellow at Harvard University in 1990.

[2] Balaji joined Tan Tock Seng Hospital as a consultant neurosurgeon in 1991, where he reorganised the way stroke patients were treated, arranged for the neurosurgical intensive care unit to be computerised, introduced stereotactic brachytherapy for dealing with brain tumours, and chaired the National Neuroscience Institute's planning committee.

In 1994, he moved into private practice at Gleneagles Hospital, establishing the first stereotactic radiosurgical treatment system driven by a linear particle accelerator in Singapore.

[6] Balaji left the medical profession to stand as a People's Action Party (PAP) candidate in the 2001 general election for the Cheng San–Seletar division of the Ang Mo Kio Group Representation Constituency (GRC).

The electoral division, helmed by the Lee Hsien Loong (who became Prime Minister on 12 August 2004), was not contested,[7] and Balaji was declared elected to Parliament on 25 October 2001.

He was Senior Minister of State for the Ministry of Health from 12 August 2004 to 29 May 2006,[2] handling matters such as the extension of the Human Organ Transplant Act[8] to Muslims and the 2003–2004 severe acute respiratory syndrome (SARS) outbreak.

[6][10] For his constituents, he set up the Cheng San–Seletar Neighbourhood Club, and petitioned the Housing and Development Board for the lease of the Seletar Hills market, due to be torn down, to be extended.

[3] Ang Mo Kio GRC was contested by the Workers' Party of Singapore in the general election of 2006, and Balaji retained his seat, the PAP winning with 66.14% of the votes polled in the constituency.