After graduation, he worked as a country editor of a weekly newspaper, reading law with the goal of entering the bar at the same time.

He remained a clerk until President Ulysses S. Grant made him cashier of the United States under Treasurer John C. New.

Gilfillan, rising steadily through the ranks, was appointed Treasurer by President Rutherford B. Hayes, and he served from July 1, 1877 to March 31, 1883.

During this period he was also Sinking Fund Commissioner for the District of Columbia; he was not paid for his service on the board until a bill of Senator George P. McLean of Connecticut passed Congress, awarding him $4,750.

As Treasurer, Gilfillan acquired a reputation for integrity in his dealings, launching anti-corruption investigations, avoiding politics, and attending to his business.