He was noted for his propensity for physical violence as well as a dapper appearance that led The New York Times to describe him as "looking like a prosperous banker or broker" and contemporary chroniclers as "smooth-faced, high-featured, well-dressed, a Gangland cavalier" and "a fop in matters of dress".

[6][7] His nickname, Biff, was a period synonym for "punch" or "hit", and it was coined in response to a youthful fight in which Ellison, then working as a bartender, knocked unconscious a customer who refused to pay for a beer.

[11][12] As one writer observed, "The politicians loved [Ellison], for he was a valuable man around election time, the mere sight of his huge bulk being sufficient to prevent many an honest citizen exercising his right of franchise".

[13] Ellison came to wider public notice in the summer of 1902 after assaulting a police officer, Detective Sergeant Jeremiah Murphy, at Henry Wulfer's Sharkey's, a Fourteenth Street saloon that stood opposite Tammany Hall.

On March 27, 1903 the police of the 15th and 18th precincts came back, this time armed with a warrant, an axe and a sledge hammer and raided Ellison's pool hall.

The second for the murder of "Eat-Em-Up" Jack McManus, a friend of Ellison, and the third for Kelly failing to equitably distribute political campaign money of which he had custody in the district.