The building is eight stories tall and was designed by Rudolphe L. Daus in a mixture of the Beaux-Arts and Renaissance Revival styles.

The remainder of the building contains ornamental details such as a curved corner with an oculus window, as well as a deep cornice on the upper stories.

The company's business grew so rapidly that it moved some operations to another building in 1904 and constructed a six-story annex at 360 Bridge Street between 1922 and 1923.

In 1943, the company sold off the building, which has remained a commercial structure ever since, accommodating offices, laboratories, and educational institutions.

[9][12] The southern and western elevations of the facade, which respectively face Willoughby and Lawrence Street, are each divided vertically into three bays and are highly similar in design.

[10] At the easternmost section of the Willoughby Street elevation is an additional, narrow bay without ornament and clad almost entirely in brick.

[10] On Willoughby Street, the easternmost bay contains the building's main entrance, a double-height arch with an entablature above.

Recessed beneath the center of the archway is a rectangular door frame with molded earpieces, receivers, wires, and other telephone-related motifs.

The corner of Lawrence and Willoughby Streets contains a stoop with metal balustrades, ascending to a glass door.

[10] On the second to fourth stories, the facade is clad in brick and contains grooves at regular intervals, which give it a rusticated look.

A protruding cornice runs above the facade on the fourth story, continuing onto the narrow Willoughby Street bay.

[10] On the fifth and sixth stories, a pair of engaged columns flanks each of the three primary bays on Lawrence and Willoughby Streets.

[5][9] The structure is largely constructed with a steel frame, although the foundations are composed of concrete walls measuring 4 to 5 ft (1.2 to 1.5 m) thick.

Air was drawn down the shaft to the basement, passing through several filters, and then was supplied to the offices inside the building via fans and ducts.

[5][12] The eighth floor originally housed the New York and New Jersey Telephone Company's primary Brooklyn exchange and could accommodate 5,000 to 6,000 subscribers when the building opened.

[31] All of the trunk lines from Nassau and Suffolk counties were moved from 81 Willoughby Street to the Atlantic Avenue building, as were the storerooms, repair shops, and supply rooms.

[36] The New York Telephone Company filed construction plans in November 1920 for an eight-story annex between Bridge and Lawrence Streets.

[37] The company began constructing a six-story annex at the northwest corner of Willoughby and Bridge Streets in April 1922.

[41] The annex at 360 Bridge Street was completed in October 1923 at a cost of $1.5 million; it housed three central office departments that could not be accommodated in the old building.

[42] During 1923, New York Telephone hired Western Electric to install one manually-operated and one machine-operated central office at 81 Willoughby Street.

[51][52] After 101 Willoughby Street was completed in October 1931,[53] the company continued to use the older structures to accommodate central office equipment.

[54][55] The New York Telephone Company sold the building, which at the time housed the Sperry Gyroscope Corporation, to an investor from Manhattan in August 1943.

[56] Poly Tech's Institute of Polymer Research moved to 81 Willoughby Street in late 1946, operating five laboratories there.

[57] The same year, the Sperry Gyroscope Company moved its training school to the building, occupying the second floor;[58] the space could accommodate 80 daily students.

[50] Lebanese-American businessman Albert Srour bought 81 Willoughby Street in 1990 and cleaned the building's cornice and facade, as well as replaced the elevator.

[14] The Municipal Art Society's Preservation Committee, along with local civic group Brooklyn Heights Association, began petitioning the New York City Landmarks Preservation Commission (LPC) to designate over two dozen buildings in Downtown Brooklyn as landmarks in 2003.

[67] Christopher Gray of The New York Times referred to the building in 2008 as "a robust structure dominated by six great three-story arches", which nonetheless had "delicate and inventive" decoration.