The building is shaped like an irregular pentagon, with a chamfer cutting diagonally across what would typically be the southwest corner of a rectangular slab.

The facade is made of gray-green glass and polished granite, which Barnes intended would give the building the appearance of a prism.

Adjacent to 590 Madison Avenue's southwest corner is a privately owned public space covered by a glass structure, which contains chairs, tables, and bamboo trees.

[15][16] Adjacent to the building on its southwest corner is an enclosed atrium, a privately owned public space covered by a glass structure.

[18] The building was designed as a pentagonal wedge, with a large chamfer that cuts into the rectangular massing at the site's southwest corner.

According to Barnes, this shape was chosen to highlight the presence of the public atrium, which otherwise would have been overlooked due to its relatively nondescript location in the middle of the block.

[20] The presence of the atrium at the southwest corner of the lot, and the fact that the tower only occupies 40 percent of the site, allowed the rest of the building to rise without setbacks.

[21] Structurally, 590 Madison Avenue is a pentagon,[11][12] with columns closely clustered on three sides (the southern and western facades are relatively short).

[24] The inclusion of the cantilever was in keeping with Barnes's tendency for "symbolic rather than purely structural expression", as author Eric Nash described it.

[19] Originally, the atrium was meant to provide access to the adjacent Bonwit Teller department store building to the west.

[17] The atrium was originally conceived as a space that was open to the outdoors, but wind tunnel testing showed that doors needed to be added at each end.

To create the fountain, Heizer sheared off the top of a large rock and cut grooves into the surface before setting it on supports hidden within a stainless steel structure.

American sculptor James Rappa was hired to select the granite from the LaCroix quarry in Saint-Sébastien, Estrie, Quebec, and classify it with three grades of quality.

The rainwater could then drain to ground level through a notch used by the building's window cleaning equipment, preventing ice buildup and granite staining.

[9] By 1969, IBM had hired architect I. M. Pei to draw preliminary plans for a skyscraper replacing its old Madison Avenue headquarters.

[6] During the early design stages, IBM ran computer models to determine how to minimize the building's energy usage.

[2][65] The building would allow IBM to consolidate many of its offices across the city, which at the time comprised 1.5 million square feet (140,000 m2) at 23 locations.

Barnes decided to use gray-green granite, so his design team traveled to several quarries across the world, including in Canada, Norway, Czechoslovakia, and Africa.

[40] The first exhibition in the building's public gallery, "Innovation in IBM Computer Technology", opened that month, showing devices made by the company over the previous five decades.

[28] Minskoff planned to use forty percent of the atrium space for exhibiting sculptures and remove many trees, tables, and chairs.

[103][104] STRS Ohio instead opted to refinance the building in 2015 with a ten-year, fixed-rate senior mortgage of $650 million, provided by Goldman Sachs.

[18] Other tenants include:[111] Before construction commenced, New York Times architecture writer Paul Goldberger said in July 1978, "IBM promises to be a dignified addition to midtown Manhattan", despite his feeling that the design was "conservative indeed".

[22][120] As construction proceeded, Goldberger remained ambivalent about the tower, saying that the pentagonal shape and cantilevered entrance were "dramatic gestures" but that the facade looked too much like stone.

[22][121] Der Scutt, architect of the neighboring Trump Tower, said in 1981 in response to criticism of 550 and 590 Madison Avenue: "I can't find anything oppressively hideous in IBM or AT&T.

"[122] Goldberger was greatly disappointed when the building was completed, writing in 1983 that the design has "generally fallen short of expectations", particularly in its atrium, entrance, and facade.

[22][37] Architectural critic Brendan Gill likewise said of IBM: "Their idea of social awareness was to include a museum and an atrium lobby.

[39] In the book New York 2000, Robert A. M. Stern wrote that the IBM Building "was a challenge to the prevailing Modernist taste for glass, but it lacked the iconoclastic panache" of Philip Johnson's pink-granite design of 550 Madison Avenue.

[126] Mildred Schmertz of the same magazine said that, with his design for 590 Madison Avenue, Barnes "has demonstrated that a building that faithfully follows the esthetic canons of the Modern Movement can, like postmodernism, be 'contextual'.

"[21] Ada Louise Huxtable wrote that "IBM's taut, refined skin of granite and glass upstages AT&T like a suave fashion model next to a fussy dowager in a home-made dress".

Anthony Paletta of The Daily Beast said in 2022 that 590 Madison Avenue "features a very appealing atrium" in addition to the lobby's large collection of art.