Forging a close friendship with co-founding Butthole Surfers members Gibby Haynes and Paul Leary, Kramer was available when the band needed to replace their bassist later that year.
Kramer bought a Höfner Beatle bass and joined the Butthole Surfers for a string of shows in the US, and for the band's debut European tour in 1985.
Prior to his purchase of Noise New York, he recorded several songs from the band's Rembrandt Pussy Horse LP there, including "Florida" and "Two Parter".
[6] The studio was to serve as a mainstay for artists and bands both local and international, and Kramer became one of the busiest indie music producers in New York City.
In 1992, Kramer formed the band the Captain Howdy with Penn Jillette, and together with guest artists Debbie Harry (of Blondie) and Billy West, they made two albums together, both released on Shimmy-Disc.
In January 1992, Ween went on a brief tour of England with Kramer on bass and Claude Coleman Jr on drums, culminating with the recording of four songs from The Pod for a John Peel Session at the BBC.
In 2023, Shimmy-Disc released Kramer's duo collaboration with new-age pioneer Laraaji, entitled Baptismal, and a remixed and remastered double LP by Allen Ginsberg (originally produced by Hal Willner in 1989 as a single LP) titled The Lion For Real, re-titled for this re-release as The Lion For Real, Re-Born, featuring Marc Ribot, Arto Lindsay, Bill Frisell and many other musicians hand-picked by Willner.
Also in 2023, Shimmy-Disc released a six-EP limited-edition wooden box set of 7" vinyl featuring Britta Phillips (Luna), Paul Leary (Butthole Surfers), Rob Crow (Pinback), Jad Fair (Half Japanese) and Danielson, and David Grubbs (Gastr Del Sol).
He dubbed the studio Noise New Jersey, and continued to produce recordings there,[6] including, most famously, Urge Overkill's cover of "Girl, You'll Be a Woman Soon".
This phase of Kramer's career culminated in 2002 when he composed the music for Fortune's Fool, the Tony Award-winning Broadway play directed by Arthur Penn.
Kramer had been appointed assistant director on Arthur Penn's next Broadway play (Sly Fox) when his mother Rosalyn was stricken with a debilitating stroke, which drew him to Florida in 2003.
[citation needed] After moving to the Fort Lauderdale area, Kramer worked for the James Randi Educational Foundation from 2004 until February 2006.
Kramer also maintained the foundation's video library and oversaw the digital transfer of over 700 archival VHS tapes to DVD, comprising the most complete document of the life and career of James Randi.
In January 2008, Kramer toured 14 cities in 14 days in Japan with Mike Watt and Samm Bennett in a dueling bass trio named Brother's Sister's Daughter.
[10] In January 2020, Kramer was named one of the 12 curators of the Joyful Noise Recordings White Label Series, with David Lynch, Lydia Lunch, Son Lux and others.
As of January 2023[update], Kramer lives in a small town near Asheville, North Carolina, where he continues his activities as a composer/producer and maintains a modest recording/mixing/mastering facility in his home.