In 1992 he formed Dirty Three, an instrumental rock band, with fellow mainstays Warren Ellis on violin; and Mick Turner on guitar.

White has also played with various other artists including PJ Harvey, Mark Kozelek, Bonnie Prince Billy, Cat Power, Courtney Barnett & Kurt Vile, Smog, The Blackeyed Susans, Kim Salmon's STM, Venom P. Stinger, The Tren Brothers, and Nina Nastasia.

[3] Late the following year he replaced Peter Rippon on drums in a noise rock group, The People with Chairs up Their Noses, alongside Mark Barry on bass guitar, David Palliser on saxophone and lead vocals, and Jim Shugg on lead guitar.

[3] In 1984 they provided a cover version of Don Gibson's 1958 hit, "Blue Day", on the various artist's compilation album, Asleep at the Wheel, for Au Go Go Records.

[8] In 1985 White formed another band, Venom P. Stinger, as an avant-rock ensemble with Dugald Mackenzie on lead vocals (ex-Sick Things, Brainshack); Alan Secher-Jensen on bass guitar (Brainshack, Beachnuts); and Mick Turner on guitar (Sick Things, Fungus Brains, The Moodists).

[3][9] In 1992 White and Turner formed an instrumental rock group, Dirty Three, with Warren Ellis on violin (ex-These Future Kings).

[11] In the same year White and Ellis were also working in the rock bands, The Blackeyed Susans,[12] Kim Salmon's STM and Charles Marshall and the Body Electric.

[3][10] The following year White and Turner reformed Venom P. Stinger, and recorded another album, Tear Bucket, which appeared in 1996.

[10][12] In June that year Dirty Three issued their self-titled album on Torn & Frayed Records for which Australian musicologist, Ian McFarlane, praised White's "sympathetic drumming".

[17] AllMusic's Thom Jurek noted that White "is not an accompanist here, he is a collaborator, even though he didn't write the songs", with the production emphasising his "full of bassy tom toms and wispy brushwork" where White "floats, digs, sputters, halts, and pushes through the music, just as [Nastasia's] playing guitar and singing does".