Plymouth should have snatched the lead on five minutes, centre-back Leigh Bromby failed to snuff out Bjarni Gudjonsson's right-wing cross and striker Micky Evans failed to add the finishing touch from six yards.
Midfielder Nick Montgomery tried his luck from long range before United broke the deadlock after ten minutes. Neil Shipperley steamed in to connect with a Paul Ifill centre and keeper Romain Larrieu fumbled the ball over the line.
United defender David Unsworth saw his 30-yard free-kick stopped by Plymouth's French keeper before the Pilgrims caught United napping in the 19th minute, midfielder Paul Wotton hitting the foot of the post with a speculative strike from 25 yards
Republic of Ireland international Alan Quinn turned the screw after being stopped by full-back Anthony Barness, his right-foot free-kick cutting through the wall and flying into the bottom right-hand corner from 25 yards.
United keeper Paddy Kenny brought off a stunning stop minutes later blocking Evans' fierce strike as the little striker connected with a Gudjonsson cross.
Quinn powered another long-range strike over the top as he linked with Steve Kabba before Plymouth almost found a way through, Gudjonsson beating Bromby in a race for the ball before lashing in an angled drive palmed away in spectacular fashion by Kenny.
United escaped in the 54th minute as Evans connected with Tony Capaldi's corner, only to see his close-range effort cleared off the line by Montgomery.
The Sheffield side, stung into action, hit back minutes later, Plymouth keeper Larrieu foiling Kabba and then stopping Shipperley from eight yards out.
Quinn almost snatched a third on the 84th, his dipping free-kick from the edge of the box tipped onto the bar by Larrieu.