The Saints last won at St Mary's against Norwich back in September and McGoldrick was left to rue his missed opportunity.
The striker was played through by midfielder Adam Lallana and skipped around goalkeeper Romain Larrieu before firing in a right-footed shot.
Plymouth left-back Chris Barker performed heroics by getting back to hack the ball off the goal-line.
Southampton beat high-flying Reading at the weekend and looked to start this game in the same rich vein of form.
In the 11th minute Jordan Robertson teed up McGoldrick whose shot was deflected by Barker just wide of the left-hand post.
Barely two minutes later Morgan Schneiderlin set up Bradley Wright-Phillips who hit a rasping right-foot drive that was well saved by Larrieu.
Midfielder Schneiderlin was next to try his luck 25 minutes later with a shot from 20 yards which went just wide of the right-hand post.
The Saints were well on top at this point and, from a Lallana corner, Robertson headed over just two minutes later.
Southampton should have taken the lead three minutes before the break when defender Chris Perry hit the post.
Andrew Surman swung over a cross from the right and Perry ghosted in but his header just shaved the outside of the right post.
Plymouth's top scorer, Paul Gallagher, climbed off the bench after 56 minutes to give his side the cutting edge they had lacked. The on-loan Blackburn forward almost scored with his first significant touch of the ball two minutes later when he fired a right-footed shot just over the bar.
It was Perry who again went closest to breaking the deadlock when his header from a Lallana corner went just wide of the right-hand post.