Substitute Capaldi converted a cut-back from fellow benchman Mathias Doumbe to bring about furious protests from the Sky Blues' players and staff, who complained that the goal came after the extra-time allocated by referee Lee Probert.
Until Northern Ireland international Capaldi's late intervention, last season's Second Division champions looked like heading for a rare home defeat.
They went behind on the stroke of half-time when captain Paul Wotton dallied on the edge of his penalty area to gift Stern John an easy goal.
As Wotton hesitated, Dele Adebola pressured the skipper and John seized on the loose ball before placing it past exposed Argyle goalkeeper Luke McCormick.
The Pilgrims forced five corners in the first 20 minutes, with centre-back Graham Coughlan going closest with a header that flashed over the crossbar.
The Pilgrims were denied a penalty soon afterwards when City centre-back Steve Staunton clearly handled, but the visitors almost netted against the run of play when Andrew Whing was allowed a free header from Stephen Hughes' corner.
Coventry held on comfortably during the second half and it took a fine piece of goalkeeping from McCormick to prevent Gary McSheffrey from doubling the visitors' margin.
That save proved vital. The clock had ticked past the five minutes originally added on when Capaldi converted, bringing protests from the Coventry contingent that, not only was the goal scored after the game should have ended, but also that Adebola had been fouled in the build-up.