The win doesn't lift them out of the bottom three but does keep them in touch with the rest of the league.
The first half was largely forgettable with both teams struggling to create chances front of goal.
Scott Taylor almost gave the visitors the lead after just ten minutes when he wriggled through the Gillingham defence but his shot from six yards was well blocked by Steve Banks.
Darius Henderson had Gillingham's best chance ten minutes later when he ran on to a long ball over the top, but his lob from the edge of the box fell just the wrong side of the post.
The game opened up after the break with both teams looking for the win. Plymouth should have taken the lead on 55 minutes through Taylor.
David Friio charged down the right wing and drilled the ball across the box for Taylor to side foot home from six yards, but the former Blackpool man hit the ball straight to the hands of Banks.
The miss proved even more costly for the visitors when Gillingham took the lead just minutes later through Andrew Crofts.
The 20-year-old, who signed a new four-and-a-half-year deal before the game, met Nicky Southall's free-kick with diving header into the far corner for his second goal of the season.
The game Second-half substitute Tony Capaldi went down in the box under a challenge from Chris Hope, but referee Singh waved away the appeals. Capaldi was incensed and confronted the referee to protest.
The game flowed from end to end as the visitors went in search of a deserved equaliser, but almost went two behind with just seven minutes left on the clock through second-half substitute Matt Bodkin.
Plymouth were unable to clear their lines as the ball bounced around their box and fell to Bodkin, making his league debut, only the drill the ball just wide with Romain Larrieu beaten.
The game ended with a frantic flurry of attacks from Plymouth and they almost got their point through Connolly who jinked into the box, but his well struck shot was well saved by Banks.
It was his last action of the game as he was sent off just moments later for scything down Darren Byfield.