Striker Nick Chadwick netted the fastest ever goal in Argyle's history, lobbing Palace goalkeeper Julian Speroni after 12 seconds and before a visiting player had touched the ball.
Although Palace pressed and probed Plymouth for the remainder of the match, it was the home side who went into the holidays the happier when Northern Ireland international Tony Capaldi netted a second in the fourth minute of injury time.
Former Everton striker Chadwick scored his third goal in five games to become the Pilgrims' leading scorer when he rounded off a five-man move straight from the kick-off.
The ball was played back to Mathias Doumbe, who pumped it forward for Michael Evans to head on for Chadwick to beat the out-of-position Speroni with a delicate shot.
Palace nearly equalised either side of half-time. First, England international Andrew Johnson hit a post after Doumbe had presented him with the ball and allowed him to bear down on goal.
Then, in the first minute of the second half, Doumbe headed Ben Watson's corner off the line at the near post.
Argyle had to face wave upon wave of Palace attacks during the second half, and the visitors were twice denied equalisers by the linesman's offside flag, but the Pilgrims held firm and still managed to create a couple of chances themselves.
The best came when Hungarian international Akos Buzsaky – a second-half substitute – curled a free-kick around the defensive wall just the wrong side of Speroni's right-hand post.
With the storm now weathered, Argyle broke against the run of play in the final minute of time added on with David Norris seizing on Paul Wotton's pass to play Capaldi in for a cool finish.