The red card was one of the easier the referee will ever have to award and Walton's petulance meant Argyle had to play with 10 men for most of the match.
Barnsley capitalised and De Silva opened the scoring with a header in the first half. The home side made hard work of seing off Plymouth but they did in the last minute with a Jamal Campbell-Ryce penalty.
The defeat leaves Plymouth in 13th place.
Next up a South Coast classic when Southampton visit Home Park on Boxing Day.