Because those who weren't all for the Jacobin cause were thought to be secret royalists or whatever and their presense was a threat to the Committee and the Jaobins in charge. Robespierre and his allies who basically afraid and did whatever they could to ensure their grip on power was tight, obviously it did not work
