This reasoning is shortsighted. You stop at their actions instead of trying to understand the intent behind why they engaged in those actions.

They did not simply “want to attack and vandalize”. Vandalism and Violence are a means, not an end. They sought to engage in those means for a purpose.

There is nothing inherently wrong with engaging in these actions themselves. It is what one is trying to accomplish by doing so that is called into question.

If those people had been there to overthrow our capitalist overlords and instill a socialist regime, then I would have 100% supported their actions. Yet, they weren’t; the reason they did so is to bring about a fascist regime. That’s why what Babbitt did was appalling. That is the key difference.