James says something in James 4:11 that can be confusing:

Brothers, do not slander one another. Anyone who speaks against his
brother or judges him speaks against the law and judges it. When you
judge the law, you are not keeping it, but sitting in judgment on it.

It does not follow that judging another person automatically judges the Torah, but if we read this verse in light of the next one, something becomes immediately clear. Here is James 4:12:

There is only one Lawgiver and Judge, the one who is able to save and destroy. But you–who are you to judge your neighbor?

James is not talking here about the need to render judgment in life — murder is wrong. Nor is he prohibiting labeling someone — murdering is done by murderers. Instead, he is talking about a kind of slandering — the kind that leads to verbal damnation. And he is talking about one person disagreeing or defying the Torah, which somehow supports what James is saying. And he is talking about a person who, in effect, takes the place of God. This reading of 4:11 is supported by 4:12.

Now one final point: What does James mean when he says someone speaking against and judging the Torah? In light of other texts in James, I think there is grounds to think James might have the law of love — 1:25; 2:8-10 — in view. In other words, the damning slanderer is not acting according to the law of love.