First, no one inferred that he was racist, they said what he had said sounded racist. Second, why do you think it's "extremely serious" to consider something someone said racist? I consider racism itself extremely serious, but there's no consequence for expressing racism in a quasi-anonymous comment environment. There's not even any consequence for being called a racist. I am utterly confused as to how saying that racist statements sound racist could be "extremely serious", in that we should refrain from doing so. What consequences do you expect to follow?
Actually being a racist? Bad. Pointing out that something sounded racist? An opportunity for the person who said that to apologize and reframe or withdraw, perhaps leading to a less racist environment.
You're missing the point. It's unavoidable that if you say "you sound racist" that that attributes a trait to the person rather than their behavior, and is not much different than saying "you are a racist." The rule of thumb is address the behavior rather than the person, which is why the corrective response is supposed to be something like "wow, what you said sounded racist" It's a subtle but important difference. Then you are inviting the person to ask themself internally "did I mean what I said?" or "do I want to sound racist to other people?" instead of going on the defensive because they as a person are being labelled a cultural undesirable. If you make a statement about the person themself there's no avenue forward for discussion or correction, you just made an enemy. But if you make it about the speech, even if they get defensive about it you can clarify that you are addressing the behavior.
There are consequences here because there is username, user identities are often shared across sites, and there is a karma system here. They may be mild, but obviously karma is consequential or it wouldn't exist as a concept on the site. Your score is your aggregate reputation among other users. People fear saying things that will lower their karma.
Actually being a racist? Bad. Pointing out that something sounded racist? An opportunity for the person who said that to apologize and reframe or withdraw, perhaps leading to a less racist environment.