"I hate Asians"

I was at a Boston Celtics game with a friend up in the balcony. As the game went on, my friend and I, who are Vietnamese American and Chinese American, noticed that someone kept yelling in our direction. We did not think much of it until it got much louder. Turns out it was a young white man who appeared to be drunk and was very focused on us. He screamed out things like “I hate Asians” and “go back to your country” and that he wanted to throw us off the balcony. I called security and they came and escorted the guy out. I couldn’t help but notice that he had friends with him. They tried to get him to stop, warning him that he’d get kicked out. Despite that, I’m still frustrated that they – and the other people around us – did not call him out for being racist