My guess is that people see such strong parallels between him and John Rocker that they just assume he must be racist.
Quite frankly, I don't care. If he's a racist and he puts up good numbers for my team, I am willing to tolerate that. Of all the sports, baseball is the one to which racism is most antithetical; it's the closest I've seen to a colorblind society. One would imagine that spending time playing baseball would wear down racist ideas.
If he's a racist and he puts up bad numbers, well, then we just have one more thing to heckle him about.
"Hope...may be indulged in by those who have abundant resources...but its nature is to be extravagant, and those who go so far as to stake their all upon the venture see it in its true colors only when they are ruined."