Drawing a line on the color chart would admit something that oppressors don’t want to admit. Not in a million years would someone who looks at the same resume with two different names “Richard Duckworth” and “Jamelle Washington” and chooses Richard admit what he or she is doing. Not in a million years would the cop who beats a black man when he wouldn’t have beaten a white man take the Harvard racism tests.
It is precisely the fact that this line exists in the head of every American at a varying degree of color and other features without documentation or admission that it is a problem.