Its fear. The racist in the beginning didn't want their white children
exposed to black sexuality. They feared that would be the end
of their family tree. Of course that only fired up the curiosity. White
men left their white wives to have sex with slave girls to the point
the white girls got jealous and got themselves in trouble by screwing
the slave guy, of course they yelled "rape" to save themselves which
got the poor black guy in trouble. Trouble usually meant torture and
death. So white women were basically under control not to have
sex with any other men except "white". It's the 21st century now
I don't think I need to say more, but I'm sure you've noticed how things
are going from some of the answers above me.