Oh it's more complicated than that, but getting in to the psychology of fascism involves a lot of typing and there are more articulate authors who have already explained it at length.

it is that they live in an environment where they are told that Hispanic people are a threat, that caravans are "invading" the US to pillage it

They already hated brown people long before that.