Being anti-Jew is the only possible common factor among them.
If you mean among blacks and nazis, then maybe.
If you meant muslim/nazi alliance, then not really the only.
muslims and nazis have many other common factors, besides antisemitism which is their main one. Both muslims and nazis hate Christianity, western civilization, gays, women, freedom, democracy, I can go on and on.