Rivlin obviously doesn't understand why we say 'All who are hungry, come and eat, all who are needy... come and join us'... It is not because we think everyone should be at the Seder, but we are inviting those who really want (have a need) to come close to our G-d.
As this explanation from aish.com:
Hungry and Needy
Rabbi Tom Meyer
All who are hungry ― come and eat. All who are needy ― come and join the Passover celebration.
It's hard to believe that as you're reciting this on Passover night, a crowd of homeless people will be hanging outside your door. Or that if you say it loud enough there's going to be a rush into your house. So what does the Haggadah mean?
The message is that we cannot have true personal freedom unless we care about other people ― both their physical and spiritual needs. That's why the Haggadah says: "All who are hungry... All who are needy..." Don't these two things sound similar? The first one refers to physical hunger ― if you're hungry come have a bite. The second is spiritual ― if you have any kind of need, join us.
Why is caring about other people so crucial to our own sense of freedom? Because we cannot get out of our ego unless we care about other people. A person has to get outside himself and realize that the welfare of others is part of his own happiness and freedom.
The majority of arabs in Israel, and throughout the world, have zero desire to become close to the G-d of Israel, rather they are trying to prove we have no G-d. Aside from the arabs who do respect Judaism it would be foolish to go out an invite arabs to a seder. I would be more concerned about the security of the Jewish guests if there were arabs at the seder.