Primates make terrible pets. They're extremely intelligent by animal standards, even the lower primates, and they need a lot of stimulation and enrichment in order not to get depressed. They will often engage in sexual activity that may embarass you in front of guests, etc. They can be sweet and loving when they're young, but as soon as they hit puberty, but they can turn into monsters who can maim or kill their owners in aggressive, non-provoked displays.
Also, they can carry most human diseases and can pass those on to their owners. Chimpanzees are believed to be the original source of AIDS, from humans eating bushmeat in the Congo.