I dont know if this discussion of the Pirkie Avot {Ethics of the Fathers} helps explain this situation any better, but I will post it in the hopes that it does answer some questions. Although it is not related specifically to 'love' it is concerned with how we relate to others..
http://www.torah.org/learning/pirkei-avos/chapter5-13.html?print=1Chapter 5, Mishna 13"There are four character types among people. One who says, 'What's mine is mine and what's yours is yours' is of average character, and some say, this is the character of Sodom. [One who says] 'What's mine is yours and what's yours is mine' is unlearned (lit., [of] the people of the land). [One who says] 'What's mine is yours and what's yours is yours' is pious. [One who says] 'What's yours is mine and what's mine is mine' is wicked."This mishna compares four types of individuals in their attitude towards others. The most straightforward of the two are the pious and wicked ones. Whereas the pious one gives to others asking for nothing in return, the wicked one tries to get what he can with no reciprocation on his part.
Slightly more complex is the "unlearned" person. He is generous towards others yet hopes to receive in return. He is basically a good-hearted person who cares about others. The commentator Rabbeinu Yonah observes that our mishna calls him an "am ha'aretz," which literally means one of the "nation of the land." This expression is often translated as an ignoramus (it's tossed around quite freely nowadays), but more precisely, it means someone who is a full-fledged member of society --- and generally a productive and contributing one. He realizes people must do favors for one another and cannot live as islands. And he does his part to make life more livable for all he comes in contact with. Yet he is not entirely selfless. He hopes to receive as well as give. His interest in himself -- equally strong as his interest in others -- indicates that he does not see giving as a virtue in its own right but weighs relationships in terms of his own bottom line. He displays an ignoranc e as to the true value of giving. Nevertheless, he has a true Jewish heart -- he's one of us -- and through his acts of generosity he may one day become a person of truly giving spirit.
The final person (actually the first one listed) is one who neither gives nor takes. He is the subject of dispute in our mishna. The first opinion is that he is average, neither good nor evil. We may see him as the secular concept of a good person, crudely -- but accurately -- do whatever you want so long it doesn't hurt anyone else. Neither hurting nor helping others cannot be characterized as good nor evil. I do not wish the harm of others, neither do I care to help them. This is certainly secular: it lacks the religious sense of duty towards mankind, yet neither can it truly be considered evil. Perhaps it is the highest form of morality the secular world has to offer. And at least in a sense, it wouldn't seem that bad a way to go.
The second opinion takes a much harsher stance, comparing such a person to the wicked people of Sodom. Now certainly this fellow could stand to improve and ought to have a stronger sense of obligation towards mankind. Yet how does his evil even approximate the depravity of Sodom -- which G-d deemed so vile as to destroy the very land upon which it rested?
There's a fascinating passage in the Talmud (Sanhedrin 109a) which describes the mentality of the Sodomites. (In fact we see parallels to it in some of the strict immigration laws of more recent times.) What brought the Sodomites to such repugnant evil? The answer in a word was that they saw the lushness of their own land and desired it for themselves. And so, they banned all travelers and visitors. (Today, after its destruction (Genesis 19), it is the area of the Dead Sea, one of the most barren and uninhabitable places in the world.) Why share our richness with anyone else? As soon as you "make it," the shnorers (more respectfully: "fund-raisers") line up at your door. Who needs it? Not of course that they intended anyone else *harm* to be sure. They just wanted the comfortable life for themselves. "No Solicitors" figured prominently at the gates of their city (a rather common sight nowadays -- although most of the time the newly-arrived Israeli "solicitor" has no idea what the word means...). We wish you all the best, but keep your suffering and misery to yourselves!
Step two, continues the Talmud, the Sodomites enacted all sorts of cruel as well as absurd laws banning hospitality and charity, and doing away with all semblance of fair play. No one had any form of obligation towards others or was required to pay for any infraction. The Talmud lists such laws as that if one would hit his fellow, his fellow would owe *him* medical expenses for the service of blood-letting. Other choice rulings were that if A would cut off the ear of B's donkey, A would keep the donkey till it grew back, or if A would injure B's wife causing her to miscarry, A would take her till she again became pregnant.
The people of Sodom likewise decreed punishments for acts of charity. The Talmud records that when a young girl gave some bread to a poor person, they punished her by smearing her flesh with honey and tying her up on a roof, leaving her to die a horrific death at the hands of the bees. (The Sages state that it was this final breach of humanity which sealed the fate of these G-dless and despicable people.)
How did simple selfishness transform itself into such vicious as well as asinine cruelty? Simple -- it was not a transformation at all. They simply lived out their own selfishness to its logical conclusion. They didn't want to share, they wanted to live for themselves -- and so naturally they went to any and all extremes to free themselves of the burdens of hospitality and humanity -- as well as all moral accountability.
And the lesson for us is that there is simply no such thing as a person who focuses wholly on himself yet is still an "average" individual. If we accept that all humans are created in G-d's image, we will naturally feel obligated to help them and will share in their suffering. We might even see our wealth and talents as a responsibility: as gifts from G-d to be used in the manner He wishes. Once humans are in G-d's image, they are all equal, significant, and deserving of our love and respect.
If, however, a person feels others' problems are not his problems, his life will be one of selfishness and unlimited cruelty. He will sleep easily at night, blissfully unconcerned with the suffering of others; in fact he will go to any extreme to see that their plight does not interfere with his personal comfort and equanimity. On a national level, it is difficult for the isolationist mentality of "It's not our problem" to avoid spilling over into the national character of a country's citizens. (Such an attitude could not be more UN-Jewish, incidentally -- even if no one has the means of saving the entire world.) On a personal level, someone who thinks exclusively about himself has not reached the maturity of character required for friendship, marriage, or any type of serious relationship. Such a person will begin to see others as objects -- pawns to be used for his own selfish ends, then to be discarded when no longer useful.
(This also explains why among other things Sodom was known for sodomy. Relationships are one-way. Love is not giving: creating and becoming responsible for a family. It is taking: getting whatever sick pleasure I fancy.)
The sad but realistic conclusion is that one cannot "basically be a good person" if his raison d'etre is selfish. If we're really generous, neither hurting nor helping others can be considered "average." But average people -- if they do nothing to elevate their souls -- will not long remain so. You either grow or you fortify yourself in your mediocrity -- going to every cruel excess to remain oblivious to the rest of mankind. Without a sense of love of and appreciation for others, refusal to help will turn into a lack of concern and eventually an icy callousness towards the needs and suffering of others. Goodness cannot be defined in a vacuum. If you care about and sympathize with others, you are a human being yourself. If you are a person who at best chooses not to hurt others (because -- well, otherwise they might hurt you), your life is one of survival and self-preservation -- and ultimately of pathetic emptiness.