Here is the story from Talmud concerning how Halacha must be decided by majority opinion...
http://www.torahtots.com/parsha/devarim/nitzav3.htmDOWN TO EARTHThe Talmud (Baba Metzia 59b) explains, "it is not in heaven" as follows:
After the Torah was given, it was no longer "in heaven." Hashem does not make Torah decisions in Heaven. Halachic (Torah law) decisions must be decided by human authorities following the guidelines given to Moshe at Har (Mt.) Sinai. It is Hashem's will that the Sages apply the laws of the Torah to the best of their human understanding. Decisions must reflect the opinion of the majority of a Bait Din (Jewish court), who are the final authority in all cases of Torah law.
The Talmud (ibid) brings this story to prove its point.
The Sages were debating whether or not a certain type of oven could become tamay (impure). The majority of the Sages ruled that it could. Rabbi Eliezer ben (son of) Horkenos held that it could not.
Rabbi Eliezer ben Horkenos, perhaps the most outstanding Sage of the generation, cited many proofs in favor of his position, but the Sages, who were the majority, would not accept these proofs.
Rabbi Eliezer ben Horkenos declared: "This carob tree will demonstrate that the Halachah (Torah law) follows my opinion."
A miracle occurred whereby the carob tree uprooted itself and replanted itself 100 cubits away. (some say, four hundred amot).
The Sages replied: "Halachah is not established on the basis of a carob tree.* "
[*Since Rabbi Eliezer was a very righteous man, the tree might have been uprooted at his command. This does not prove, though, that his ruling was correct.]
Rabbi Eliezer ben Horkenos declared: "This stream of water will demonstrate that the Halachah follows my opinion." The stream of water began to flow backwards against the current.
The Sages replied: "Halachah is not established on the basis of a stream."
Rabbi Eliezer ben Horkenos declared: "The walls of the Bait Hamidrash (House of Study) will demonstrate that the Halachah follows my opinion."
The walls of the Bait Hamidrash began to tremble and fall, and the Sages feared that any moment they would collapse.
Rabbi Yehoshua called out to the walls: "Why are you interfering in a Halachic debate among Sages?"
Immediately, out of respect for Rabbi Yehoshua, the walls did not collapse, but out of deference to Rabbi Eliezer, they did not return to their original upright position either. They remained slanted.
Rabbi Eliezer ben Horkenos declared: "The heavens will attest that the Halachah follows my opinion."
A bat kol (heavenly divine voice) proclaimed: "Why do you contest Rabbi Eliezer? The halachah always follows Rabbi Eliezer's teachings."
Rabbi Yehoshua rose and declared:
"It is written: 'It is not in heaven." ' (Devarim ibid).
What is meant by; 'It is not in the heaven'? Rebbi Yirmiah said: It means that we don't listen to a bat kol in matters of Halachah, for the Torah was already given to man at Har Sinai.
Rabbi Yehoshua continued:
"We don't listen to the bat kol because You (Hashem) already wrote in the Torah at Har Sinai (Shmot, Exodus 23:2) 'According to the majority (the matter) shall be decided.'*
[*R' Yehoshua understood this to mean that Hashem would never interfere with the judicial process through which the law is decided. Accordingly he interpreted the Heavenly echo to be merely a test of whether the Sages would hold their ground. And the next story proved him correct.]
Later, one of the Sages, Rabbi Natan met Eliyahu Hanavi (Elijah the prophet). He asked him: "What did Hashem say during this argument?"
Eliyahu replied to him: "He was laughing and saying (with satisfaction), 'My sons won me in the discussion.' "*