It would be impossible to speak comprehensively to this, but there are certain factors that are fairly common that can be pointed to in answer to this question.
First, it's important to note that this question not only applies to the Bible, but to absolutely any work of literature. If we were to read a biography of an individual, and the biography claims to be completely accurate, there may be a place in the biography where a person is accurately quoted as saying something that is completely inaccurate. Would this make the biography itself inaccurate? Not at all.
Additionally, idioms (cultural phrases with commonly understood meanings apart from the literal meaning) are used in most types of literature, and in general they are completely understood by the reader of the day, but removed from cultural context, these idioms often lose all meaning. For example, if I told you that yesterday I worked all day trying to resolve a particular issue, would you think that I worked for a literal 24 hours on that particular issue, from exactly 12:00 AM through exactly 11:59:59 PM? No one would think that that is what I meant. Would you even think that I meant a full 12 hours, from exactly sunup till exactly sundown, without stopping to eat anything or to even use the restroom? Well, no, you wouldn't think that either. In fact, if I said that "yesterday I worked all day trying to resolve a particular issue", you would probably think that I spent maybe 8 or possibly 9 hours, breaking of course for lunch and to use the restroom occasionally, although if you found out that I only worked for seven or maybe even six hours on it, you probably would not think of me as a liar. On the other hand, and this is really rather amazing I think, if in fact I worked from exactly 12:00 AM through 11:59:59 PM, an exact 24 hour time period, and all I told you was that "I worked all day on it", you would be perfectly justified in accusing me of misleading you! "You didn't just 'work all day'," you might say, "you worked a full 24 hours, down to the last second, with no break whatsoever!"
Sometimes, even in the English language, what we say is not really what we mean.
William Lane Craig provides another type of example that we commonly use in the English language. If I were to tell you that "he made as good a first impression on her as Mr. Darcy did on Elizabeth Bennett," I am in no way suggesting that the events and characters of Jane Austin's Pride and Prejudice are real instead of fiction.
My point in providing these three different examples in the English language is to demonstrate that you can't always take every sentence you read in a wooden-literal sense in anything that you read, including the Bible. So the basic question asked above, "how do you decide when to take the Bible literally?", can just as easily be asked of any other book, or even of any other communication in any language whatsoever. How do you decide when to take the newspaper literally? How do you decide when to take the President's televised speech literally? How do you decide when to take Colin Powell's autobiography literally?
So now to the basic answer.
Most of the time, quite honestly, it comes down to good-old common sense.
Context is a big determinant of this. Take for example Joel 1:4:
"What the gnawing locust has left, the swarming locust has eaten; And what the swarming locust has left, the creeping locust has eaten; And what the creeping locust has left, the stripping locust has eaten."
If we took this in a wooden-literal sense, there would be nothing for the stripping locust to eat because the creeping locust ate everything that was left. Yet there would have been nothing for the creeping locust to eat either, because the swarming locust already ate everything. So, if we take it in a wooden-literal sense, it makes no sense at all.
However, using common sense and reading the general context, it is fairly easy to deduce that the author of Joel is employing the use of a common literary device known as "hyperbole". Essentially, he is exaggerating reality to drive home the point about how truly devastating the situation was.
We find another helpful example in Genesis 18:15, we see Sarah, Abraham's wife, saying "I did not laugh."
If we take this to mean that Sarah obviously didn't laugh because the Bible says so, the meaning of the next portion of that same verse becomes nonsensical. After Sarah claims that she did not laugh, the Bible tells us that the Lord God replies, "No, but you did laugh." How can the Bible both affirm that she did laugh and that she did not laugh at the same time and in the same way? The answer is that it obviously does not. The author of this part of scripture is simply informing us of a conversation that occurred. One of the statements in that conversation was a lie, and the Bible faithfully records that lie for us.
With respect to idioms, things can get a little more difficult. Idioms tend to be culture-specific, and we are not in first century Palestine. Most of us, when reading the Bible, are not familiar with idioms from first century Palestine. So how do we recognize them?
Study, research, and good commentaries can help out enormously in this area for the serious Bible student. Additionally, many Biblical idioms appear many times in similar contexts. For example, to say that someone "slept with his fathers" is an idiomatic expression whose meaning is probably fairly obvious even without context. It means that person died. This phrase is employed all over the place in the historical narratives of the Old Testament (1 Kings 2:10, 11:21, 11:43, 14:20, 14:31; 2 Kings 8:24, 10:35, 13:9, 13:13; 2 Chronicles 9:31, 12:16, 14:1, 16:13, etc). When we see this same phrase appear many times in similar contexts, we can deduce that the phrase itself was a common expression, and often common expressions are in fact idiomatic.
The type or genre of literature also plays a part in determining whether the words and phrases should be taken literally. In apocryphal literature of the time, for example, like the Book of Revelation, it was standard practice to employ metaphors to stand in-place of the real meaning. This was common in any book of the apocryphal type, and so when we see apocryphal literature in the Bible, that is what we should expect -- lots of metaphors. The author of Revelation, the Apostle John, makes this much easier for those of us not familiar with this genre of literature by telling us at the very beginning of the Revelation that the things he is seeing are actually symbolic. After telling us about some strange things he saw in his vision, he says in Revelation 1:20:
"As for the mystery of the seven stars which you saw in My right hand, and the seven golden lampstands: the seven stars are the angels of the seven churches, and the seven lampstands are the seven churches."
So now we know for certain because John makes it undeniably clear even for people unfamiliar with apocalyptic literature... John is seeing things that merely represent other things.
With respect to parables, there are many told by Christ in the New Testament Gospels. They generally bear similar marks: the people are usually unnamed; the time frame is unclear; and the specific location is not important; there is generally a meaning or application to be applied to our lives clearly stated in the text itself.
Consider the differences between the following two passages of scripture:
Matthew 25:1-13
"Then the kingdom of heaven will be comparable to ten virgins, who took their lamps and went out to meet the bridegroom. Five of them were foolish, and five were prudent. For when the foolish took their lamps, they took no oil with them, but the prudent took oil in flasks along with their lamps. Now while the bridegroom was delaying, they all got drowsy and began to sleep. But at midnight there was a shout, 'Behold, the bridegroom! Come out to meet him.' Then all those virgins rose and trimmed their lamps. The foolish said to the prudent, 'Give us some of your oil, for our lamps are going out.' But the prudent answered, 'No, there will not be enough for us and you too; go instead to the dealers and buy some for yourselves.' And while they were going away to make the purchase, the bridegroom came, and those who were ready went in with him to the wedding feast; and the door was shut. Later the other virgins also came, saying, 'Lord, lord, open up for us.' But he answered, 'Truly I say to you, I do not know you.' Be on the alert then, for you do not know the day nor the hour."
What are the virgins' names? Dunno. What is the bridegroom's name? Dunno. When did this happen? Dunno. Where did it take place? Dunno. Is there a clear meaning or application behind it? Yes -- we should be on the alert then, for we do not know the day nor the hour when the Son of Man will return.
The passage above bears all the marks I mentioned of being a parable.
Now check out the second passage by comparison:
Luke 2:1-5
"Now in those days a decree went out from Caesar Augustus, that a census be taken of all the inhabited earth. This was the first census taken while Quirinius was governor of Syria. And everyone was on his way to register for the census, each to his own city. Joseph also went up from Galilee, from the city of Nazareth, to Judea, to the city of David which is called Bethlehem, because he was of the house and family of David, in order to register along with Mary, who was engaged to him, and was with child."
What are the relevant people's names? Caesar Augustus, Quirinius, Joseph and Mary. When did this happen? While Quirinius was governor of Syria. Where did it take place? Judea, specifically, the city of David which is called Bethlehem. What is the application behind it? Uh...
This does not appear to be a parable. The author pretty clearly wants us to take his words as though they are simply an accurate account of historical events.
So, to summarize, the ways that we determine which pieces of scripture should be taken literally and which should not are generally the same as how we would determine which portions of other literature or communications should be taken literally. The principles are the same, and when the differences in culture or language make it too difficult to tell simply by reading the text, a good commentary (like the one linked to below) can be very helpful.
Comments
Excellent. My common sense tells me that no divine being was involved in the writing of the bible. Common sense tells me that the bible was written thousands of years ago by ignorant desert dwellers who, through no fault of their own obviously, had no idea how the world or the universe worked, were very superstitious, and would believe any old "magic" stories because they didn't know any better.
Common sense...ain't it great?