Why does the poet look down the road as far as he can see in "The Road Not Taken"?

This well-loved poem by Robert Frost uses a narrator's journey to represent decisions in life. As he walks in a "yellow wood," the narrator comes to a fork in his path and must decide which path to take. These paths in the woods represent two different choices, each which looks fairly "equal" upon first inspection.
It is at this point that the author says he "looked down one as far as [he] could / To where it bent in the undergrowth." In effect, the author is trying to determine where this path will lead him. He is imagining the possible outcomes of this path. He envisions how difficult the path will be. But, just as in life, it is impossible to know how the path will turn out because it eventually "bends in the undergrowth," leaving the vast majority of the path unknown. Since he realizes that there is not much chance that he will ever revisit this particular fork in the road, it is important that he make the right decision. So he pauses to reflect, predict, and collect his thoughts.
After all this speculating, the narrator decides to "take the other, as just as fair, / And having perhaps the better claim." For whatever reason, he decides that the road he pondered isn't the right choice for him after all and feels confident that he has made the best decision.
(Or does he? But that's an analysis for another day . . . )