It's odd that Christians have such a clear mental image of the birth of Jesus. They'll build little Nativity scenes with tiny stables, mangers, and "wise men" to display on a table in the living room, assured that their reproduction of the scene is in accord with the accounts given in the Bible.
What's odd is that the mental image comes from mashing together two incompatible accounts of the circumstances, those given to us in the gospels of Luke and Matthew. (Mark and John attach little or no significance to the birth of Jesus -- Mark because he seems to have God adopt Jesus as "his son" at his baptism, and John because he has Jesus sitting beside God in the heavens since the beginning of time.)
I'll skip over most of the discrepancies between Matthew and Luke on the subject of Jesus's birth -- that they give conflicting years for the birth, a full decade apart, for example, and give completely different lines of descent from King David to Joseph, Jesus's (step?-)father -- and will focus on just one: the confusion involving Bethlehem and Nazareth.
Luke and Matthew had reasons for wanting Jesus to be (1) a resident of Nazareth but (2) born in Bethlehem, in order to fulfill "prophecies" of the Messiah. Bethlehem needed to be the birthplace due to its status as the "city of David," David being the king of the Jews from whom the Messiah was supposed to be a descendant, but Jesus needed to be "from Nazareth" because of a prophecy that Matthew claims identifies the Messiah as "a Nazarene." Yet neither Nazareth nor "Nazarene" is mentioned anywhere in the Old Testament, and no Bible scholar has identified the source of the "prophecy."
In any case, it seems that Luke and Matthew, each inventing their own Nativity narrative independently, came up with very different ways to make those prophecies work.
Luke, in the narrative that has become the more familiar, has Joseph and Mary starting out as residents of Nazareth and traveling temporarily to Bethlehem for the birth; Matthew, almost unnoticed by any Christians, has Joseph and Mary being permanent residents of Bethlehem, being forced to flee following the birth, and settling in Nazareth afterward. The birth taking place in a stable in Bethlehem, in a manger, occurs only in Luke's version, and contrasts with Matthew's, in which the family lives in a house in Bethlehem rather than temporarily visiting the city. The "wise men" appear only in Matthew's, and their presence, according to Matthew's description of their travels, is incompatible with Luke's timeline of events.
Let's look more closely at the narratives, starting with Luke. In his story, Caesar Augustus decreed that "all the world" (the part of it controlled by Rome, anyway) should be counted in a census for tax purposes -- and one requirement in the census was that every head of household should return to the town in which his ancestors lived. While occasional censuses did take place in that era, there is no record, in any document outside Luke's gospel, of a census of the entire Roman world all at once. There was a census of Judea, newly established as a Roman province, in the year 6 CE, at the outset of the governorship of Quirinius (as Luke mentions), and that would have included Bethlehem. But if Joseph was living in Nazareth, as Luke claims, that census would not have applied to him, since Nazareth was in Galilee, not Judea, so there was no reason for him to leave Nazareth. And there is no record of any census anywhere with such a "go home" requirement, which would entail such massive uprooting of families that it could not have escaped the attention of historians, and there is no conceivable reason for attaching such a condition to the counting: governments taking censuses didn't do it out of idle curiosity to find out where people were from.
So Luke's "census of all the world that required everybody to go home" is a matter of pure literary invention, for the purpose of forcing Joseph to leave his Nazareth home and take his 9-months-pregnant wife (and there is inconsistency in various translations as to whether he was actually married to her yet) on the arduous journey.
Matthew, in contrast with Luke, has Joseph and Mary living in a house in Bethlehem at the time of the birth, not arriving and desperately looking for a place to stay. There is no mention of Nazareth at this point at all. As for the "wise men" of Matthew's narrative, who don't appear in Luke's story (some local shepherds showed up instead) Matthew describes them as "Magi" from "the east." The term indicates they would have been Zoroastrian priests, who would have had to travel hundreds of miles, from Iran, a journey on which they did not set out until after the birth. And on that journey, which would at the very least have taken weeks, they didn't go straight to Bethlehem, but instead went first to visit King Herod (dead for a decade at the time Luke sets for the Nativity), thinking, it seems, that Herod would be thrilled to hear that a new "king of the Jews" had been born. (That Herod was NOT thrilled is the one single believable part of either narrative.) In any case, even if the Magi hadn't made the side trip to see Herod, Luke's version would have had Joseph, Mary, and the baby long gone from their temporary lodgings in Bethlehem by the time the Magi got there.
Matthew's imaginative invention for getting the family out of Bethlehem was having Herod decide to order the murder all of the babies under age 2 in Bethlehem (the age limit is another indication that Matthew pictured the Magi as arriving long after the birth). Of course, such an infanticide covering an entire city would have made the world take notice, yet, strangely, it went unreported by anyone in the world but Matthew. Even the famed Jewish historian Josephus, who was not at all a fan of Herod and went into detail chronicling the many atrocities committed by Herod during his reign, makes no mention of this one.
Of course, Joseph and family escaped the murder of babies, because an "angel" in a dream told Joseph to leave town and go to Egypt. Why Egypt? Because Matthew wanted to take the opportunity to have Jesus fulfill yet another prophecy: when Joseph was told by another angel that it was safe to return from Egypt because Herod had died, Matthew claims that Joseph's trip back home satisfied the prophecy in scripture (Hosea 11:1), "Out of Egypt I called my son." The problem here is that if you read the full verse ("When Israel was a child, I loved him, and out of Egypt I called my son"), and continue reading the verses following it, it is clear that the verse has nothing to do with a Messiah, but is instead a reference to the Exodus: that is, it's not a prophecy at all, but instead a poetic description of a past event. (That is a common feature of most of the "prophecies" cited in the biography of Jesus: they are actually about something or someone else and often aren't even prophecies. Or in some cases can't be found in scripture at all.)
After Joseph and family come back from Egypt, Matthew has Joseph afraid to go back to Bethlehem because he didn't trust Herod's successor (and yet another angel warned him), so, as Matthew puts it, "[Joseph] went and lived in a town called Nazareth. So was fulfilled what was said through the prophets, 'He will be called a Nazarene.'" Not only is there no previous mention of Nazareth in the narrative, there is no hint now or earlier that Nazareth was a place where Joseph had ever lived before (nor, as I said, any indication of which "prophets" Matthew might be referring to). If Nazareth rather than Bethlehem had been Joseph's home (as Luke has it), then Matthew would have not have needed to give an explanation for Joseph going there, certainly not one that involved who the new king was and angels giving warnings.
So just as Luke had whipped up a historically insupportable scenario to explain Joseph's travels, Matthew's entire set of events -- the involvement of Herod, his edict against Bethlehem babies, and the flight of Joseph and family to Egypt in consequence -- are all a literary invention to get Jesus to return "out of Egypt" and start living in Nazareth instead of the hometown of his birth, Bethlehem. And Matthew and Luke clearly disagree on where Joseph lived to begin with.
Interestingly, the publishers of my copy of the Bible (New International Version) helpfully added section headings to the various separate passages of the text. The two paragraphs about Joseph fearing Bethlehem and settling in Nazareth are headed "Return to Nazareth." In the absence of any indication at any point in the story that Joseph had ever been there before, and Matthew's wording at this point suggesting he hadn't, the heading seems a clumsy effort to somehow make Matthew's story of Joseph's travels blend with Luke's.
So each account of Jesus's places of residence includes details that aren't compatible with the other one, and each contains crucial motivating events (the "go home" census in Luke, the baby murders in Matthew) that never happened.
Matthew and Luke created their stories out of pure imagination, with only one common requirement: that they should explain Jesus being born in Bethlehem and ending up in Nazareth. That they came up with completely different solutions is not surprising. But they didn't realize these two accounts would end up in the same book, so that people could compare them and say: Wait a minute.
Yet Christians never do.