We've done both. Seems like the earlier days were less busy but that depends on when the holiday falls. Like this year, Jan 1 is a Wednesday, so maybe people are staying throughout the rest of the week (using only 2 vacation days from work) and maybe even through the weekend. But I bet on Jan 6, that Monday, you would see a bit of a break in the crowd levels before the Marathon people start coming in for the following weekend.
Like others said, every possible week in WDW has something going on, and something to think about.