I'd never really thought about it, but there's really no good reason to celebrate the new year when we do. It's important to keep a calendar that everyone follows for the purpose of communication and business, of course. (Although plenty of folks still use other calendars for various reasons, as the Wikipedia calendar page mentions in the comments.) Picking a New Year's Day other than January 1 wouldn't make sense now. But it had to start somewhere.
Gregorian calendar reform set January 1 as the standard first day of the new year, among other things.
Prior to that, there really wasn't a standard start to the year. March was the most common time that the new year was celebrated. That actually makes more sense to me - springtime feels like a more logical place to start a new year than the dead of winter. (In the northern hemisphere, anyway.)
On the other hand, January does have some logic behind it when it comes to the length of the day. The December solstice is the shortest day of the year, so the days are getting longer by the time January 1 rolls around. A good time to start the new year, as the days lengthen.
If the Gregorian reform had a slightly different thought process, we might be all sitting at home quietly every December 31. And staying up late every February 28th! (And sometimes 29th.) Regardless of how arbitrary the date may be, it's good to celebrate the passage of another year. Happy New Year, everyone!