Well, the first thing I would say is to keep in mind that December is the high season for travel all over Latin America. So hotels/hostels, flights, etc. will be a lot more expensive and popular travel destinations will be a lot more crowded than usual. But you probably already know that.
I second San Andrés as a fantastic beach destination. It's really, really beautiful and while it has a fair number of tourists, it's not nearly as touristy as places like Jamaica, Grand Cayman, and other Caribbean islands that I have been to. It certainly doesn't cater to the spring break crowd or anything like that. I've been all around the Caribbean, I've been to tons of Mexican beaches, I've been to Hawaii, but never have I ever seen the ocean look like it does in San Andrés. The colors are astounding. The food is great, too, and there are a lot of very affordable places for excellent seafood (conch, mmm.) You have to fly there from mainland Colombia, though. But San Andrés is only a two hour flight from Bogotá, if I remember correctly, and the Bogotá airport is really nice.
If you don't want the hassle of getting to an island but still want a beach, Cartagena is spectacular (but pretty crowded in December). You have to take a ferry to get to the nice beach, but the walled city is magical. And in December there are pretty lights everywhere. It's also an awesome place to spend New Year's Eve.
As for a non-beach destination for December? Actually, anywhere else in Colombia is great because that whole month is one long celebration. Everyone is out and about and doing stuff, and there's just a really nice vibe in the air. My Colombian husband only spent one December in Argentina and after that swore he'd never do it again, and now after having lived in Colombia, I totally get it. Every city has its traditions, but most people tend to agree that Medellín is the best in December. The alumbrados (Christmas lights) are the best in the country, the people are delightful and you won't lack for tasty things to eat, whether you want to stuff your face with chicharrón or organic vegetables.
I swear I don't work for the Colombian tourism board hahaha. But seriously, if OP or anyone ever plans to travel to Colombia and has questions, don't hesitate to PM me.