What factors should I consider when looking into Travel bottle warmers. I currently travel with a regular sized one I use at home, but it gets too annoying having to carry it around.
The factors that are important to you! But seriously:
Obviously as a bottle warmer it needs to warm milk. Since it's warming it for a baby, it needs to not make the milk too hot, which requires either a timer or sensor. Assuming you aren't carrying around a generator, it needs a power source, probably a battery. You explicitly mention weight, so that's a factor to consider. If you have finite monetary resources, the price has to be considered. Consider where you'll be using it; an unstable warmer that requires a perfectly flat surface may not be practical when out and about. You also mention comfort when carrying, that implies the need of an ergonomic handle or gripping surface.
My advice when buying any product is to write its function, a verb, and the types of matter and energy it interacts with, nouns. Then establish requirements and preferences for how these nouns go through the verb process. I could continue walking you through this exercise as above, but no one has a better idea of your needs than you.
I suppose this is basically the modern design engineering process for idea generation applied to evaluation, but it's proven in the other capacity and should work well here too.