I've tried knit caps and beenies without a pom and everytime I feel boyish. The pom on the top adds a girlish element and gives the appearnce of more body to your head. The hat can make your head appear flat and squished, so the pom serves the same purpose that teasing your hair or adding a high pony tail would. It also helps give you more height which is a huge plus if you are short.
This two toned one is the Charlie-Ivory Zig Zag Beanie from Gigi Pip but is currently sold out. I also love their Jessica Waffle Knit and the Mia Cable Knit with a Liner if you need more ideas.
I have a floppy brimmed hat and it works for summer but looks out of place in the winter. For both fall and winter styles, look for a flat brimmed hat. A wool felt material also helps it flow with your sweater and jacket combinations. My flat brim is the Pheobe from Gigi Pip and is the perfect size and material for the colder months. The Dotty and Della Grey (which is on sale!) would also meet this description.
Wearing a sailor top as an adult can make you look childish and silly, but if you are matching your child, suddenly it is the cutest thing ever! Same goes with hats- everything is 10x cuter when you and your child match.
Jack and Julie Co. Etsy shop can make any size and color combination for you. I special ordered these for Oliver and I. His is a toddler size in "greige" with a black pom pom and mine is adult size "deep green" with a white pom pom (slouchy style). She can also do fur poms instead of yarn to add a differnt touch!