A Simple Ideological Inconsistency

When I am trying to explain how it is an okay thing for a person to be attracted to very specific things, I tend to say that 'You can't control what you like'. But when I remember Epictetus' Enchiridion, I repeat the mantra, that how you feel is the only thing you control. I still have to figure out how I really feel and come up with a model that reconciles these two beliefs.