Shouldn't you chill the mixture following getting it off the warmth and before putting it into your ice cream device? A typical flavor that will always be a favour