Can I refuse to contribute to a gift for a colleague I dislike?

With bank transfers for gifts increasingly common, slyly avoiding contributing to a colleague you don’t like is a sensitive and difficult prospect.