Caprice, freak, fancy, whim, whimsy, conceit, vagary, crotchet are comparable when denoting an arbitrary notion that usually lacks a logical basis and therefore may be unsound, impractical, or even irrational.
Caprice emphasizes the lack of apparent motivation and implies a certain willfulness or wantonness.
Freak suggests an impulsive, seemingly causeless change of mind, like that of a child or a lunatic.
Fancy stresses casualness and lack of reflection in forming an idea and may sometimes suggest a kind of harmless perverseness in the idea formed.
Whim and whimsy suggest not so much a sudden as a quaint, fantastic, or humorous turn or inclination, but whim often stresses capriciousness, and whimsy fancifulness.
Conceit suggests more strongly than whim or whimsy the quaint, fantastic, or erratic character of the notion formed and also may suggest the firmness and persistence with which it is held.
Vagary suggests still more strongly the erratic, extravagant, or irresponsible character of the notion or fancy.
Crotchet implies even more perversity of temper or more indifference to right reason than vagary; it often is applied to a capriciously heretical opinion on some frequently unimportant or trivial point.