Because they symbolize a "political-religious claim to power" -- which, of course, they do.
"Exit polls show Swiss will accept minaret ban: TV," from AFP, November 29 (thanks to Inexion):
GENEVA -- Exit polls show that Switzerland will accept a far-right call for a blanket ban on the construction of minarets, Swiss public television reported Sunday, describing the result as a "great surprise."
Partial results from the poll which closed at mid-day (1100 GMT) indicated that the German-speaking canton of Lucerne accepted the ban, while French-speaking cantons Geneva and Vaud voted against.
The Swiss People's Party (SVP) -- Switzerland's biggest party -- had forced a referendum under Swiss regulations on the issue after collecting 100,000 signatures within 18 months from eligible voters.
It claims that the turrets or towers attached to mosques from where followers are called to prayer symbolise a "political-religious claim to power."