While I should study Switzerland political structures a bit more, I don't think it is any kind of utopia.
It is my understanding the referenda are more local in concept, not national. And while local assemblies allow just about anyone to attend, the participation is far from universal.
There is is a Swiss parliament, complete with politcal parties and the limitations they bring to the deliberative process.
One of the drawbacks of referunda is that when these are passed into law, there is often a lot of conflict with previous legislation. This leaves the civil service in charge of deciding which laws will be enforced.