I think that with silicon you're relying on the jubilee clip (or similar) to do all the work, whereas with rubber over time it starts to weld itself to the metal, so even if there's a problem with the seal from the clip, the fact that the rubber has joined forces with the metal will save the day.
That said, there's plenty of people who've been running silicon hoses without any issues. I've got silicon ones on (for about 9 months I think) and so far so good.