Silence removes all noise that your character makes from movement while in stealth mode. Effectively, this means that you must be seen (or smelled) to be detected. It is an almost entirely worthless ability, unfortunately, since the Muffle illusion spell (and the corresponding boots enchant) provide the exact same ability without wasting a perk.
While the Silence perk is a poor use of your limited points, the muffle effect is absolutely invaluable to a stealth-oriented character. Combined with the Invisibility illusion spell it allows complete undetectability to the vast majority of creature types, and it allows you to pickpocket and move around in close quarters far more easily than you could otherwise.
In general, stealth has been dramatically nerfed in Skyrim compared to previous Elder Scrolls games, and close attention to shadows and line-of-sight is now extremely important to avoid detection -- the old "sneaking in broad daylight" doesn't work now at any skill level with any amount of gear or perks.
From my personal experience, animals (and dragons) appear to have a significantly higher chance to penetrate stealth (and invisibility) than humanoids; presumably this is a "keen scent" ability of some kind.