Is there a difference when sneaking with heavy armor compared to light aromor or can you do it effectively either way?
Sneaking has a lot of parts to it, and yes, armor is one of them.
Light Armor is easier to sneak in due to weight. It is also much faster to get the weightless perk in the Light armor tree, which negates the affect of wearing light armor while sneaking. The light armor perk is called 'Unhindered' and is 3 perks deep, requiring 50 skill in light armor.
Heavy Armor is obviously, heavy. This makes it have a very noticeable affect on the ability to sneak while wearing it. The perk for weightless is deeper in the heavy armor tree compared to Light armor. The heavy armor perk is called 'Conditioning' and is 4 perks deep, and requires 70 skill in heavy armor.
However, in the end, both will be able to be used without an affect on sneak, assuming you get the perks.
I highly recommend you wear nothing (or just a robe), and wear sneak necklace/rings/potions and cast muffle(illusion spell) if you are having trouble with a specific area to sneak past. Just put your armor in the 'quick slots' and switch it off when you need to sneak until you get the armor perk of your choice.
Buy muffle spell and get about 150 mana and you can sneak in any armour: you'll make no sound at all. You can run while sneaking with no penalty, just like a silence perk. You can go ahead and get whatever armour you wish, but be sure to use a dagger or larger weapons will get you spotted: weapon weight still does count as a visual trigger. On the other hand, spells, like a fireball in your hand, will not give you sneak penalty.
You will have no need for unhindered perks in any armour. However, by not getting hitting at all you will have a hard time leveling your armour skill up, which means you will end up spending a small fortune on armour training. It's worth it: do not level armour, get muffle and use whatever armour gives the better enchantment, no matter if it's light or heavy... or just looks good.
Bottom line is you don't even need to spend points in illusion if you won't use your mana for anything else — muffle will take all your mana away, but it's worth it for a 3 minute protection.
Otherwise, level illusion to Adept and get all the "spells work on higher level mobs" perks so that you can calm opponents and then just go behind them, press control and backstab (you won't even need a high sneak, just the assasin blade perk).
This is an easy solution as you will spend like 4-5 points in mana and 100 gold or something like that for the muffle spell. You can buy it from the court wizard Farengar. In order to buy it i don't think there is a required illusion level to be shown at the store like most other spells.
The weight of the armor only affects speed of movement. Your sneak penalty is higher in heavy armor; you can hear the armor clunking around. Although if you use the Muffle enchantment, which makes your footsteps silent and reduces armor noise by 50%, and you get the Sneak perk Muffled Movement, which reduces it by 50%, you can practically make no noise so having light armor makes little difference.
Ultimately, it is up to whether you want 50% stamina regen boost or sheer defensive power. The choice is mainly in your combat style: do you stand and fight or dodge and manouvre yourself around? I personally play a heavy armor sneaker with sword and dagger. Although light armor is probably better, because using the Alchemy/Smithing trick, you can cap your armor rating at 567 so light armor would give you less sneak penalty and the same armor rating. The only Heavy Armor perk which is better is Reflect Blows, which gives you a 10% chance of reflecting damage rather than Deft Movement (10% chance of dodging the blow).
You should choose the armor that you think looks coolest and max out its armor rating. You don't need any perks in Alchemy to do it.
Heavy Armor and Light Armor skills both contain perks that reduce the weight of worn armor to zero, meaning you can sneak with no penalty while wearing full heavy armor sets.
This site tells you all you need to know about sneak, this is where i got the info above: http://www.uesp.net/wiki/Skyrim:Sneak
I wear master robes, the dragon priest mask that gives 100% extra magika regen, the Dark Brotherhood boots and gloves (light armor), the boots have a muffle enchantment, and the gloves give you double sneak damage with one-handed weapons, so with the assassins blade perk, my ebony dagger does 30x sneak damage! (no high sneak skill required)
On one hand I have my ebony dagger, on the other the expert level illusion spel;: invisibility. Using the Dark Brotherhood boots, enemies cant hear me, and thanks to the invisibility they cant see me, making it impossible for them to detect me, this way you can even stealth-assassinate dragons!
The invisibility spell does wear off when you kill someone, so it's wise to either train your illusion, or have a large amount of magika. I also suggest choosing the quiet casting perk in the illusion tree, allowing you to assassinate an enemy, turn invisible again, and assassinate another, without being noticed.
The other thing with Muffle spells and effects is that 100% muffled noise from Daedric or Dragon Plate is still louder than 50% of Thieves Guild or Shrouded Armour. Of course, Nightingale Boots of the highest level muffle armor sound by 350%, so those boots plus any armor plus muffle and an invisibility spell with the Quiet Casting Illusion perk renders you undetectable. I did this and found everything WAY too easy, even on Master difficulty, so I went back to 1-2 Sneak perks and put the rest into combat and pickpocketing/lockpicking. Self-gimping is one of the most rewarding ways to play, you are not overpowered and so long as you don't mind dying it will get you a lot of challenge and experience, even if you play on Adept (medium) dfficulty.
Heavy armor makes it very difficult to sneak, light armor also makes it a bit harder to sneak. Of course this is all based on the weight of the armor, so lighter armor would make sneaking easier. There is a perk in each armor tree that make the armor weigh nothing, which will make sneaking really easy.
Make sure to put perks in sneaking for added effectiveness.