It's just very, very tricky. The rule of thumb is to watch the animation. The common mistake is to try and link too soon - you literally have to wait until the last animation has finished. Also, some moves just don't link. The timing windows in SFIV are really strict - which allows for a real range of skill levels.
Generally speaking, to get long combo strings out, you need to work more with lights and mediums - so you might jump in with HK, standing LP, crouching LK, crouching LK
There are several methods for linking moves:
Cancelling: Most common, where you
break out of a normal by executing a
special. Same goes for specials to
supers, and supers to ultras.
Cancel: Where you execute a move but
hit the focus attack during it to
cancel, generally used in the opening
of a special
Kara Cancel: you start a
move, but hit throw during it, allows
you to throw from extended range.
Have yet to pull this one off
Plinking: Hitting a descending
strength of attack almost
simultaneously - i.e hk -> mk
Here's a combo for you to try that should give you a feel for it
Jump in - HK
LK Spinning kick (juggle)
HP Dragon Punch
* you can focus cancel the Dragon Punch and try a fireball if you're up to it..
You'll get a good six hits out of it.