You need to pull back for 1 second to get a Full Length arrow off.
If you pull back for more than 5 seconds, your hand starts to wobble and you will lose accuracy (I'm guessing this is what you are experiencing when you say that sometimes it doesn't go in a straight line)
So in order to get a good shot off, you need to charge it for 1 second (which is what the charging bar shown is) and fire the arrow before it starts wobbling (which occurs 5 seconds after being fully charged)
If you find yourself with a charged shot longer than 5 seconds for whatever reason, you can switch out your weapon to reset it, or better yet, press right click again to 'unload' the shot (does not waste ammo)
No matter how you shoot it, the arrow is always affected by gravity, but even more so if you do not fully charge the shot (from memory the shot does less damage for shorter charge as well) so make sure you always charge for at least 1 second