In the case of Guns, not meeting the Strength requirement manifests itself as a significantly increased spread on your shots - in other words, Accuracy is substantially reduced, especially at longer ranges. Similarly, when attempting to aim the weapon using Iron Sights, Scopes, or Zoom, you'll notice substantially more 'wobble' in the camera as you struggle to manage the weight of the gun. The latter effect also manifests when attempting to use guns for which you lack the appropriate skill level. Furthermore, Damage is reduced.
For Melee weapons, Str requirements that aren't met affect attack speed, while skill requirements affect damage output, and access to 'special attacks' in VATS that do additional damage, or have additional effects.