I've found weapons that I'm not specced into that seem to be significant upgrades to what I have. What kind of negative effects are associated with wielding a weapon you aren't skilled with?
There are no longer any negative effects for using a weapon that you are unskilled with, except that you will not get the benefits from your weapon skill. It used to be that you got a debuff from using a weapon without that weapon's skill, similar to the debuff for dual wielding without the proper skill, but that was removed in an earlier update.
I have actually begun not choosing any weapon skill, as it allows me to choose another useful skill, and I find that the bonus of being able to use any weapon without having to worry about losing skills more than outweighs the loss of bonuses from the weapon skill.