As you have already discovered, using a nuke against enemy units is very difficult since a player is very likely to notice it. The trick is to only use it to harass enemy units when your opponent is otherwise distracted. When you do use it, it is best to use it against an area on the map he is not currently focused on. If your enemy is attacking you, launch a nuke on one of his bases (a successful nuke will deal massive damage to any of the bases, and destroy any workers as well).
However, I believe you may be narrowing your focus too much. The nuke is useful against more than just units, it is devastating to buildings as well. The following buildings will be destroyed from one nuke (source):
Pylon, Supply Depot, Extractor, Refinery, Tech Lab, Reactor, Photon Cannon, Bunker, Missile Turret, Sensor Tower, Spine Crawler, Spore Crawler, Creep Tumor
Of these, I would say the supply depot is the most tempting targets. Terran players will often clump up their supply depots together. One well placed nuke can put a huge dent in the enemies supply cap.
Finally, the defensive nuke. The defensive nuke will almost always be seen by the enemy, but that's fine. One of the most useful features of the defensive nuke is that it can hold off an opposing army, either giving your units time to retreat or finish building some units. Managing to kill enemy units with it would be ideal, but considering you need to place it in the opposing armies path for it to work, it's very unlikely your opponent won't notice it when moving his army.