Silver enemies are guaranteed to drop at least one gem and sometimes they drop two, so they're typically worth the fight.
As for tactics to make fighting Silver variants easier, they are still susceptible to headshot stuns and double damage. You can stunlock Silver Bokoblins fairly easily with arrow headshots interspersed with your melee strikes. Lightning/shock-based attacks such as Thunderblades or Urbosa's Fury will stun Silver variants just as well as Red or Blue. You can also use shock to disarm them, reducing their damage significantly. And while we're talking elemental, Frost-based attacks will freeze them just as easily. An Ice Rod or Blizzard Rod makes any Silver variant a cakewalk.
If you have Stasis+, you can use that to lock your enemies. Though it won't last for very long, it's enough for you to get in some free hits or get out of the way of an attack.