Yes, it's possible to complete the game with a sign heavy build. You'll still be using your sword, but with heavily developed Aard and Igni signs, you can get through most of the game with pretty minimal swordplay. Igni in particular is extremely potent at high levels.
Again, you won't be able to completely neglect your blades, but they become more a means of finishing off weakened foes, or a way to do a little damage while waiting on endurance to recharge, rather than your primary means of damage. Fortunately, you have enough Bronze talents to basically take everything, and enough Silver talents to maintain pretty reasonable competency with a few styles - Strong Silver in particular, synergizes with Igni brilliantly.
Be warned, going Sign heavy will likely increase your dependence on Alchemy somewhat - potions are a must for keeping your Endurance up.