They do use different game engines. Black Ops uses a modified IW3 engine (originally used in CoD4 and WaW), while MW3 uses the IW5 engine. One of the major differences (in multiplayer) - taken from here is that movement speed is determined by the currently equipped weapon in Black Ops, while it is determined by the primary weapon upon spawning in MW3.
Black Ops allows you to dive to prone while sprinting, while MW3 does not. Each game has different materials that can be penetrated by bullets.
One difference a lot of people notice/complain about is that Black Ops runs at a lower fps than the Modern Warfare series (which runs at 60 fps).
As far as offline play goes, the campaigns are obviously totally different - different styles of storytelling, different objective types. MW3 does not have Zombies, as already mentioned. Survival Mode vs. Bots in local matches is very different too. Bots allow you to play TDM/FFA against up to 9 bots for standard games. Survival Mode runs until you die - there is no end to the game.
Many people will notice different differences, so some things are obviously a matter of opinion, but these were a few changes that stand out commonly.