This is possible from the beginning, I'm running around electrocuting enemies with my right hand and blocking with a shield on my left hand pretty much since the beginning of the game.
Left clicking assigns the item or spell to your right hand and right clicking assigns it to your left hand. So you want to right click on a shield and left click on a spell.
To activate the spell or block with your shield the mouse buttons are reversed from assigning, you activate your right hand with a left lick and your left hand with a right click.
Favoriting just puts the item or spell into a list you can access via the
Q key, you can equip from the inventory directly or from your favorites list.
It should look like this, where both the shield and the spell are marked in the inventory and favorite list: