While it is possible to do what you want, it is unfortunately not a very seamless process.
With physical (disk-based) games, this is no problem. Anyone can put any disk in any PS4 and play. It's much more complicated for digital games.
Unfortunately, accessing both your games on both machines will not be possible. Each of you will be able to ...