For a holocall to work, there needs to be someone to start the conversation, but then the others can participate from wherever they are -- I think they have to be out of combat at the time, and I believe there's code to stop you from being attacked while you're in a holocall -- but I haven't tested those.
It is worth noting that there are various dialogue options that you CAN'T do while on a holocall -- for instance, anything that involves handing the target something you obviously can't do while you're not there, so it will be grayed out. As a consequence of this, there will be some points in a holocall where you literally cannot do anything, and will need to wait out the timer in order for the conversation to proceed.
You still get the social points, though.