I have an Xbox one and my brother has an Xbox 360. I couldn't figure out how to join his party or invite him to mine, or even initiate a private chat. Is this possible or am I missing something?
No, you can't voice chat between Xbox One and 360. There was a Q&A session with Microsoft on IGN in August when this was confirmed:
Is there cross-console Party Chat? i.e. since it’s all “Xbox Live,” can people on Xbox 360s Party Chat with their friends on Xbox One?
WHITTEN: For a variety of technical reasons – including the sound quality that I talk about below – we don’t support voice chat between the two systems. You will still be able to send and receive text messages to friends on both systems, however.
This doesn't say anything about joining parties across consoles, however given that cross-console voice chat doesn't work I wouldn't expect to be able to join parties across consoles either.
You can send text messages across consoles, however.