How do the major factions (federation, empire, alliance) expand? If a given power (one of the eight "senators" etc) conquers a rival system does it count for their faction? Do combat zones outside the realm of the 8 powers affect the territory of the 3 major factions? Can powers from the same faction be each other's enemies?
You are asking multiple questions which would, together, take quite a bit to explain. I will try to sum up a quick answer and would like to direct your attention to the manual as PDF, specifically the Powerplay section (PDF page 75, manual page 112) to get in-depth answers to your question.
How do the major factions (federation, empire, alliance) expand?
They expand by having their member powers expand. Powers expand in three phases, each which takes at least a cycle (typically a week) to complete: Preparation, Expansion, Control.
In the preparation phase a potential system must meet specific requirements (population, no other faction direct control or exploitation) and the power must have enough resources to expand. Then a potential system must make it to the top 10 of prepared systems to start an expansion attempt. During the attempt several criteria must be met. If the attempt is successful, the system becomes, in the following cycle, controlled.
If a given power [...] conquers a rival system does it count for their faction?
I do not believe it is possible to directly conquer a rival system. That system would need to loose its current power's control or exploitation via the turmoil and revolt mechanics (basically the reverse of the above) to become available for expansion (as above). However, it might be possible during a war scenario but I haven't seen it happen (yet).
Do combat zones outside the realm of the 8 powers affect the territory of the 3 major factions?
They might as the combat zones might be linked to a power's expansion attempt. Check the expansion attempts for the powers to be sure.
Can powers from the same faction be each other's enemies?
Yes and no. They are rivals but not so much enemies. Rival powers can, for example undermine or oppose expansion attempts. To quote the manual, page 112: "[...] Ultimately, every power stands alone."