Based on the way you are talking, it sounds like you are fighting with a single Pokémon, in this case, you will never be able to "take" the gym, as it's already claimed for your team. Gyms have two "modes":
Fighting a Gym of an Opposing Team
When fighting a gym with a different team, you will be asked to select up to 6 Pokémon to battle with. In this case, when one of your Pokémon are defeated, the next in line will jump in, and you will continue the fight.
For every Pokémon you defeat in the gym, you will lower the prestige rating of the gym. Lower it enough, and the lowest ranked Pokémon in the gym will be kicked out. You can continue this until all Pokémon are removed, and the gym is neutralised. At this point, you will be able to claim it by putting one of your own into the gym.
Fighting a Gym beloning to your Own Team
When you fight a gym belonging to your own team, you are asked to select just a single Pokémon. Using this solitary Pokémon, you will then battle all the trainers in the gym. For each one you defeat, you will raise the Prestige level of the gym by a certain amount.
Once you have raised the prestige level enough, another trainer slot will open up (up to 10 max). At this point you, or another member of your team, will be able to add one of their Pokémon into the gym, further strengthening it to attack.
Because you only have a single Pokémon to increase the strengh of the gym, it is a good tactic to put claim it with the lowest CP Pokémon that will be put in. This will allow everyone to raise the rank further. Additional Pokémon should ideally be added in increasing order of CP.
You will personally only be allowed to add a single Pokémon to each time at a time. So if you wish to have several in a newly claimed gym, you will need to have other members of your team put theirs in too.