Possible Spoilers
So after having finished what I believe was the 'Call to Arms' mission, I automatically received the Semper Invicta mission, which has me talk to Paladin Danse at the police station. There, I spoke to him, and he asked me if I wanted in again (I told him I didn't want to join, as I liked Nick Valentine and didn't want Nick to dislike me), and I said no. He said ok, and the target above his head didn't disappear. Does the mission want me to join the Brotherhood of Steel?
Yes, you will need to join the Brotherhood of Steel to complete this quest. There is no way to finish Semper Invicta without becoming a member of the Brotherhood of Steel.
From the wikia for Semper Invicta, the walkthrough states:
After the discussion, Danse asks the Sole Survivor to help both Rhys and Haylen doing their respective job.
- Cleansing the Commonwealth (repeatable)
- Quartermastery (repeatable)
The quest will complete once the requisite quests have been completed (either one). The requisite quests are repeatable whether or not this quest has been completed.
As noted by DCShannon, the part of the quest where you officially join the Brotherhood is the "discussion" part of the quote from the wikia.
The requisite quests that Rhys and Haylen give you are official Brotherhood of Steel assignments, which you can only complete as a member of the Brotherhood of Steel. If you choose not to join, you cannot have these quests given to you, which prevents you from completing the quest.
