There's a number of steps you should take in order to deal with this issue:
First, are you using any mods? Any mods that interact with workshops/settlements/the tunnels beneath the castle are known to interact poorly with this quest. Disable any mods you might be using and see if that helps.
Second, have you cleared all Mirelurks and Mirelurk nests within the castle? Some of the nests are in hidden in strange spots--around the back parapets, in the bedrooms of the castle--and if your NPC allies tried to clear them out on their own sometimes not all of the hostile eggs within the nest will trigger. The Castle won't be considered allied to you until every nearby hostile is cleared out, including hostile Mirelurk hatchlings still in their eggs, so make sure everything's dead.
Third, there is a known glitch where you get the 'not allied to this settlement' message when attempting to interact with the workbench even after anything is dead. You can see discussion of it here and here at the Steam forums for the game. (It seems that this glitch may come from an enemy falling beneath the map, or the game not considering all enemies dead even when they are.) On PC, players have reported success in fixing it by using the 'killall' command; however, since you're on PS4, if you are in fact seeing this glitch your best bet is to reload from an earlier save and re-attempt the quest. There may also be a mod available on the PS4 that gives you access to the console to allow you to try console commands as a fix--I'm not sure what the current state of PS4 mods for this game is.