In X3: Albion Prelude, the settings are such that a joystick POV hat and trigger can be used to navigate through menus. However, every time I select an option that closes the menu, my ship fires a shot. Is this something I need to worry about? Considering that most of the time when I'm using a menu, I'm facing a friendly, am I accidentally shooting them? If so, is there a setting that will prevent this?
Friendly fire is definitely possible in this case, but the severity of it depends a lot on your reputation with the race of the target and how much damage you are doing. When your rep is fairly high, you'll have to do significant damage before they turn hostile. At that point, your rep will take a hit.
As an example, my pilot, of a rank of 'Federation Guardian' can take the shields entirely off of an argon TS before they start becoming hostile (this is done with 6x concussion impulse generators, so its really only a few salvos). In another example, I can keep up a sustained salvo of 5 seconds on an Argon laser tower before it turns hostile.
Short small bursts of fire from your fighter, even if it hits, will most likely be ignored. If you happen to hit a station, you will often receive a warning message telling you that continued hostility will be met with severe consequences, but even then it takes several salvos before they turn hostile.
Now, if you were flying a capital ship with 8 Incendiary Bomb Launchers, on the other hand, accidental fire may be much more destructive >:)
Last but not least, you can apologize for your accident. If you open up comms with a hostile target, you can often apologize by saying that it was an accident, and if your relations with the race is high enough, they'll turn neutral. This won't work with a station, however. To make a station neutral, you'll have to find a hacker to grant you access once again.
One final trick to turn a hostile ship neutral (assuming your rep with them is high enough). Is to simply scan it with a scanner (make sure you have a police license before doing so or else you lose reputation). This may or may not always work but its worth a shot.
As for figuring out a way to disable the accidental shooting with a joystick, asking this question on the egosoft forums is probably your best bet.
As the other answers have already clarified, you should worry about causing friendly fire if your weapons are powerful enough.
You can avoid this in most situations by using the weapon group feature as a "safety catch":
There are four weapon groups which you can select by pressing "1" through "4" on the keyboard. By default, group 1 will include all your weapons and groups 2, 3 and 4 will be empty.
If you are playing with a gamepad with limited buttons, you may want to map the weapon groups to long presses of the D-Pad or something.
You should be worried about friendly fire. It can cause you to get into hostilities when you are not trying to start a fight. Luckily, the game give you some tolerance so most of the time, a single shot won't cause you to suddenly be in a fight. The higher your reputation, the more "misplace" shots they will tolerate from you.
As you get more powerful weapons, accidental firing will become much more of a problem. Accidentally firing top of the line capital ship weaponry will be a hostile making problem. Although capital ship flying tends to be the bigger danger as your ship on autopilot just runs through anything smaller, turning your machine in a reputation ruining death juggernaut.
It is a good thing to prevent unintended firing. You can remap your controls so that they won't also include menu navigation and firing. You do this in the "options" "control" settings.
Remember to save often and save in different slots so you can always roll back to a good save point if you accidentally start a war when you didn't mean to.