A basic troubleshooting guide(by no means conclusive, and based on personal experience):
For determining the actual cause, try changing settings, disabling/enabling mods, and seeing if anything changes. If it changes for the better, you're going in the right direction. Minecraft has so many different aspects, it's very hard to figure out the exact problem.
Step 1: Know your mods. Certain mods will interact badly with each other, e.g. Logistics pipes and EE2, when hooked up together, make a lot of lag.
Step 2: As you mentioned, RAM is important. Everything depends on how many mods you are running, how many players you have, how many dimensions and chunks etc. If you haven't dedicated enough RAM, or have used the wrong arguments, you'll be slow on everything. For reference, the right way to dedicate the RAM is:
-Xmx[amount of memory maximum, followed by the measurement of it, e.g. 4G, 512M] -Xms[amount of memory minimum, same format]
Step 3: As previously mentioned, certain devices can cause lag. Get notifications on your server saying that 'lag causing devices will be removed, and the player who owns them may be punished', or something to a similar effect. If this doesn't work, consider a world reset.
Step 4: If none of the above options are effective, consider investing in better hardware, or a server hosting provider with a good reputation and non-ripoff prices. CreeperHost is apparently very good, but I've never personally used them. Good luck!
Side note: a Hexxit server on a Pi? You can run a vanilla MC server on a Pi without too much difficulty, but Hexxit... Not optimistic.