According to the Official Wild Shadow Forums:
Guild chat is a cross-server message system only seen by members of your guild. Messages in guild chat will appear as green text; to type your own messages into guild chat, type "/guild [message]" or "/g [message]".
Guild Fame is accumulated by getting fame on guild members. It can be spent on guild hall upgrades.
The guild hall can be accessed from the lower part of the nexus, directly to the left of the vault portal. In the guild hall, there are a few important locations:
- Guild Chronicle: From here, you can view the current members of the
guild, and the guild fame they have contributed. The Founder,
Leaders, and Officers can change their ranks and remove them from the
guild.
- Guild Hall Upgrade: This is where the Founder and Leaders can
spend guild fame to upgrade the hall
- Guild Board: This board contains a large text field which can be
viewed by any member, and edited by the Founder, Leaders, and
Officers.
Apparently, the Guild Hall Upgrades are even more ridiculous than the cost of forming the Guild itself; the first two upgrades cost 10,000 Guild Fame and 100,000 Guild fame.
Each successive upgrade will increase the size of the guild and add additional cosmetic features, such as rugs, weapon racks, tables, etc.
Other than that, it doesn't specify any other sort of benefits. Though, the inclusion of a Guild Chat that goes cross-server can be pretty beneficial in the long run, I'd say.