There are multiple ways of increasing your "rarity chance" either via gear that has that effect or via the "Item Rarity" support gem. Now what kind of items will that actually affect? Will that increase the chance for getting yellow and unique items? Does it increase the chance of getting skill gems and orbs?
From the mechanics thread:
Increased Item Rarity Increased Item Rarity % modifiers increase the chances of an item being magic, rare, or unique. For example with a total of +100% increased item rarity, you'd get twice as many magic items, twice as many rares and twice as many uniques from normal enemies. This modifier has no effect on the number or type of currency items, scrolls, or gems that drop. When in a party, only the modifier from the player who lands the killing blow on an enemy is counted. If one of your minions gets a kill, the minion's IIR is added to yours and the total is used.
Magic, rare, and unique monsters have an Increased item rarity modifier for drops.