I'm a little late to this, but this is what I've had work for me:
In Minecraft 1.8+ we have the /testfor command, you can use this to scan the players Inventory slots (all of them, or specific) for items:
example: diamond
/testfor @p {Inventory:[{id:"minecraft:diamond"}]}
to test for an exact amount of the required item include:
,Count:#b
within the confines of the "id" brackets
however, this will only search for players with a stack of the exact number you select, anything more or less will not fit the criteria; so for this application it's not useful.
For your vending machine to work properly I would recommend having this command in a clock nearby that tests for players in a specific range, so the finished command would look something like this:
/testfor @p[x,y,z,r=2] {Inventory:[{id:"minecraft:diamond"}]}
(xyz are directly in front of the button)
have this connected to an output command block with a setblock command:
/setblock # # # unpowered_repeater
dv for output direction needed from button
with a redstone dust on top connecting to a torch that powers this command:
/setblock # # # air
to clear the space if the criteria isn't met
then your repeater needs to be touching these commands:
/clear @p diamond 0 3
/xp amount @p
Image:
There are different variations you can have for the setblock changer, this is only a simple version for proof of concept.
clear
I think takes a parameter that allows you to remove x items instead of all of them. You can also test for a minimum number of items, so say a player needs 3 diamonds in order to buy something, you can test for that.