The difference between the two is that they are used in two different "modes" of the game, with a graphics set having much finer-grained control over the visual display.
The original, default display mode for Dwarf Fortress is to emulate the look and feel of a traditional ASCII rogue-like, by using only the 128 standard characters from the ASCII character set plus 128 "extended" characters added by IBM to their Code Page 437, which most MS-DOS-based PCs used (and most English-Language Windows PC's still have as their "OEM code page").
Technically, in its current form, it's not really an "ASCII" game -- it's not printing text to the screen based on the system's code page. Rather, it's drawing glyphs from a "character set" file, in which the default glyphs exactly match the IBM CP 437 set:
However, you can replace these with your own character sets, by swapping out cells in the image table with your own. They don't need to look like the CP437 glyphs, but the position in the table still corresponds to the ASCII value used by the game, meaning even the rendered text will change if you do this:
Note, in this image, some of the punctuation, plus the upper-case X, look different than you'd expect. In places where the game prints strings of text, those changes will show up. (This is a problem with some character sets that replace lower-case letters with other things, making it sometimes hard to read the text.)
Alternatively, you can switch the game into graphics mode by telling it to use a graphics set, but only for creatures. In this mode, the game no longer just picks one of 256 glyphs to display things with, but rather, uses a much broader set of tiles that reflect more of the characteristics of a given creature. In this case, the creator of the graphics set provides one or more tables of graphics tiles, plus a set of index files that tells the game where to find a tile for any given "raw element". For example, the popular Phoebus graphics set has a 12x21 table (not entirely full) just for humans:
and similarly dense tables for goblins and dwarfs, plus a few extra ones for other creatures/monsters/animals/etc.
It's important to note that graphics sets can only replace the tokens used for creatures. When rendering things like plants and buildings and objects, the game still uses the glyphs from it's character set, so typically, you would have both a graphics set and a character set, from the same third-party source, in use at the same time.