Neither and both. Confusing?
Here's the deal: Minecraft uses quite a deal of processing power, both with the CPU and the GPU. Texture packs are handed by the GPU.
Now consider the following: Loading an 8x8 texture is 1/4 of the load of loading a 16x16 texture (that is the default size). However (and here's the catch), despite the less work, you would only notice any difference if the GPU was bottlenecking the process. In other words, if you weren't running at top graphical performance, you may see some small difference.
The reasoning here is also valid for HD texture packs: the bigger the texture, the more load the GPU is subject to and your performance will drop accordingly.