PlayStation and PlayStation 2 memory cards, along with the Memory Stick Duo cards used by the PSP, all use flash memory which is a form non-volatile memory that doesn't need power to maintain its state. So they won't lose their saves the same way your old GameBoy cartridges did because they don't have batteries that can go dead.
However nothing lasts forever and there are two ways your memory cards could end failing through normal use, even if you take the utmost care of them. The first is that since they have metal contacts like cartridge games, these contacts can wear out from repeated use just like they did with cartridge games. You may have heard stories of people blowing into their old Nintendo cartridges to get them to work. Do not do this, or do anything else to try to clean the memory cards, it will just make things worse. In practice I've never heard of memory card failing this way. Along with newer game cartridges they're probably manufactured to better stand up to repeated insertions and removals than those old game cartridges. A common number of mentioned on the Internet is 10,000 insertion/removals for an SD card, which means it'll last 27 years if you plug it in and pull it out once every day.
The other problem is that the flash memory itself will wear out eventually. They can only withstand so many erase/program cycles before the area of memory being written to fails and can longer store data. For example, one of the flash chips used in PlayStation 2 memory cards is rated 250,000 erase/program cycles. In theory this is a lot, and you should never see a PS2 memory card fail this way, but in practice I've heard of some cards have failing because of memory errors. At least part of the problem is the way saves are stored in PS2 cards, causing the same area of the flash memory to written multiple times, over and over again, each time you save.
The endurance of flash memory is proportional to the size of the flash memory. Higher capacity flash chips are rated for much less erase/program cycles than lower capacity chips. This means memory cards for the original PlayStation (128 KB) should have a much longer endurance than PS2 cards (8 MB), especially given the simpler way saves were stored. The much more dense memory used in Memory Stick Duo Pro cards for the PSP (up to 32 GB) should have less endurance, but overwriting the same area of memory over and over is much less of problem.
Note that both of these potential failures are the result of actually using the memory cards. Just sitting on the shelf, safely stored away, none of these problems will manifest themselves. No one knows how long flash memory will last like this, but it seems it should be for very long time. Much longer than battery on your old GameBoy cartridges in any case.