The 4GB refers to a 4GB memory unit for the system. You use the storage for saved games, user profiles, downloads from Xbox Live, and "downloading" a game disk to your Xbox for faster loading. The only thing you would be missing out on that really matters is space for demos, videos, and other content from Xbox Live, as you won't have much space for them. Game demos are several GB each, so the built-in 4GB won't hold many.
The upside is that there are several choices to upgrade the space in your system. USB thumb drives can be used for storage space in an Xbox 360. You can buy 16GB flash drives from ~$30 any more, so that's one way you could go. Another method is to buy one of the internal Xbox hard drives. According Aaron Greenberg, via Joystiq, 'this 4GB unit "absolutely does have that expansion bay that we have the opportunity to use in the future."' So you will be able to buy any of the Xbox 360 S-capable hard drives to put in your expansion slot.