You should use maxmemory to avoid this, but before to reach this condition Redis will start to be very slow of course, it's hard to reach this condition without to note it.

Btw the way to go is maxmemory, that will be in the next tar.gz and in days into Git. maxmemory also allows to use Redis with a memcached semantic of volatile keys expiring to make room for the new ones when we are low on memory.