Well, I use /home with luks on some machines and user directory (encfs) on others, but have all of them with swap encrypted or no swap partition at all.

Since the keys/other private info might leak there, you are not doing it right, unless you have sysctl swappness level set to 0.

Not only that, /tmp and other temporary directories might also be another leaky place...