If you run it on an encrypted volume on the VPS, you get encryption at rest (e.g. if someone steals the disks, it's encrypted), but still your VPS provider is able to decrypt it (otherwise it just couldn't run).
Normally I either encrypt a non-boot drive (if the VPS provider offers such a thing) or use gocryptfs. It’s still a pain though when reboots happen, unless you also put your key there. Application layer encryption makes it easier.
gocryptfs is great, I use it to encrypt storage in embedded scenarios where the OS doesn't have the userspace tools or kernel modules to manage encrypted block devices.
If you run it on an encrypted volume on the VPS, you get encryption at rest (e.g. if someone steals the disks, it's encrypted), but still your VPS provider is able to decrypt it (otherwise it just couldn't run).