Oh, yeah, that drives me crazy too. Usually, I cd to the root of whatever project I'm working on, and I want all paths to be relative to the root of that project (though since I started to use ctrlp[1], it's less relevant).

1: https://github.com/kien/ctrlp.vim