Indeed. I don't recall uploading data to LinkedIn, but it's possible I gave it permission to access my email at some point. If I did so, it would have been years ago. Problematically, that data has now been stored on LI servers, so I can't revoke access.
It may not be directly from your address book: if you know enough people who have linked-in accounts, are linked to you, and have fallen for one of the access routes, then a lot can no doubt be inferred about you from them.
