I think part of your success comes from having created the ability to, de facto, buy Bitcoins via credit card. Any credit card (e.g., stolen ones). That's also your biggest legal challenge. That's what should scare you. But if you can sort out those details and regulations with a good lawyer, you shouldn't be afraid of success and taking projects as far as they'll go.
One of the steps to de-risk the cards will be a waiting period- new cards won't be accepted unless they've been activated X days/months. That should seriously lower the chance of stolen cards getting through.
