People regularly assert that credit cards are factored into the cost of goods, but I posit that's mostly untrue. The cost of most things, especially "luxury" goods like restaurants, is set by the price the market will bear—not the cost of providing that good. Anecdotally, it's certainly not cheaper that equivalent goods and services are cheaper in Europe despite the lower credit card usage.
It's very common for supermarkets in the UK, for example, to compete on margins on common food - not necessarily packaged/prepared food, but definitely the basics, fruit/vegetables/bread/milk/flour/eggs/etc. People have little loyalty to a specific supermarket and often have 2-3 within reasonable distance, so there's a very strong pressure to keep down the cost on these items - they're pretty much the proverbial widgets. The lower the cost to sell an item, the lower the supermarket can charge. Some supermarkets take technical losses every year, with most having falling profits every year.
This sort of thing affects poor people more than it affects people who are eligible for a credit card with a good rewards program in the first place. I'm sure you really don't care that your potatoes are 2-3% higher in price than they otherwise would be.