I interact with plenty of companies - not a statistically relevant amount, but dozens. I'd put this at about 50% right now. I certainly deal with a not insignificant number of meat proxy CEOs, which is highly annoying for the people worrying about the details. But it's not impossible to work with. I just accept the overhead and hope it gets better.
Hang in there. I've yet so see an even modestly successful business that didn't have this or some other bleak problem at one point in time. I'm sure they exist, I just haven't seen one yet :D
My gut says this one can be turned around. Even if 90% of people just happily prefer ChatGPT for this (which I'd doubt), the remaining 10% is still a lot to fund a small venture very well. Just gotta find them effectively.
Even the "giving up" route can be pretty juicy, there's companies that pay decent money for a "distressed asset". And there's also investment firms (like mine) that go for a minority stake and help turn things around.
Just speculating (I use zero automation when recruiting), but automated candidate ranking systems could well still look for signals that were valuable years ago, but no longer are. It's not a given that people using this stuff even understand the criteria under the hood.
Same story as SEO, I presume. Google surely understood their ranking criteria deeply, yet couldn't exactly win that fight.
Google could have easily won that fight by blacklisting scrapping spam sites and spam portals. They just didn't.
There weren't that many with high ranking. I could filter out most of them in like a week of working normally and just right clicking them with the blacklist extension in Firefox.
Nowadays it is probably harder though when Google have broken the internet culture.
If you blacklist them just for yourself, the sites don't care, no economic impact. If Google does the same, they'll work around. There's lots of spam-only sites, but they also often cleverly avoid detecting by mixing in content ranked as valuable/legit. SEO also certainly "attacked" otherwise legitimate sites like Reddit, although what GEO is doing there is far worse I imagine.
But there seemed to be some time lag for a SEO spam sites to climb. Removing the ones that make it would mess up the economics at the time. Like the Stack Overflow mirror sites or like spam phone number pages with no other information but the number.
And I guess plenty of search queries would have no moderation. But I mean I spent like 10 minutes. You could have 10 FTE doing that.
Nice analysis! I would perhaps add that LLMs are pretty helpful at (2) in my experience, as long as you don't turn your brain off. (As a Google/SO replacement, mainly.)
I have about three modes, working with LLMs:
A) Just use them for (2), do (3) and (4) myself.
B) Just use them for (2) and (3), but with very fine grained instructions - to the point where it barely saves me time.
C) Use them for (2), (3) and (4), but with this workflow, it quickly degrades into slop for me.
I still get the best results with (A), but for a quick PoC, (C) is hard to beat.
What's hilarious (or sad) is that I see a lot of people throw them at (1) as well, which seems to have a high chance of inducing AI psychosis and creating software nobody understands, let alone needs.
I agree. If you use the AI for (4), it's pretty much guaranteed to hallucinate whatever it wants and then approve it. For small projects, it might be good enough. For anything serious, it's not.
I find I get the best results when I just use it for (3), and sometimes a bit for (2). Going in with at least an idea of how to make it work has the best chance of ending up with a good changeset. And if you already know how to make it work, don't leave it to chance. Tell the LLM what to do, then let it do it.
Regarding (2) figuring out how to do things at a higher level (e.g. the general architecture of an application) is pretty different from figuring out things at a lower level (e.g. how to fill text with a certain gradient in CSS - the stuff where years ago you would have spent hours reading documentation or browsing StackOverflow until you figured it out, if you weren't familiar with the topic). So you could use an LLM just for (2b) and (3) and do the rest yourself.
> the stuff where years ago you would have spent hours reading documentation or browsing StackOverflow until you figured it out, if you weren't familiar with the topic
Why is it always hours to consult docs? I find most answers within minutes (including if I have to read code) and it’s only take a bit longer when I want to understand the why’s.
The key is to make sure you have the right questions. I see people struggle with that where they only know something wrong but they can’t put it in words.
Ok, maybe you're one of the fortunate ones. With "if you weren't familiar with the topic", I meant the situation where you know what you want to do, but have no idea how exactly to accomplish it. Especially with frameworks, which generally have a quite rigid way to do things, if you're unfamiliar with that, it can take a while until you get anything done, despite good documentation.
With GenAI, we can now produce something without caring about it - or while caring about it very little.
And the parts we don't care about aren't necessarily worse, they are just... arbitrary. Could be good, could be bad, no one knows, because no one really cares.
I find that if I care about something a lot, it's a pretty similar time investment than pre LLMs. And it makes me feel invested and proud in the result, motivated to show it and improve it.
If I care about something very little, in the past I just wouldn't have done it at all. Now I might, but I feel that same disconnect you mentioned.
I think being strategic in what we do and do not care about is likely the key skill we'll have to build to actually make the best of the tech.
Not entirely my experience. LLMs can be good at two things, but unable to be good at them _at the same time_. I've had some success with getting decent simplification suggestions out of them.
Right now, the majority of companies I interact with seems to look at their now (let's assume) increased capabilities, and turns up the speed. But there's other knobs to turn up instead. Robustness, quality, user experience.
Optimistically, I think we might just enter a new phase at some point, where businesses make very conscious decisions about what to invest in to differentiate. Like, everyone has support bots now. Where do we invest, absolutely stunning human support? Or is support not that key and we should invest into being the most reliable? The most enjoyable to use?
But that defeats the whole point of AI, as far as business is concerned. You could describe most programming languages as working the same way and having the same limitation. If this is true, then AI will simply mean you'll have very highly paid AI experts (who know how to do X with AI) that you absolutely need to hire or your competition will run at 10x your speed.
Business incentives is to maximize money, which requires some quality. But these days, not a lot.
Customers ideally maximize value, which lies on some spectrum of cost and quality. Part of the reasons businesses act with disregard for quality is that the consumer ultaimtely doesn't punish them for it. Whether that's due to true consumer preference or a market lock-in is a discussion left to the reader.
Can confirm. I just had a nightmare experience with a company where the customer service sent very ChatGPT-like replies and seemed very unreasonable and AI like. I had to threaten legal action to get my money back.
Role Collapse might work in this type of situation: prompt your own ChatGPT to decide a refund with a very permissive refund policy, copy its thinking, paste into the next support message after your prompt, and their ChatGPT may believe it's its own thinking.
The Role Collapse attack is that LLMs turn out to ignore role tags (input, output, thinking) and identify role using writing style alone.
Funny you say this. I _almost_ tried this. But it did feel like a person was doing some copy-and-pasting and watching closely, so it wasn't 100% automated.
> Where do we invest, absolutely stunning human support?
That's a complete non-starter unless you're serving some extreme luxury premium market with absurd margins, and/or your customer interactions already require a lot of human involvement. So 99.9% of digital products are right out.
Good human support is absurdly expensive and will easily make your product 10 times more expensive than your competitor's. But almost no customer is willing to pay 10 times more for good support.
I disagree. I’m biased but the customer eng/support team we created at stainless was really good and that was a key differentiator during sales calls and post-sales. It’s expensive if you see it as an afterthought and just an expense you would prefer to avoid, but we approached it as an enabler role, a multiplier of sort: ensure customers feel they engage with peers, provide guidance to the eng teams, enable salespeople to engage meaningfully with technical teams, etc. For like a year it was basically just me and an awesome colleague handling the whole thing full time. We took the very explicit decision to have it as a very technical role and designed a culture that sees it as essential to the business (because it was), and that worked really well IMHO.
If you create a startup you really want to focus on a small audience that you understand and know really well, instead of chasing everyone. And for that a support team is one of the most important asset you have
the key differentiator is whether your business is aiming to provide a quality service, or is desperate to eek out as much profit as possible.
my example: A&A — uk internet service provider.
costs like twice as much as a normal service per month. but if i have a super technical query im on the phone with a fully fledged engineer in about 2 minutes flat. the kind of engineer that can discuss complex dns issues. i’ve had one help me debug a dns issue from a work site. it turned out quite quickly it was nothing to do with them but he hung around on the phone for 10 minutes anyway.
the first support person i ever spoke to was michael from wales. if i had to call up he was the first voice i would hear for years. he recently moved on and i was sad. we’d have a decent chat. big up michael from wales.
so, as it applies to at least one existing business — you are dead wrong. it might be twice as expensive as the people trying to maximise their profit margins, but it’s definitely not 10x more.
Apple has pretty stellar support in my experience, and while I don't buy Apple stuff for other reasons, this is a strong attractor. For B2B products/services, it can be a huge differeniator as well, in my experience. But it was really just an example, it might not be the call most companies make. Maybe most will still just invest in being faster or cutting costs. I just hope that's not how it's gonna be.
You gotta be joking. Apple has no support at all, and if you ever go into a shop at their "GeNiUs BaR" (after booking an appointment days in advance), you will get extremely poor service and incompetent support.
I never went into an Apple store, but the phone support issues I had (even a pretty complex bug on their end that locked me out of my account) where handled swiftly and well, and I spoke to humans that seemed to know what they're doing. I could say this should be the default and I shouldn't be impressed by it, but unfortunately it's top 10% based on my experience. I'm in Europe though, maybe they do things differently here.
Used to work at an AppleCare call centre. Apple has extremely high standards for their support advisors. I'm aware that the genius bars are terrible, though.
Counterpoint: Take a look at consumer cellular. I would say it’s drastically cheaper than verizon, at&t, or t-mobile, but the customer support is better than all 3.
Ting also has (had?) excellent customer service. Call the 800 number and the first thing you hear is “hello” from an actual human who has the ability to help you. They also didn’t charge a lot, which makes me think great customer service is a choice.
Last year I bought a book from Amazon for over $100 and they sent the wrong book. I contacted Amazon support, they said they will refund it, I don't need to send it back (throw it away if I want), and also give me a $20 credit for the inconvenience.
A month later, no refund. I contacted them again, they said unless the book is sent back, no refund. I told them about the previous conversation and they said that the CSR had lied to inflate his scores, and if I've thrown the book away I'm out of luck, he generated a return label. I complained and they threatened to close my account unless I send verification documents. I didn't, I just sent the book back and the refund processed automatically.
About a year before that I had a Samsung Fold and it cracked down the middle. I told Amazon that it was only 6 months old and it was broken, and Samsung said I need to send it to Cyprus for some reason. Amazon said to send it back for a refund, I did, and I got a warning that I'd returned something "not in original condition" and next time I'll be banned.
Last week a package didn't arrive. I queried it and there was no longer the option to talk to anyone, just a link to phone numbers for the external courier.
So I would say that Amazon having stellar support 15 years ago, building a brand on it, and then choosing to stop doing that mostly proves how wrong you are. Sure, you can still sometimes speak to someone, but it's getting worse.
FYI you can sue them in small claims for this. This is one of the languages that companies really listen to. You would have to have actually thrown the book out in good faith, though, or commit perjury. Remember the airline LLM support bot and the bereavement fare?
It might. Nothing in courts is ever certain. You could always try it, and at worst you'd be out a court fee (low, for small claims) and wouldn't get your money back because of this clause. Or they could say the clause doesn't matter.
Meanwhile Amazon has to pay someone to attend the hearing.
Note that Amazon would probably ban your account regardless. Most retail businesses don't want customers who might sue them, even if they deserve it.
Haha yeah, Uber, Amazon, Starlink, Hotels etc just send me stuff or refund without even asking anything. I use all of them all the time for everything; that indeed seems to help. Logical of course.
But that could just mean you are a more profitable customer. And that could be influencing how amenable the AI/agents are and how much money they are willing to write off to retain you.
According to the comment I was responding to, he got 3 bad experiences out of 3 orders, so 100% rate of issues. What's the expected rate of house fires?
Both you and the other commenter about his grandma's smoking habit are making nonsensical comparisons about things that are extremely rare, to support something that is supposedly extremely common, if you were to believe the parent's comment (or other stories you read online).
Amazon has a big problem with "unifying" products that are in fact not the same thing.
At one point I was looking through editions of the Arabian Nights and I noticed that, while the "kindle version" linked from the page for the paperback of one edition was an unrelated book, an actual kindle version of the edition I was looking at did exist elsewhere on the site.
I was annoyed enough that I opened a chat with customer support and asked them which of the two ebooks (one linked as "the kindle version of edition X", but with a different translator, title, cover art, etc; one not linked to the paperback at all, except for the identical translator, cover art, etc...) was the kindle version of the paperback.
A few days later, the links had been fixed, and the "kindle version" link on the paperback's page went to the kindle version of the same book!
Good example. Transitioning from an outdated framework to a modern (or sometimes "slightly less outdated") one is probably one of the few situations where you do not want to change semantics at all.
And in my experience, these are _dangerous_. People go into "while we're at it..." mode, and it quickly turns into a big 2.0 kind of thing that takes forever.
I would argue that LLMs can speed this kind of thing up, but not by an order of magnitude or anything, just a bit. Unless there's high risk appetite.
It still kind of blows me away that almost any LLM usage for coding isn't viewed as "high risk appetite"
Building products that no one really knows the internals of is crazy to me, and the methods people have of trying to mitigate that problem seem half assed at best
As someone who currently automates the payroll flow generated by someone who doesn’t actually know what it does, I can confirm I am going crazy. My boss will do nothing about it because her boss can’t get finance to let us hire more people. I plan a strongly written resignation letter whenever I find something else.
Sounds like you might work on a team with some agency to say no to management.
We have some and sometimes marketing comes back with some extra revenue from a partner if we build out feature X Y or Z for their new product launch. The contracts are signed so engineering has to do it or we’re blamed for lost revenue.
A few of those a year and you eventually end up in a similar situation.
That sounds nice. I hope once I have secured my legal status in my new country (few more years to go) that I can take such a risk. Not everyone is in the same position to take that sort of risk.
I think everyone is right on this question :) I have certainly worked in places where there were enormous code bases written in dead languages going back to the 1970s and I was part of the collective belief that nothing could be done about it and our job in the 2000s was to put lipstick on the pig by burying it behind a web portal, if you remember that short-lived fad. In that kind of environment I would have _loved_ an exact port to a modern tech stack from which we could begin the very slow and careful evolution. Speaking to people who work there, AI has indeed changed at least their perception of what is possible and they have traction on porting it that was considered impossible for decades. Whether it works for some definition of “works” remains to be seen, but it might be self-fulfilling because the belief that it can be done will mean it is tried more often and some of those tries will probably succeed.
With that said, I’ve also recently done a rewrite in a completely different sense, taking what used to be a web app and rebuilding from the ground up as a desktop app instead. Having the original code base for core concept reference, but rethinking the whole UI more than a decade on was IMHO a much better approach in that case.
Depends on how much you're asked to burn I guess. Until one point, it's actually helpful. Then there's a point where you can do stuff that's semi helpful but doesn't get in the way. But then I would imagine you reach a point where you have to come up with a token burn strategy and some kind of narrative for your manager that's in line with it. I bet at that last point, it gets taxing.
Probably like eating. Having to eat less to loose weight isn't great. Eating anything you want without worries is great. Having to eat more than you want to gain weight, not great.
