Skills are mostly snake oil, the way people use them (the aspiration to download kung foo from a celebrity).
There was a time when maybe it mattered (last year), but with good repos and good prompts today's agents can find exactly what they need without any skills.
"Skills" as developer macros can be useful, but at most those are things shared with the team (in the repo), not something you download from the internet. If you have so many skills that you feel the need to manage them, that's a code smell.
I think waaay more people struggle with this than HN would have you believe. In the real world, not everyone is a software dev with a developer mindset to using these tools. Normal people essentially type the equivalent of "Make me X!"
and complain when the model assumes anything in their underspecified mess of a prompt. There are skills like grill-me that can potentially help these people a lot, but in the end I believe models will just be smart enough to understand your level of knowledge and intent to do this stuff on their own. They are getting much better on pushing back on poor user input already. The problem is that when they double down on hallucinations (very rare nowadays but I still see it happen in enterprise projects with the latest models). So you kind of need to know when to push back on the model as well. But for that you have to be really good at the subject.
I tend to agree. Skill files become less useful as developer skill increases.
As a skilled developer my repetitive instructions are mostly one or two sentence phrases for staring something like a highly-interactive planning session, or a self-supervised implementation session with my preferred setup of implementation and review subagents. I can specify those out by hand, or save a couple keystrokes with a tiny skill file.
But if you are not a software dev you might lack the vocabulary to tell the agent what you want. If you don't know what tenant isolation is, chances are your app will have a broken security model because you can't ask for it, and probably won't think to ask the agent for a security review either. Skills can mitigate a lot here
I still find the superpowers to be very useful, just as a way to manage the feature process, getting you a spec to go with the feature, quizzing you about edge case handling.
It's interesting as grill-me and the other Matt skills are very much positioned to people who would consider themselves as developers. In fact, I'm not sure if people who were completely new to development would have heard of him at all.
My company ran a test and found that they reduce token output on flagship model by something like 2-4x, and that number has been increasing with newer models. I suspect the increased subagent usage is driving this trend, because this means we're relying on models to do their own prompt engineering.
Yes, they are just text, and can therefore be replaced with good prompting. However, this also means they confer a real benefit: a good set of skills creates a transferable baseline, raising the skill floor and offering a more consistent experience across the organization.
I am somewhat confused by takes like this. Of course skills are just prompts, this is the whole point.
A skill is just a stored prompt you want to put more information into than you're likely to type out every time you intend to do that thing. Documentation of a business process.
What are you confused by? You're saying the same thing they said.
They added the additional claim that writing the skills down (apparently) prevents the models from having to self-prompt on the fly and therefore reduces token consumption.
Yes, skills that are actually used count toward token consumption.
The question is whether the number of tokens required to achieve a certain behavior/intelligence/quality is equal between you manually providing those tokens via skills versus the model "deriving" the "skills" it needs on-the-fly in order to produce the outcome you want.
The claim above is that the former requires far fewer tokens.
Also skills only consume tokens when they are used, and part of the value is that the model will dynamically find and disclose only what's needed (assuming the skill is "well-designed").
Their claim is not about the prompt or skill tokens, it's about output tokens - skills can help the model bypass some thinking tokens or avoid reasoning deadends, and that way reduce output token usage. That's what they seem to have found empirically from their testing. (If it's truly 2x-4x, the time savings in waiting for the output is a pretty nice benefit too.)
It's not just a stored prompt, you can attach re-useable scripts to them to offer more determinism. ex: a script that validates that a PR follows exactly the template you want, with a max of N lines per entry.
The more determinism you have, the more consistent you can be and the more leverage you can build. (yes I understand that skill calls are non deterministic).
We do that, but keep the scripts in the code and just tell them in the markdown where the scripts are, same with "references" (docs/) for us. It never made sense to me to put those in a skill dir, many are useful across skills and for humans (many written for humans before agents were a thing)
One of the more interesting benefits to skills is that many harnesses now run the inline command(s) in backticks, shortcutting the model needing to make a tool call. This is helpful for deterministically building up context content for the skill before the agent ever sees it.
We take this further in some instances and have workflows that (1) does deterministic context gathering (2) invokes an agent (3) processes a file the agent is told to produce. This has made our PR review agent much better and removed it's access to all credential files. We have a step that gathers the diff + existing pull request comments into a .review dir, let the agent process that and create a comments.jsonl, then run a script in a new step to apply the comments against the API
+1 on all of this, it is similar to the pattern I see emerging as well. For example I have open-sourced (let's be honest, it's just my personal tooling of taste) a repo on gh to try and formalize these things. I'd rather not share out of mostly imposter syndrome but essentially it's a scaffolding tool ala "create-react-app" for the stack I generally use.
Part of that is to create standardized "docs/{architecture, decisions, contracts, operations}" folders that my harness recognizes in my repos to have a better sense of where was it.
I was solving your problem.before LLMs with my personal developers Swiss army knife (hof gen/CUE+templates). I'm somewhat keen to see how I can capture the agent markdown I want in many places, like project init, in CUE via embed and modules I can import and version, skipping the need for a markdown dependency cycle management system.
It's not about giving hints to agents because they wouldn't find it out otherwise. It's about saving the work of them having to find out. Good skills files save tokens.
How do you actually measure the token savings? Do you compare the same task with and without the skill, or is it more of a noticeable difference over time?
Both, in a way. In order to be sure it's the skill reducing token usage, you measure it with and without the skill, and then you do that periodically to account for all the other variables.
Alternatively, you pick a belief based on prior evidence from either approach you mentioned, which is the natural thing that many of us do.
> "Skills" as developer macros can be useful, but at most those are things shared with the team (in the repo), not something you download from the internet. If you have so many skills that you feel the need to manage them, that's a code smell.
I have three development machines. You kinda need something like git to keep everyone in sync!
And there's still value in encoding a process in a skill - it's way more token efficient to tell the model what but also HOW to do something. Otherwise, it just spends a lot of tokens figuring out something that they previously did already.
> I have three development machines. You kinda need something like git to keep everyone in sync
I got tired of the constant git push/git pulling just for inter machine transfer, now I just have a ~/Git directory with all my projects that I just Syncthing between my desktops/laptops and Syncthing ignore any repos that specific machines don't need
Yes, skills as a "portable power" isn't really the use case for me unless it's entirely generic and even then sparingly.
I've mostly followed what anthropic suggests, which is putting less into context and more into skills, to keep the "how" out of context until it is needed to reduce context bloat.
Skills have some instructions but are primarily informed repo specific instructions and keep their context away from the rest of the repo to keep things sanitised for me.
> Skills have some instructions but are primarily informed repo specific instructions and keep their context away from the rest of the repo to keep things sanitised for me.
Skills and agents in the Claude world can also be extended and evolved over time, as they are committed "code".
For example, we have an agent which can take a statement or a support ticket and identifies the services, tenants and infrastructure components likely meant in the ticket or request. Similar to a skill, Claude can invoke this on demand in a conversation.
This started very simple, but various people spent time tuning it over the last 4-6 months. They have "taught" it to pick up on jargon from different departments, writing style of different departments, how they think about their systems.
With all of that tuning over time it has become quite "clever" in identifying the mentioned systems and - if requested - the train of thought leading to this conclusion.
Similar things are happening with skills for various task, be it Ansible integration tests, upgrade chores and so on. The first version can be fairly underwhelming, but continuously improving it after each usage can make them very powerful.
Depends on what you do. If you work with proprietary tech that is not in LLM training data and can't easily be found on the internet, you're cooked without good skill files.
Yeah, this is my use case for skills. Even with good documentation, it feels better to have things local and easy to tweak.
I'd add to that I've also used them as a style guide. The project involved taking in unstructured inputs and creating structured outputs. Lots of choices along the way, and it seemed a neat way to encapsulate decisions we'd made as a team.
Storage, well it's just for the one project, so the repo. Can't say I've used them beyond that.
I agree it depends, but I can offer another angle: By writing a few py tools and creating skills around them I was able to save tokens, so these skills were cost-effective in my case, they lowered the cost of the tasks I execute.
Yes, but isn't that kind of a waste? If there's not enough AI training data or even just the internet, how can we find information just from a command called a skill? It would be nothing?.
Let me give you an example. Let's say you're building processes using the process management tool FooTool from the company BigFoo. You tell the LLM, "make a new process." A process is just an XML file, but BigFoo is highly proprietary, so the LLM has no examples of how to make one. No public documentation exists on the Internet, so it's not in the LLM's training data and can't be searched.
So you make a skill "make FooTool process" that explains what a FooTool process XML looks like, what options there are for initializing a new FooTool process, and so on.
Now your LLM went from "Let me spend five minutes looking at other stuff in your repo to find anything that tells me what I'm supposed to do" to 20 seconds and a working process.
Not everyone use AI only for coding, for “code smell” being even applicable here. Many of my skills are just processes distilled from actual sessions doing odd tasks and coordinating different tools. It’s pretty reasonable to assume that it saves the agent from repeating that first time exploration fumbling
Where they are very useful is as a documentation source for LLMs. For example, I work in infosec and often have to reference DSLs (Cobalt Strike aggressor script for example). Having a skill which is an offline index to carved up function docs, which an LLM can use without having to think, then search for, then download huge 1 page documents with all function documentation, and pollute the context… very useful.
I did create special repo that has all skills/agents/scripts and MCP configuration. All agents ( currently 3 in use: claude code, opencode, and pi ) packed in docker container, with artifacts required by project technologies, and task at hand ( planning, code review, documentation management, web design, ... ). Nothing but a small file with list of technologies committed to project.
For evaluation, there is command to record session, repository commit, and observed problems that stored in special database, so each session can be reproduced. Developer commit reports, I do analysis, refine and evaluate system.
If you are spending time on all text forums like this, you are likely a person whose skill set skews towards the verbalization of abstract concepts. This is also the exact skill set needed to use LLMs well. If you are able to articulate exactly what you want in a concise prompt, little else is needed.
I think we tend to overlook the fact that LLMs have tilted the scales heavily in favor of those with good verbal skills. A huge portion of the population (including a portion of highly skilled software engineers) is not great at doing this. For them, harness skills still act as a kind of scaffolding; they support automated work on a project in cases where insufficient details is given in the prompt.
In my experience, while software engineers can be socially awkward/introverted, they generally do have good verbal skills, an excellent vocabulary and can be concise and articulate in writing, when there's no social pressure. There are some exceptions of course, but being able to translate an idea or set of tasks into a concise written language form is essentially what programming is.
The closest I get to finding skills useful is when I find myself repeating myself to an LLM. This tends to happen most when I am starting new projects and want to communicate basic design principles and patterns to follow and libraries to use. What I did was to factor and store these "chunks" of instruction in some text files. I then made a little script that can list what chunks are available and when given a subset will essentially `cat` the selected files to emit AGENTS.md content which I save into the new project or append to shore up an existing one.
Your observation on the readership bias of HN is a good one for people to add to their HUMANS.md before reading and commenting. :)
No, not really. Yesterday I asked Opus if it can read the logs from the sessions I have on the local ChatGPT app. It looked around and said no, that’s not possible. I said “what about these jsonl files in this folder?” It read them and said “ah yes these seem to be it!”
So I went ahead and created a skill for it. This is so that future Opus agents won’t come to the wrong conclusion the first one did. I can say “read the codex session titled ‘X’” And they will know exactly what to do and do it effortlessly.
There needs to be a word for this type of interaction because it’s so common in software engineering:
Q: I need help doing X
A: if you’re doing X, you’re doing it wrong.
I propose the word shamesplaining. What do you think?
Not saying your opinion isn’t valid. It just doesn’t answer the question and it’s disturbing that this is the top voted answer. It sounds more like a criticism than an answer.
I don’t think it’s about telling agent what to do with skills for most part anymore as well.
Problem is not having to remember to run agents in exact sequence and not having to repeat the processes.
That’s why I’ve created ctx traits and using them everyday, biggest wins for me are: sync across repos using git-based dependency manager and validator, which ensures version of the process / knowledge I’m using has been approved by me and hasn’t moved without my approval (similar to dependency managers lockfile approach).
This has allowed me to use same traits (it’s extension of skills with typed schemas and procedures, you can use skills with it as well as usual) across different repos without any copy pasting etc.
There are additional cool things like type-driven procedures with specific harness/agent associations and many more cool things coming soon.
This could be better summed up to the misuse of skills. Skills were not designed to be a way to make an agent more intelligent. Instead, skills are designed to allow agents to have certain tasks that are repeatable and predictable. It’s a misnomer really.
i agree, skills downloaded from the internet are all snake oil.
creating your own skills however good for both reducing the token usage & increasing reliability. those damn llms are not deterministic, asking same thing twice produces 2 different results.
This has been my experience (with downloaded skills), and currently my workflow is almost 100% skill driven.
Every feature I build uses a skill that does the following:
1. Read a ticket and get context on the task. The ticket was probably written by another agent after a conversation with myself about what is happening/needs to happen, etc.
2. Plan the task, asking for clarification where needed
3. Pressure test the plan, and validate the plans logic (subagents)
4. Implement
5. Runtime/local validation
6. Post PR, review it using applicable agents (database, security, code, prose...)
7. Fix PR based on feedback
I generally get excellent results out of this process, and I cannot imagine trying to orchestrate this without a skill. But I also can imagine my workflow isn't tuned to be super usable for anyone else.
I had claude define an agent for a "skeptic", the first few lines in the agent file are "You are a skeptic. Your job is to refuse to take claims on faith and instead verify them against ground truth.
You are not a general code reviewer. You are not a stylist. You are not a cheerleader. You take a list of claims (explicit or implicit), and for each one, you find the evidence — or the absence of it — and report what you found."
It goes on to describe what counts as a claim, how to verify the claims, and how to respond. It responds to each claim with verified, unverified, or contradicted.
The skeptic agent has been the most high value thing I've added to the workflow.
Hah I think if a repo is “standard” enough that an agent can navigate it freely without any help or direction, maybe it’s not worth having altogether?
Even fable _regularly_ stumbles as big repos or custom configurations, even for projects that fable itself built with high dev quality standards and modern design direction.
It just can’t hold it all in its context and will be forced to do “software archeology” all the time to figure things out - yeah it will work _most_ of the time, but to truly be able to scale and have autonomous agents reliably work and mold your codebase you need a lot more structure - tests, lints, compilers, validators etc. Your “skills” or policy files are there so agents can resolve issues and heal things themselves without your explicit direction.
If I have several tabs, each holding an agent team, with each agent spawning subagents as it sees fit, all of that apparatus has to ground itself _somewhere_ and if you don’t make decisions yourself, it will make decisions for you, save them in its own skill files, but some of these you might not like.
That's maybe the case if you always use the most popular framework and restrict your environment to a basic setup. But as soon as you e.g. build a website with SolidJS, Bulma and vite++ (vp), at least the models I tried are all the time somewhat confused, want to steer your project in a certain direction and build strange workaround so it works the way they are trained on.
Same with mcp. I want them to use the jsdelivr cdn instead of them scraping github against the rate limit. etc. But if I dont explicitly state to strictly use the $%!$@@! mcp for searching in repositories they simply ignore the mcp and even if clearly instructed, they still often fall back to gh.
Putting every detailed instruction in the AGENTS.md would just unnecessarily bloat the context and it works well enough to just instruct them in the AGENTS.md when to use which skill. Yet I agree that Skills are not some voodoo magic to provide your model super capabilities.
They're literally just documentation with a hat on. I don't mind a skill saying where the docs are but an overreliance on skills is simply proof someone isn't able to reason about the gestalt
My approach is different. Skills I write mostly use Python to save on the agent needing to run its own loops.
I use these loops to monitor the CI build and PR approvals rather than having the agent poll, and even Opus gets the commands wrong enough to make it worth it.
Last week I wired up a skill for the agent to share screenshots in PRs via specific S3 buckets and AWS CLIs. Again, the agents guess at the right commands often enough to make it worth being explicit.
Sure these could have gone in CLAUDE.md, but not every agent needs the context.
And at the company level, I can push skills to everyone’s Claude via the Teams function, they don’t need to edit configs or even know what a skill is.
the problem with that, that i'm seeing is just the sheer proliferation of internal skills. There are now so many that I can't see how they can possibly remain maintained.
It feels like at the project/product/middle management layer this type of 'big pile of skills to do some very specific task' is very popular. I think this is probably for a few reasons.
I think the largest factor is that these task management items are really just like .. calling a few different apis and slapping it through some jinja to post to github and create a jira ticket.. whatever. For an engineer, we can knock that out over coffee. But for this middle management layer, not a lot of them have the will or the skills to pop open the ide and code themselves a tool. Until the advent of skills that is. So I think they are a bit drunk with power. Which yaknow we'll see.
The skills are starting to become the documentation for these types of things as well as a way to automate it. This is great, but this type of 'documentation' is exactly the type of thing that goes non-updated for years in some dark corner of confluence. So I suspect the little used skills are going to fall to 'context rot'.
As an engineer, I keep all those skills in one very specific claude project and keep that largely separate from any given claude session that's helping me design a feature.
TIMTOWTDI. "[T]hings shared with the team (in the repo)" implies project-specific skills. See e.g. https://github.com/mattpocock/skills for a stellar example of third-party skills for battle-hardened workflows.
I think eventually, some form of self updating llm memory will replace skills, but in the meantime it is token inefficient for the model to have to parse the entire repo from scratch every time, and skills are an imperfect way to shortcut some of that (drawbacks being the skills are sometimes wrong or outdated)
I find them useful for deploying task specific agents, like reviewing Jira tickets, or otherwise ensuring compliance in open format submissions.
Otherwise I agree, and you don't even have to be that verbose with prompt engineering these days as LLMs have gotten increasingly good at figuring out what you want.
Woaaah buddy this is such a wrong statement that I’d delete it if I were you.
Can’t believe that people confidently spew blatantly false statements like this.
Skills matter, a lot, to every action that requires the AI to find stuff out, so that it doesn’t have to find the same stuff out again. Operating a website, building PowerPoints the way you like them, operating across different surfaces like APIs + GUIs…otherwise the AI has to relearn how to do it every time.
Be confident about things you know. Study about things you don’t.
I don’t feel strongly on skills either way, but why would you suggest GP delete their comment, without which we wouldn’t even be having this (in my view productive) discussion?
I exaggerated. But I'm so deep into these things that it's like someone writing emphatically 'colours don't exist,' or 'toenails don't grow on their own.' (Dunno why I thought of this second example)
There was a time when maybe it mattered (last year), but with good repos and good prompts today's agents can find exactly what they need without any skills.
"Skills" as developer macros can be useful, but at most those are things shared with the team (in the repo), not something you download from the internet. If you have so many skills that you feel the need to manage them, that's a code smell.