I've used CLI tools for years, and it would probably take me 5-10 minutes of scanning the man pages to reproduce all but the simplest of those snippets. Granted, this boils down to the sporadic use-case, but that's the entire point.

The replace examples are all clean. It would mean I would not be juggling little peculiarities of arcane syntax, such as which sed flags to invoke, or whether to use exec or xargs.

I might have an issue with doing a replacement in JS for node, though. But for those who are already using node, why not.

I quickly reach a point where I just think "Fuck it, I'm writing a Python script for this."

A simple tool that only takes a couple of minutes to learn, then a five second look at the man page to refresh the syntax is very valuable, particularly if it focuses on doing the one job that people want to do most often.

> The issue is that it's 5-10 minutes to scan each man page every single time I want to use sed, find, xargs, etc."

Which means you could put in a one time investment, write a wrapper shell script, and be done.

And if you insist on disagreeing still, then that's fine, but the this whole point was brought about in response to:

> I've used CLI tools for years, and it would probably take me 5-10 minutes of scanning the man pages to reproduce all but the simplest of those snippets.

If the problem occurs too frequently, it makes more sense to make the solution re usable. Which is exactly what 'replace' is doing here.

Exactly. People tell me "Nobody needs ack, you just write wrappers around find+grep" and I say "You could, but I published it and now people don't have to."

"[Laziness] makes you write labor-saving programs that other people will find useful, and document what you wrote so you don't have to answer so many questions about it. Hence, the first great virtue of a programmer."

I don't have to worry about updating my find command lines to account for new types of source code or source control folders in unfamiliar programs, I need only ensure that ack is kept up-to-date.

C++ once compiled to C, but even then people wouldn't claim they were using C, they'd claim they were using C++. "I don't use C++ because it's C with 'classes', I use C++ because it sets up my vtables and overloaded functions"

It clearly serves a purpose and it's apparently useful. Most importantly, it exists, so there's no need to argue over how you think it should have been implemented.

I'm sure if you were to release your own [script / wrapper], someone would find use for it, in the same manner that some have apparently used 'replace'. I honestly fail to see the big deal here.

What you fail to take into account is that there's definitely a niche for Node.js-based development tools such as this script: The GNU userland is not ubiquitous.

Node.js did not come with a system command to search and replace across files - now it does.

I fail to see why you will ever need to rewrite this?

If its of any help I think its time for the Unix community as a whole to sit down and self reflect. If people are beginning to think something like 'sed' as too level to use, then better job needs to be done to make them more accessible and usable. There are enough examples on this thread where people are saying that they look at sed as an API on top which a program needs to be built up on. And not a complete program in itself. Also shell is not a full fledged programming language.

Any new API on top of them would only be present on a fraction of machines, and thus would reduce the major advantage of them in that they are available everywhere.

One advantage of her Javascript replace is that she can drop it on any machine with a Javascript runtime and it will work, unlike sed.

And the real truth is, she wrote something that solves a problem she has, which is the measure of her success, not someones arbitrary comments.

Probably, but I can't really speak to that.

Either way, we were originally talking about the syntax & implementation. It doesn't make a lot of sense to object to a hobby script on the basis of development time.

Such wrappers aren't lingua franca anymore - nowadays they are as arcane and as COBOL, since [for most X] it's easier to write a perl|ruby|JS script to do X than a shell script to do X.

Starting from scratch (say, from a Win background) it might be easier to learn a language like Haskell or Scala than to learn the shellscript language with the whole pack of sed/awk/grep/etc flags and tricks.

Moreover, there are times when you'll want to do this stuff from within another program. What are you going to do, shell out to a find(1) pipeline? Then people like me will be giving you shit.

There's nothing wrong with rewriting sed in Javascript.

Indeed, especially since there's sed and sed.

The examples given use -i, but no luck with BSD sed, where you will use -i~ at best. Also, lack of -e before the expression may make things blow up at times. And the sed regex does not support stuff like +, so many times I end up replacing sed -i~ -e with perl -pi -e or ruby. Besides, sometimes your looping logic is not implementable with a find easily or at all, especially when its context is readily available in the original language, so maybe you use the CLI at times, but this CLI might just be a frontend to a lib part saving you from forking a thousand times over.

Are you talking about BSD sed here, because GNU sed does support + with the -r flag.

Moreover, if you want to use this as part of your node app, then it's a good reason to reimplement some of sed's functionality.

I still don't see why you'd laugh or ridicule someone else for doing something like this, and sharing it (if anything at all, it serves as a good tutorial on file manipulation with javascript/node IO.

Often not possible depending on what authorizations and hosting you have.

And although its getting a bit off-topic at this point, doing replacements directly in the file isn't even a good idea. The standard "move $file to $file.backup, run sed against $file.backup and pipe it into $file" method is not just recommended because it is portable, it also saves your ass when you messed up your search+replace and broke all your files.

You'd be surprised what is restricted in a lot of corps beyond the "approved" software list. It is amazing what big things get approved and what little things don't.

Here's what print0 does - it separates the filenames with a null byte so that names containing spaces and newlines can be interpreted correctly.

Hopefully someone else will find it useful.

On the bright side, normalizing file names before batch-processing them was a good idea too.

Seriously? find/xargs is nearly impossible to use (safely) without it.

So you know who you're dealing with here, I used Emacs for something like 10 years before I figured out how to do regexp search/replace.

I'm pretty comfortable with my abilities as a software developer. If I was Steve, I'd be about as embarrassed about the idea that someone's "level of understanding" of find|sed|xargs was worth commenting about as I would about the rudeness. Of course, if I was Steve, I'd have apologized privately and not fed into the shitstorm, too. :)

I also think that binary sized string should be the default for shells and command pipelines; plain text was great in the 60s, but it's now the 21st century and I use too many shell flavours on too many systems to remember their arcane quoting rules, locale rules, unicode bugs, etc.

Regardless, it's a shame that there's not a standard separation of UI from functionality in all these utilities - more than once I've wanted to just be able to dynamically link libsort.so from another program because writing files and exec'ing ... urgh.

But it's not. It's a command-line tool. To replace the best, most powerful, most flexible, most stable tool (and additional tools) made for the command-line for this exact purpose. This is the definition of reinventing a square wheel. It's just a bad idea.

(p.s. by using xargs (with or without -print0) you can use one option to parallelize the function, vs writing it all in your own code. but i guess getting mad over a fork/exec call is a better idea)

If you can replace a long line of pipes with a simple command line, then why not?

When admins use it, they often simply shell out to the identical console tools. Why? It's easier. They just do everything you need, they're universal, and they work well together.

Go ahead and write a new Perl in JS, and nothing will change; the console tools will still be there, better than anything else.

I agree that it simplifies the workload. Unfortunately, it does not reduce the workload.

So if the program has overhead when loading, using a -r option instead an outer find loop can make a magnitude of difference to program operation.

by the way: "-print0 | xars -0" is really idiomatic, you know? Lucky you didn't have a single filename with spaces to handle...

It's the shell, by the way, that cares about "spaces in filenames". Not dicking around with shell metacharacters is just one of the many wins you get by not shelling out to accomplish basic programming tasks.

Note that 'xargs -P' behaves very similar to 'make -j' and is often useful as a simple work-unit scheduler.

Still, someone probably rewrote something similar to sed in JS

The sed/xargs/etc. examples look increasingly like impossible-to-memorize line noise to me, while the --include, --exclude etc. options in `replace` are comprehensible and seem to follow standard command line argument practices.

Again, this is from a very entry-level unix tools perspective, but there it is.

It also seems to me that reimplementing things with the stated goal to make them easier to use has to be one of the best justifications around, no?

I hope I sound as respectful here as you did in your comment -- I certainly appreciated seeing a clear response to her library, so thanks for that.

That replace library does what it's supposed to do - it replaces shit easily.

Don't like it - don't use it.

If people want to do more - then they'll figure it out for themselves. For everyone who already knows more - eat some humble cake.

This is a false dilemma and completely misses the point. The replace library was written by a person who either never heard of or never learned how to use SED. Just because they created their own hack as a workaround doesn't mean it's the right thing to do.

By this logic, there is no way for beginners to write much of anything since there are existing implementations of almost any idea, and it should be disallowed ever to make a new thing which could replace any old thing.

"It's similar to sed but there are a few differences:" and then goes along to list out the differences.

I would expect this person to 1) have heard of and 2) have learned to use sed in order to write a tool that is similar but has slight differences in syntax to sed.

As for editors - there is a niche for emacs and there is a niche for notepad. If a program is weaker on some aspects but clearly superior in another aspect, then it has a valid niche and making it really is the 'right thing to do'. Usability/discoverability is a very important aspect of software, and sed is a bit lacking there.

Are you honestly at HackerNews posting that someone hacking together some code to scratch their own itch is not "the right thing to do"?

Yes. And people can site Pi to the 10.000 digit. Doesn't make it any less useless.

Calling the basic operation of these cornerstones "useless" speaks more about your disposition than the tools, I think.

Yes. Sad isn't it?

It's not as if they were perfect when the were designed.

    $ replace "Foo" "Bar" 

Not to mention that sed -i is a GNU extension.

I haven't used replace but it seems like a pretty innocent piece of helpful code for someone who likes JS.

Heather, you've created a nice and useful tool with a modern tool-set, did a very decent job coding it and took the time to open-source it. In my eyes at least it puts you in the top 0.01% of coders in the world.

However I used it so infrequently that whenever the need would arise, I would invariably screw up the order of the 2 (only 2!) arguments. I could have added support for --help, --usage or similar, but that seemed like overkill.

I eventually just learned how to "properly" use find and grep (both of which I was already using anyways, which helped me remember the command line for each).

With that said I do agree that I'd never use shell script or Unix userland on purpose when writing scripts that are intended to be portable across different Unices (or even Windows), that's the kind of thing I'd reserve for personal use on my own userland. Hell, even on one machine you never know with some distributions what kind of shell you'll login to by default...

I haven't spent much time learning unix utilities, so that might explain it. But I can't help but shudder at find . -type f ! -name '.min.js' ! -name '.py' -print0 | xargs -0 sed -ri 's,var,let,g'

It is the usual tradeoff between simplicity and features.

There is only one case that justifies reimplementing things

Or, um, how about the one that goes "I felt like it"? That's sufficient justification in my book.

I'm siding with mrb on this one. Learning flexible tools gives you immense power in comparison to low-learning-curve ones. The time invested is considerably smaller than the time spent figuring out how to solve edge cases with simple tools. This is something which appears to be impossible to communicate to people with words. Learn some of these tools and you will see for yourself.

By 'learn' I do not mean, for example, becoming familiar with hjkl for navigation in Vim. I mean mastering the tools to the point that using them requires no effort - where just as you experience the impulse "I want a glass of water" and without any effort go to fetch one, you experience the impulse "I want to create a new class in file X, navigate to file Y then modify method Z", your fingers start moving and suddenly you're done. I get frustrated watching my IDE using co-workers write code - it takes them a very long time to do very simple things. Their tools are narrow and do not facilitate expression - they constrain it.

Pre-emptive strike on the argument that code completion is better - Eclim (Eclipse with a Vim frontend) solves this problem very well. Nonetheless, I have it installed and configured but can barely find a use for it.

Even in the consumer world, this is Apples strongest selling point for example. Make products simple and easy to use, but still do 90% of what anyone would want to do with it. If that isnt enough, there are other things to use.

I think the same can be said over replace. I find your skills with sed/grep impressive and its certainly very useful in some cases (mostly devOps) but in case i need it myself i would just look it up and in the meantime use simpler tools and get the job done.

I would rather use my language of choice to implement the above (mine wouldn't be JavaScript, but it clearly was Heather's) than to research all of those arcane command lines.

Rather obviously YOUR language of choice for this kind of problem IS sed. So OF COURSE you would use a shell script wrapping sed etc. to implement these functions.

But guess what? To handle all of those options at the command line would still take a pretty large fraction of 173 lines of code, if you added in an equivalent level of parameter checking and option handling.

And if you want to make MY eyes bleed, hand me a file with ~150 lines of logic in shell script that spawns cryptic command lines. Despite not REALLY knowing JavaScript, I would much prefer to maintain the JavaScript than a shell script with the above commands.

PS: I will do it in 15-20 lines, if only to prove a point, if someone sends 1 BTC to 1F43fceWvwznv4rH7s1B3du2w5VznFJY58 :) (yes, my time is valuable - I have other things to do)

When I DO see non-trivial shell scripts, they are just this side of arcane gibberish. The command lines above are completely opaque to anyone but an expert in EACH of those tools (I know some of the tools, but not all well enough to know EXACTLY when I need to backslash a paren, or how many backslashes to use, or when I need quotes or not...).

When I see the JavaScript that spawned this conversation, despite not knowing JavaScript well at all (probably about the same level I "know" shell scripting) I was able to follow everything that was happening to the point where tweaking it to add features would have been easy.

The objective shouldn't be to write code in the fewest possible symbols. Everyone would use APL (or a more recent evolution of that language) if that were the case.

The fewest possible readable and modifiable lines of code is my objective. Shell scripting tends to have lots of obscure and unpleasant syntax to accomplish the most basic flow control.

I've had a sed book on my shelf for years and I've used it more than once, but 99% of the time I end up giving up on sed and using a GUI-based tool instead of trying to concoct the cryptic lines given above as examples.

I absolutely would rather use the clean interface of replace, written in JavaScript or anything else, than to memorize that steaming pile of cruft.

1. You can pipe find/sed/grep/cut/awk/whatever to other utils. Today you might be replacing, tomorrow you might be analyzing text logs, and the more core utils you learn the more things you can do with them.

2. When you're ssh'd into another box that doesn't have that spiffy replace utility, now you don't know how to do an inline replace.

Fair.

This boils down to an individual's needs. I think most people in a position to be making this choice are fairly well aware of what's possible with standard nix CLI tools, and can decide whether or not to invest the time to learn them.

In my experience the ROI for nix wizardry is fairly high at first, but begins tapering off. There are a half dozen or so command chains I use daily without which I could hardly see myself working, and the rest I may not touch for weeks or months on end. YMMV.

> 2. When you're ssh'd into another box that doesn't have that spiffy replace utility, now you don't know how to do an inline replace.

It doesn't make sense to optimize a workflow around an edge case. You don't often hear this brought up as an argument against dotfiles, key bindings, vim plugins or what have you.

If you run nonstandard, you basically have to learn vim/vi as a default install for an OS and another with all your add-ins and customization. Sure the key bindings are similar, but I don't have my macros and why is my tabbing funny, there's no line numbers, and damn it, the backspace key doesn't work. I forgot I set that in my .vimrc.

Sure, I can remember most of the latter with set tab and set no, but when you're popping around systems, switching to root (with it's on .vimrc), keeping applications as stock as possible is very helpful.

I personally have a slightly custom .vimrc for my local use, but I don't rsync my dotfiles across the globe for my other home directories. I basically try to keep it as code development (with .vimrc) and sysadmin (may not have niceties, but I'm not editing code).

But as soon as I want to do something more sophisticated, I again reach for the tools I know better.

No I can't use my GUI tool on a box I'm ssh'ed into, but I can push my changes to a git repository and pull them to be local, where I can do complex things really quickly and without having to memorize tons of obscure options and corner cases.

In a case where that's not as feasible (hundreds of megabytes of log files, say), and what I need is beyond my find/sed/grep skills, I usually drop into my language-of-choice and produce a tool that is much easier to use for my purposes. Honestly I haven't touched awk in a long time because the pain is so much greater in awk to accomplish what I can in my own more familiar toolset.

I get that those tools work fine for any old-school Unix hacker, and the combination is quite powerful in its own right. I don't have the desire to BE an old-school Unix hacker, though, and my tool-set is quite powerful enough to accomplish what I need without going through that particular hazing ritual, thank you.

My need for tools like this happens rarely enough that even if it would be faster once I'd learned the tools to type out the appropriate command line, the amortized time to GET to that skill level in those tools would be far more than the time it takes me to just use my "less optimal" tools.

2. Perhaps you don't need to do this. Or perhaps you'll work it out when you ssh into another box.

I feel these examples serve more as a selling point for why one would use something like `replace` over a cacophony of flags and pipes. The interface for `replace` is simple and elegant. It does one thing, and it does it well. I think that's commendable.

P.S. The `sed -r` flag does not exist on POSIX systems like OSX. You'll have to use something like perl -pie, or write a convoluted buffering loop.

You are right in telling that sewing a few utilities using pipes can emulate 'replace'. But remember you are talking about learning those 'few' utilities first. Trust me unless you have practiced your way out of these thing with hundreds of hours with experience over years its humanly impossible to remember and use a tool like sed to its full capacity. The more utilities you add to your pipe chain the number of ways in which these utilities can be used more or less follows a exponential curve.

Now if some one comes along and gives you a version 1 of a tool that can simplify these things and give you simple way of doing things in a command instead of doing it with a ten pipe chain commands, he/she is ideally solving a problem of a great importance.

As some one who loves the Unix CLI, I have often wondered why people spend a whole day writing a java program what could have easily done with sed and awk in a line or two. But these days I understand why. Learning and using sed/awk to their full potential is a time consuming task. You not only have to learn them factually through a manual. You also need to study the problem cases for which they are appropriate. Then you need to go and put their solution patterns in your brain. Then you need to map all problem patterns with solution patterns. Then you need to learn all the arcane special cases. Generally this comes to a person over years practicing those utilities.

As a matter of fact Perl was invented because using C, with all these utilities was getting very difficult.

Now if somebody can give things like these. Which do many things out of the box, without you having to go through the pain of learning things and practicing over the years. They would have indeed given you a tool of great value.

In my experience even if some one fails working on something like this, by third or fourth version of the tool they generally come up with a thing of great value.

Think of this like Vim Vs Sublime Text 2. In case of vim there are ways of solving problems. In ST2 many of those solutions are automated and sometimes given to you out of the box.

The argument is that basic shell knowledge should be trained in order to not lose skill (or to improve). At my work, these skills are useful when facing unexpected urgent needs.

Having a short script is a very poor benefit (simpler syntax and reduced key typing) for a big loss: - loss of knowledge - loss of understanding of what the script is doing - need to learn the name of the script which is not a reusable knowledge.

Learning the basic usage of sed, awk, shell is an investment that is quickly rewarded.

Learning all the intricacy of "replace" needs to know many javascript details and how they interact with shell escape sequences and your LOCALE configuration. I am sure it is less documented than the sed command.

Some people just don't live in the shell. We're taught to build upon useful building blocks to make useful things - why would you not abstract away the monotonous tasks?

This kind of reports are useful for people doing similar activities.

Who are you to decide how she may spend her time? Why in the world do you think that she has to justify to you what projects she creates?

Frankly the code is hardly deserving of derision. It looks perfectly capable to me, and the syntax is far superior to arcane unix commands. Just because you'd rather write a sed wrapper than write this code doesn't mean it makes sense to deride others for trying to move the human race forward.

  replace 'var' 'let' . -r
  find . -type f -exec sed -ri 's,var,let,g' {} +

  replace 'var' 'let' . -r --include="*.js"
  find . -type f -name '*.js' -exec sed -ri 's,var,let,g' {} +

  replace 'var' 'let' . -r --exclude="*.min.js,*.py"
  find . -type f ! -name '*.min.js' ! -name '*.py' -exec sed -ri 's,var,let,g' {} +

In the context of discussing the open-sourcing of the script, the issue isn't really whether or not writing the JS is simpler than learning "3 find options", it's whether using the JS is simpler than learning "3 find options".

You are wrong about #2 too: I have said this 5 times already: you can write a very short (much shorter than 173 lines of js) a wrapper shell script exposing an interface as easy as "replace".

Please read what I wrote!

"There is only one case that justifies reimplementing things: if your tool has the requirement of supporting the exact js regex syntax..."

(Please read what I wrote, really!)

b) Regarding those that do want to "run the command line," can you agree that there are users who really want and need to use JavaScript regexps from the command line and not some other regexps with different semantics?

c) Can you accept that these are the valid use cases that your "use sed" arguments just don't cover?

I've read your arguments and I've never seen clear acceptance of a) b) and c). I've also seen that you had to carefully think and even improve your "use sed" implementations, giving also the good argument in favor of her approach. Having the tools that demand less from the user is a good thing. Can you accept that (d) too?

Can you see why we discuss, when you just quote yourself as "There is only one case"? That's why I claim you're wrong. There's not only one case.

You just use it.

Let's face it. Most people will never bother learning shell commands in and out. I've been a *nix user for 20 years and I still have to be checking man pages, and I often stumble upon basic shell command options running differently in different shells.

Other than for sysadmins, shell commands are going the way of the dodo. This isn't new, it's been happening for many years already. Increasingly capable computer people from the newer generations will be focusing their skills and time elsewhere. Arguing against this reeks of grumpy old coder bitterness. I know my shell stuff and it serves me well. I use vi/vim daily. I don't impose my usage patterns or expect that something "existing" means it's appropriate for everybody.

No, Perl doesn't.

PCRE was written because a lot of people liked some of the ideas that were first introduced in Perl's regular expression engine. But PCRE was a reimplementation, and never was all that compatible with what Perl was doing.

Other than "re", there's "re2" and "regex". None are PCRE but "regex" is close enough most of the time.

Don't get me wrong. i actually _know_ the f* syntax because I'm and old crazy fart like that by now. But when it comes to use it, it takes a while to type. Sometimes I make typos. Sometimes i forget one specific option or character and I gotta look in man again.

The simplified syntax wrappers make it f' easy.

The other day I made a csv column grepper. It just calls awk and assign numbers to column names, so i can just grep by column names no matter their order.

So yeah, a huge line of awk can do it. An even less understandable line of grep can actually do that too. Some other tools as well. Mine is just ./supertool colname colname colname ... | awk "$2 != blah" |column -t

(For the record, there were similar wrappers in python and perl, each of which were 5 to 35x slower (200 to 1000 lines script) than my 20 lines bash script, which is another complain I have. Reimplementing that stuff in high level languages is generally a bad idea, specially when you don't understand what happens underneath.

I'm just making the point that it's not just about learning, it's about writing the code. If you're the type of programmer that is only looking for a means to an end, this isn't for you, but if you're like me, the fun is in figuring it out yourself without the help of libraries.

In a professional context, I understand[1], but this is a personal project someone wrote to scratch their own itch. Why is it unjustifiable? I can think of plenty of reasons to reimplement something but I think the burden is really on you to say why it shouldn't be done. If you think replace is a waste of time just don't use it - simple as that.

[1] I've spent the last two years maintaining an application written in a homegrown web framework - don't get me started.

- Node.js can be installed on Windows and sed/find are not available on this platform.

- Many of my coworkers don't understand sed/find syntax, and even if they could learn it, they don't have plenty of time.

Btw, it's terrible that a few guys make rude comments about a free and open source software. If you don't like, don't use it or if you think you can make it better, fork it or shut up.

A tool's a tool. If it's useful to you, use it. If you don't need it, then don't. Why does this have to turn into a debate about why someone shouldn't have invented another tool because one already exists?

Look around the world, there's competition and choice in literally everything. Why can't they be choice in what CLI tools to use?

That said, the javascript solution does look a bit long winded.

Also, please don't take it as a negative comment on your comment, I actually appreciate the solutions and it makes me consider sed again.

It would be far quicker for me to write an one-use program in a number of languages than to find out how to do this in the sed oneliner style that you show - ergo, the sed approach has drawbacks in some use cases, and alternatives should be tried.

Depending on how much you need to this sort of workflow, this could be extremely valuable.

And why assume a POSIX command line?