You absolutely need to be aware of and manage language and locale settings diligently. This is a big topic, but generally, whenever possible, use UTF8. And you generally don't want to change your desktop OS or applications' locale or language settings other than that.

However, if you are processing data files and are aware of the implications, and can get away with treating data as binary instead of character streams -- for example, any of the command sequences that use sort/uniq for uniqueness or set union/intersection/difference -- then using C locale is way faster. As in, the difference between doing something in an hour on a big machine and rewriting a whole pipeline in Hadoop. A good fact to know.

It's obvious I (the author) didn't make the simultaneous truth of both these points clear. I'll rewrite.

It's not worth the "performance improvement".

    $ echo $'a\naa\nb\nå'| LC_ALL=C sort 
    a
    aa
    b
    å
    $ echo $'a\naa\nb\nå'| LC_ALL=nn_NO.UTF-8 sort 
    a
    b
    å
    aa

    $ echo  $'∨\n∧'|LC_ALL=C sort -u
    ∧
    ∨
    $ echo  $'∨\n∧'|sort -u
    ∨

In general, I'd never do it for performance only, it would have to involve one of the issues you've pointed out (there are some components I build that require it, erroneously IMO).

Knowing that "LC_ALL=C can improve performance but cause UTF problems" is awesome information to base a design on. A blanket statement like "never use LC_ALL=C" is not.

Again, not worth it -- if only for performance sake, generally speaking.

You can apply the same argument to any other obscure encoding scheme, not just UTF-8. The answer is really to pick the encoding which will suite the most usecases and won't be painful to use.

Fact of the matter is people have to get out of the habit of going "oh, that will never be relevant for me", because a) that's unlikely to be true for everybody (for example in a server environment you can't even write perfectly good North American names like Étienne let alone perfectly good older-than-the-country names like Þórr) b) it's unlikely to be true even for you as you will discover the first time you try to read a file written by somebody else c) it promotes intellectual laziness that makes later internationalization efforts difficult to impossible, because they're trying to convince people like you to care about stuff they find inconvenient but that is of vital importance to your customers.

I know of a man whose first name is Þórr who outright refuses to deal with any organization that will not let him write his name, and I have a hard time disagreeing with him.

I wish I had that luxury sometimes. Here's a pet peeve. I live in France, where people expect everyone to have a one-word family name (it seems). It's really common for people and especially computer systems to "correct" my name (which is of the form Me van der Somebody) to something like Me VANDERSOMEBODY or Me Van Der Somebody. A particular web page (my employer's, actually -- where I have to fill in my name for my "profile page") does the latter, and refuses to allow me to fix it. I have submitted bug reports, and just get WONTFIX back. :(

It's really rude, so I can imagine that for a person called þórr it's a nightmare everywhere except Iceland.

This is a horrible assumption. UTF8 can show up just about anywhere. 99% of the utilities mentioned in the article support UTF8, which means they will output UTF8 if they think that's a good idea. And that can show up at any point. Exotic filenames, error messages, fancy boxing or indicative characters in the output...

Don't expect Latin-1, ever. Respect specified encodings, and default to UTF8. Never, ever think you'll be fine with Latin-1 when dealing with an UTF-8 output just because you speak english.

The trick the original article specified is a hack to get better performance in very controlled situations, it should never be put in a rc file like it recommends because it will break and you won't know that is what's causing it.

Edit: Here's a few examples of very real potential breakage, to show you how bad an idea this is: You're debugging a python script. You open a python shell and start pasting some of its lines to figure out what's happening. OOPS, there's UTF-8 in there.

    [4:08:48] adys@azura ~ % python -c 'print("I like the letter Š")' 
    I like the letter Š
    [4:08:54] adys@azura ~ % LANG=C python -c 'print("I like the letter Š")'
    Unable to decode the command from the command line:
    UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc5' in position 25: surrogates not allowed

Should I keep going? See how "awk '{print tolower($0)}'" behaves when it's dealing with utf-8 input. Maybe that awk is deeply hidden in a 5000 line script. Maybe it deals with web data, filenames, anything. And suddenly you have unicode escapes in data that doesn't expect any.

Now how's that one line in your rc file, which you wrote maybe months or years ago, working out for you? Are you going to know the issue is your LC_ALL variable?

How does he fly? All flights I've ever been on have required me to ASCII-fy my name.

  setenv LC_ALL en_US.UTF-8
  setenv LC_COLLATE C             # use the ASCII sort order