Except for perhaps Lisp languages, almost no language makes compile-time computing and code-generation so easy. This allows for some really powerful language features that can be designed as libraries, and puts this power in the hands of "regular developers" rather than only in the hands of template wizards.
Yeah, I really love that at compile time you can meta-program in almost the same language that you use for normal run-time things. I think the comparison to lisp is apt. It almost feels like lisp macros.
Think C++ template metaprogramming but much, much easier and thus, seemingly more powerful. It's not that you couldn't do the same in C++, but D's metaprogramming is so much more accessible that it makes you want to use it.
There's plenty you can do with D but not C++, see regex, bitfields, swap member-by-member with checks for mutual pointers, the pegged library for grammars, etc etc.
I don't know, Boost has me convinced that even things like regexes in C++ templates could be possible, if your compiler has a big enough stack to handle very deep template recursions. It's theoretically possible to build a regex parser in C++ templates, right? It would be horrible, but possible.
> It's theoretically possible to build a regex parser in C++ templates, right?
You'd probably do large portions of it in "constexpr" types and functions rather than relying on recursive metaprogramming techniques.
They are also planning on adding overloading based on whether a function is constexpr, which is important if you want the same library to support both compile-time and run-time matchers.
So it's all theoretically possible already. But D has the advantage here in that it's not just possible but usable.
I guess the best comparison to Python would be the dunder methods. You know how in Python a class really is just a dict, right? Like, foo.bar is pretty much syntactic sugar for foo.__get_attribute__(foo.__dict__['bar']) in most cases. Or how a + b is sugar for a.__plus__(b).
Well, imagine that all of the things you can do in Python by messing with dunder methods, function decorators, or metaclasses could be precomputed by a compiler and would not even execute at all during runtime. It makes everything really fast at runtime and easy to precompute at compile time.
Yes, and plus you get type safety with all of that! When I'm writing in D, my editor marks all the type errors (and typos) in my code, almost as fast as I make them. The DMD compiler is so quick that there's almost no lag. This leads not only to a rapid development cycle, but a much higher level of reassurance (than I've had when writing, e.g., metaclass hacks in Python).
Decorators (in D, 'user defined attributes', or UDA) work differently. It's not a function-composition feature, but more of a tagging feature. You write a class, function, etc., tag it with custom attributes; and then have a separate compile-time function walk over your code, find the decorators, and augment the code based the meaning of the tags. (I used to wish that D had adopted Python-style decorators, since they are easy to reason about and implement, but I can see the logic of the more general UDA system that they adopted.)
In practice, though, you would often use templates to achieve the same effect. Given a memoize template (really, just a memoize function), and an expensive-computation function,
auto fastComputer = memoize!expensiveComputer;
produces roughly the equivalent of
@memoize
def expensiveComputer(): ...
but with opportunities for compile-time optimization.
I've dabbled with Nim. It's also a great language with great metaprogramming features. Plus a fast compiler that generates very lean code. (The real reason I stick with D over Nim, above all else, is RAII -- once you have it, it's really hard to live without. Okay, that, and also ranges, and array/string slices... they are fantastic to work with. Nim's got a better GC story right now, IMO, but there's a lot of activity in the D community to become more competitive in that space.)
I found that some of the Nim's MP features, in particular AST macros, are a little harder to work with than D's templates and compile-time function evaluation. You get a lot of flexibility, but the cost is high. I'm not a big fan of how "mixin" is used in D to splice source-text into a generated function, it's certainly less principled than an AST transformation, but in practice the resulting code tends to be concise and easily read, whereas the AST-macro approach introduces a lot of accidental "noise" and complexity. The complexity raises the bar for reaching for a compile-time solution, where in D it seems equally as natural to write compile-time code as it does to write regular code. (Not to pick on a strawman, though -- AST macros are not Nim's only MP tool.)
The Nim MP feature I never really played with was the rewrite rules. They seem interesting in theory, but maybe a little too magical for my liking!
I have a lot of respect for Nim. I am glad we live in a world with so many options. Like Walter said in an earlier comment, it's an embarrassment of riches. :)
Thanks for the writeup. I've just been evaluating Rust, D, and Nim, and reading about your experience has been helpful.
Can you elaborate on how D is planning to sort out the garbage collection? Nim's GC is extremely fast and thread local, and can be disabled without breaking libraries (according to the author, it does something with memory regions that I haven't 100% grasped yet).
I've googled about D's garbage collector and it's apparently been discussed as the language's biggest flaw since 2013, but I can't find any information whatsoever on what's being done in that regard.
Like Nim, you can also disable GC in D, and you can avoid GC altogether by not allocating GC'd memory (GC is only ever triggered at allocation time), or at least by preallocating and then disabling the collector. Raw allocation is always possible, and you can "emplace" D structs and classes into unmanaged memory.
The collector itself isn't being improved as far as I know -- at least I haven't seen any initiatives mentioned recently with that goal. In fairness, I haven't been following the community activity very closely in the past few months, but I think that's accurate. Nim definitely has a technical advantage re: its GC implementation.
The bigger movement has been the "@nogc initiative", which started with adding a @nogc attribute to the language (the compiler can verify that a function tagged with @nogc, and all of its callees, do not allocate GC memory). There is an ongoing initiative to make more of the Phobos library @nogc-compliant, to take advantage of this feature. There has also been a lot of work on custom memory-allocators [1], and I think the plan is to incorporate into Phobos where it makes sense, so you can have functions which take custom allocators, have thread-local allocators, etc.
I don't speak for the community or the dev team, but I think the long-term goal is to make the GC a feature that is available when you want it, but that isn't a dependency for using the standard library. Either through custom allocators or through @nogc guarantees, you'll be able to ensure that your program's memory management is deterministic.
Thanks for the info. It's pretty hard to get clear, updated info on these languages as they have yet to gain that much traction (and they tend not to be backed by big entities that have a PR budget).
For anyone else who's evaluating D and looking into its GC situation, the most recent blog post on Dlang.org (https://dlang.org/blog/2017/03/20/dont-fear-the-reaper/) seems to embrace the presence of the GC, but also ends by saying the next blog post will describe how to go without the GC. So there is indeed awareness/activity on that front!
There has been work done to improve the collector but nothing has made it into mainline. For example one implementation was missing parts for Windows but worked in Linux thus it couldn't be pulled in and used.
