> In the long-term, if we could statically eliminate the possibility of races, that would eliminate the need for most of the memory model. That may well be an impossible dream, but again I’d like to understand the solution space better.
Unless I'm mistaken, this is an impossible dream as long as shared memory exists. It's the core tradeoff that distinguishes the Erlang runtime from the Go runtime (there are others, but they all stem from this).
Your goals are either memory isolation for better distribution/concurrency/clustering/fault tolerance/garbage collection or shared memory for ability to work with large datasets more efficiently.
It's one of those details that changing it would essentially create a new language. You'd have code, packages and libraries that either worked that way or they wouldn't.
IMO, this is an area where Go gets into dangerous territory of trying to be all things to people. Be great at what you're good at which is the "good enough, fast enough, portable enough, concurrent enough, stable enough" solution for backend services in most standard web architecture.
If people need distributed, fault tolerant, isolated, race proof, immutable run times that aren't quite as top end fast and aren't ideal for giant in RAM data structures...there's already a well established solution there by the name of Erlang (and Elixir). They made the tradeoffs already so you don't have to reinvent them.
> Your goals are either memory isolation for better distribution/concurrency/clustering/fault tolerance/garbage collection or shared memory for ability to work with large datasets more efficiently.
The isolation is not physical, but logical. Implementations are free to use zero-copying and make everything just as efficient. Theoretically compiler could even optimize message passing overhead away for systems with shared memory in some cases. The opposite is also true, shared memory is also logical and there is a lot of room for a lot of clever things, like eliminating races.
Look into Pony's reference capabilities. To put it shortly, the compiler will guarantee that there are no data races through this system, and therefore the runtime is able to use shared memory and pointers for efficiency.
Rust's "sharing xor mutability" guarantee eliminates data races by (usually statically but also dynamically) ensuring that a memory location won't be mutated when multiple places/threads can access it. Of course, the machinery required for this in Rust is relatively complicated (e.g. as a side-effect, it allows one to also forgo a GC without losing safety).
Rust lets you be precise about placing the locks exactly where they are needed, with compile time errors if a lock isn't locked when accessing data, or a lock is missing. Fixing data races with locks for arbitrary code, without more static guarantees in the language, essentially means locks have to be held for most memory accesses: it's hard/impossible to tell if that location will be accessed concurrently. (This is why CPython has a GIL: it's easier to only let one piece of code execute at a time than to handle data races in arbitrary code.)
For example, in Rust, it will stop you from accessing the value protected by a Mutex after the lock is freed, and do so at compile time, even through function calls, pointers hidden in data structures, etc.
Another example: I can take a value and mutate it locally, then pass it immutably to a number of threads which use it to compute some results, and once they are all complete, I can mutate the data structure again. If I try to write mutation code before all the threads are re-joined, the compiler will tell me that I can't mutate while there are outstanding immutable borrows. At compile time, at no runtime overhead or locking.
Unless I'm mistaken, this is an impossible dream as long as shared memory exists. It's the core tradeoff that distinguishes the Erlang runtime from the Go runtime (there are others, but they all stem from this).
Your goals are either memory isolation for better distribution/concurrency/clustering/fault tolerance/garbage collection or shared memory for ability to work with large datasets more efficiently.
It's one of those details that changing it would essentially create a new language. You'd have code, packages and libraries that either worked that way or they wouldn't.
IMO, this is an area where Go gets into dangerous territory of trying to be all things to people. Be great at what you're good at which is the "good enough, fast enough, portable enough, concurrent enough, stable enough" solution for backend services in most standard web architecture.
If people need distributed, fault tolerant, isolated, race proof, immutable run times that aren't quite as top end fast and aren't ideal for giant in RAM data structures...there's already a well established solution there by the name of Erlang (and Elixir). They made the tradeoffs already so you don't have to reinvent them.