Does anyone have any references on how you could write a distributed database that met all ACID properties? Surely there's an academic paper that says that if you do A then B then C, you are guaranteed a certain level of consistency.
We've developed a type of distributed database at my company, and I think it's pretty solid, but I need a broader familiarity with the available theory.
Aside from reading papers, it's a good idea to look through the syllabus of a distributed systems course to get a broad idea of what the problem space looks like.
Academic papers will talk about "minimal" problems like consensus, or desirable properties like sequential consistency, and expect you to already know why those concepts are important. If your experience is mostly hands-on, it may not be obvious how it all applies to real-world systems.
Say you have a complex distributed database. Forget all the bells and whistles: can it solve the problem of allowing a set of processes to reliably agree on a single Boolean value? If so, then you're trying to provide the same consistency guarantees as Paxos/Raft. So if your architecture is substantially simpler than Raft, then either you've come up with something really ingenious, or you've missed some edge cases.
For the record calling something a "distributed database" is not nearly enough. What part of it is distributed? On what operations do you want to provide the ACID guarantees? What do you promise in the face of partition?
I would be very skeptical of any database that was written by someone who didn't have a sound foundational understanding of distributed systems theory. This is quite simply one of the places in software engineering where subtle differences in promise, protocol and expectation can make a huge difference.
As well you should be. Fortunately, we're relying on some third-party libraries that are built on a solid theoretical foundation, and what we promise is limited. Our goal is to improve over time.
Consensus is the main hurdle - if you have multiple nodes that can be read from, then any values that are successfully written to the system must guarantee that those same values can be read sequentially.
Issues arise when network partitions interrupt communication between nodes; even if you require all nodes to send acks when writing, how do you deal with those acks not being received?
We've developed a type of distributed database at my company, and I think it's pretty solid, but I need a broader familiarity with the available theory.