I mean, I'm happy to have this argument. CEPH is content agnostic and that's fantastic most of the time. Cyrus replication is data aware, so it's not just replicating the data, it's doing integrity checking and data model consistency handling.
Most of all, it's doing split brain recovery; which - if we wanted CP rather than AP then we wouldn't need, but that wasn't the original design.
If I was redoing this from scratch, I'd maybe do Ceph or similar and update Cyrus to work well with it, but that would be a big change from the current design.
Anyway, I'm happy to stipulate that Ceph is great tech, without going and telling other people that it's the only choice.
Do you honestly think CEPH isn't doing data consistency handling?
I'll pay for your ticket to cephalocon if you'll speak to that effect(!)
Split brain stuff only happens when you're splitting a single threaded task and put it back together. MDS in CEPH has this problem but that's so far into the weeds here as to be off topic.
Again you're implementing something storage not in storage and taking any storage. Fine if you want to do it that way, but talk about _that_ not hecking ZFS being mah saviour.
(Btw daily driving and love that too but an email provider _relying_ on it should raise eye brows)...
I do believe we are talking past each other here. Of course ceph does data consistency, but it sure doesn't assert that a modseq is monotonically increasing or that an mailbox/uidvalidity/uid triple doesn't change digest, because it's not data-model aware.
Most of all, it's doing split brain recovery; which - if we wanted CP rather than AP then we wouldn't need, but that wasn't the original design.
If I was redoing this from scratch, I'd maybe do Ceph or similar and update Cyrus to work well with it, but that would be a big change from the current design.
Anyway, I'm happy to stipulate that Ceph is great tech, without going and telling other people that it's the only choice.