Why can’t you just use strongly consistent reads for all your DB reads, with retries on 500 responses? According to CAP theorem increasing consistency should probably lower availability, but can’t the decreased availability (increased 500 responses) be handled fairly easily using retries? (assuming you are fine with a small percentage of queries taking a bit longer due to retries)
Using DynamoDB as an example, but this can be generalized to any noSQL cloud offering – It also seems like DDB with on demand scaling will simply increase your read capacity units (RCU) used if you turn on strong consistency, incurring a higher cost ($) but keeping the same latency on db queries, so it seems like the negative is only higher cost. It seems like you can just keep vertical scaling the DB’s processing power to meet your needs. Is it actually plausible that with a noSQL cloud database with a high traffic level, you cannot just throw enough money at it, and it could hit some scaling limit to make strongly consistent reads slower?
And then with regard to the entire question generalized to distributed systems, what does it actually mean for a distributed system to be ‘strongly consistent.’ I’ve heard this used to describe systems before but I don’t actually know it means beyond ‘all DB interactions being strongly consistent.’
My second question might be more basic level but necessary to understand the cost of providing consistency, but why do consistent read issues actually occur, ie. why does stale data occur (in single queries like read/write, not transactions with multiple reads/writes per transaction)?
From what I understand, with decreasing probability any time after a write occurs, it’s possible for one reader to read correct data, then a second reader to read stale data AFTER the first reader reads correct data (correct me if this isn’t actually true, but my understanding was it is). Why does this happen? Doesn’t a read just involve a read from some location on the disk?