Client Datastore API: default consistency of 1.0 when testing

[mitchhentges]

I just spent a few hours debugging flaky tests, and I finally realized that it was due to the "consistency" settings that I was using.

When LocalDatastoreHelper.create() is called, perhaps a default consistency of 1.0 should be used? Otherwise, integration tests to verify proper API usage will "randomly" fail, even if the tests are fairly small.

[mitchhentges]

For reference, I've made a minimal reproducible suite of tests that would (within a few runs) "randomly" fail. The tests are fairly simple, so I was a little surprised that the default values bit me as they did 😉

[mziccard]

@mitchhentges Thanks for you interest in google-cloud. You are right, LocalDatastoreHelpe.create() starts the emulator with a consistency of 0.9. The Datastore service is not strongly consistent, hence we decided to offer a default method that creates a non-consistent datastore emulator.

The consistency level used by LocalDatastoreHelper.create() is documented in the javadoc. LocalDatastoreHelper also exposes a create(double) method that allows you to set your favorite consistency, see the javadoc.

[mitchhentges]

Other than having the same behaviour as "production" datastore, what's the point of having a "test" datastore with a default consistency of 0.9?

IMHO, if someone is using the LocalDatastoreHelper, they're using it for tests. Probably, they want to verify that their integration with datastore is working (passing tests if integration is good, failing tests if integration is bad).
Perhaps I'm shortsighted, but I cannot imagine a case where having an inconstent datastore is helpful in unit/integration tests.

Finally, to be honest, I've got everything working on my end, so this isn't an issue for me. I just don't want future consumers of the "Client Datastore API" to run into a similar issue that I did

[mziccard]

Perhaps I'm shortsighted, but I cannot imagine a case where having an inconstent datastore is helpful in unit/integration tests.

Some Datastore operations are eventually consistent, testing against an eventual consistent emulator is useful in all cases you want to verify that your code is not depending on strong consistency when it should not. Having your integration tests pass against a strongly consistent emulator don't necessarily guarantee that your code will also work against the actual Datastore service.

Also notice that 0.9 is the default consistency level used by the gcloud command line tool when starting the datastore emulator:

gcloud beta emulators datastore start

Finally, to be honest, I've got everything working on my end, so this isn't an issue for me. I just don't want future consumers of the "Client Datastore API" to run into a similar issue that I did

I am sorry you spent time on this. Probably our TESTING readme or javadoc could have saved you some time.

[mitchhentges]

(fun 4 years update: mziccard was right, it's good to be aware of potential consistency issues in your tests, otherwise you could be surprised in production).