Repository navigation
Provider compliance/test suite for ADO.NET #17004
Description
To promote standard behavior across ADO.NET providers, it would be great if a test suite existed. Ideally all provider-independent tests would be extracted from SqlClient and made available via a nuget. This would be similar to what the EF Core team has done.
Activity
saurabh500 commentedon Jul 19, 2016
@roji
Are you suggesting that we add tests based on
System.Data.Commonand these tests are used to test any ADO.Net providers by instantiating the actual provider objects?Can you point me to the EF tests that you are referring to?
roji commentedon Jul 19, 2016
@saurabh500 yes, that's right. Providers such as Npgsql would extend test base classes provided by you, thus running your standardized test suite. Your tests would instantiate ADO.NET objects via the provider's DbProviderFactory and make sure that the provider's DbConnection, DbCommand, etc. behave as they should.
For the EFCore provider test suite, see Microsoft.EntityFrameworkCore.Specification.Tests and Microsoft.EntityFrameworkCore.Relational.Specification.Tests. The source code is here and here respectively. A good example is QueryTestBase, which runs various query scenarios and makes sure the results come back as expected.
It's true that ADO.NET is a mature technology which already has well-established providers, so such a "specification" test suite might be less valuable than in EFCore, which is totally new technology where providers can use the specification while implementing. But even with ADO.NET it would be very useful to automatically find divergences in provider behavior, leading to either standardized behavior or at least explicit documentation of those divergences.
roji commentedon Jul 19, 2016
Note that Npgsql contains many tests which are completely generic - i.e. aren't specific to Npgsql/PostgreSQL in any way. It is exactly this kind of tests that belong in a provider-independent specification test library.
saurabh500 commentedon Nov 3, 2016
@roji do you want to grab this task?
roji commentedon Nov 3, 2016
@saurabh500 I have a lot on my plate, but I can try to spend some time on this (not right away though). Also, I'm pretty PostgreSQL-centric and will definitely need help with making sure things are portable.
saurabh500 commentedon Nov 3, 2016
Sounds good.
roji commentedon Dec 21, 2016
Just to say that I'm unlikely to find any time to work on this soon... Hopefully Microsoft (or someone else) can pick this up.
divega commentedon Aug 1, 2017
cc @bricelam what do you think about refactoring the Microsoft.Data.Sqlite test to follow the test spec pattern we already use for EF Core provider tests?
bricelam commentedon Aug 1, 2017
They weren't written with that purpose in mind, but it could be a good starting point since they're very functional in nature.
divega commentedon Aug 1, 2017
@bricelam I think that is probably true for (at least some of) the SqlClient tests.
bgrainger commentedon Nov 21, 2017
This is something I've been interested in having for a while, too (as the author of MySqlConnector). I got tired of manually writing test cases to check compatibility across different drivers, so I created an automated test harness. 🙂
A WIP rough draft is here: https://github.com/mysql-net/AdoNetApiTest
I bootstrapped the prototype with @bricelam's SqliteCommand tests (which saved a lot of time, thanks!).
Inspired by http://seriot.ch/json/parsing.html, it dumps output like the following. (Take the colours in the cells with a grain of salt; I haven't investigated them all to ensure that they're not caused by a bug in the test harness itself.)
roji commentedon Nov 21, 2017
@bgrainger that looks like a wonderful start - I'm excited to see this.
Although I expect to be too busy until end of December, I definitely look forward to contributing to this and evolving it. In the meantime, please take a look at the Npgsql tests - a lot of them are exactly the kind of generic, functional tests that would fit perfectly into this.
One more comment - while it's really useful for this to exist as a self-contained project that includes connectors to specific ADO.NET providers, it's also important that projects like Npgsql and MySqlConnector be able to use this in their solutions, and to override certain things. In other words, there will need to be a generic abstract nuget that specific ADO.NET drivers can consume, extending the abstract test suite. Some tests may be overridden (so everything needs to be virtual), some would be ignored/skipped (not relevant for that provider), etc. etc.
bgrainger commentedon Nov 21, 2017
Thanks; that's a great suggestion (and I hadn't thought that far ahead yet...). Created mysql-net/AdoNetApiTest#3 to discuss it (without hijacking this issue).
23 remaining items
[monodroid, java-runtime] Add local date time offset monovm prop (#7331)
[monodroid, java-runtime] Add local date time offset monovm prop (#7331)
[monodroid, java-runtime] Add local date time offset monovm prop (#7331)
dotnet-policy-service commentedon Dec 23, 2024
Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.
This process is part of our issue cleanup automation.
dotnet-policy-service commentedon Jan 6, 2025
This issue will now be closed since it had been marked
no-recent-activitybut received no further activity in the past 14 days. It is still possible to reopen or comment on the issue, but please note that the issue will be locked if it remains inactive for another 30 days.