Shay Rojansky

.Net: [MEVD] More test cleanup (#13320) Another batch of test cleanup work - this almost concludes what I have planned for the tests. * We no longer create any collections with GUID names; all collections have a manually-defined name, and are deleted when a test starts running. This ensures that no collections are left behind if a test crashes or is stopped in the middle, and makes it easier to examine the database. * Removed global SimpleRecord and SimpleFixture, each test suite now has its own record and fixture. * Cleaned up and renamed DistanceFunctionTests to only focus on testing the different distance functions, and added corresponding tests elsewhere. * Added IndexKindTests to specifically cover different index types.

.Net: MEVD data type tests (#12567) Closes #12505

.Net: [MEVD] Go over all key types, add support for Guid (#13315) * Added new KeyTypeTests to cover provider key support, alongside the existing DataTypeTests and EmbeddingTypeTests; moved all three into TypeTests folder. * As discussed offline, this adds Guid-as-string key support for all providers. From research this seems to be very standard practice across a wide range of databases: JSON-based databases generally encode various types (e.g. DateTime) as string, and Guids are standard there. Sqlite is similar, with Microsoft.Data.Sqlite supporting it natively. This notably allows upper layers to assume that they can always use Guid, which helps MEDI ([see this issue](https://github.com/dotnet/extensions/issues/6973)). * Per-provider notes * Azure AI Search: string only supported, added Guid-as-string as an option ([docs](https://learn.microsoft.com/en-us/rest/api/searchservice/data-type-map-for-indexers-in-azure-search#bkmk_sql_search) encouraging mapping Guid to strings) * MongoDB/CosmosMongoDB: Added int and long support (Guid was already supported). * CosmosNoSql: Added Guid-as-string (standard practice) * Pinecone: Added Guid-as-string * Redis: Added Guid-as-string * SQL Server: removed DateTime and `byte[]` as key types (as above) and added Guid. * SQLite: The Microsoft.Data.Sqlite (the low-level ADO.NET provider we're built on top of) supports GUIDs as TEXT; so it's really not us deciding to support - it's the lower-level driver. In addition, we support mapping boolean to INTEGER (1/0), which seems similar. Added support as data property as well, not just key. Closes #11784 Closes #12182

.Net: [MEVD] Implement key auto-generation (#13466) * Adds IsAutoGenerated to key property definition and attribute. * When auto-generation is on, saving a record where the key property is the default (empty GUID, zero integer) triggers the auto-generation. It's still possible to set the key property to a non-default value, overriding the auto-generation. * Auto-generation can happen in the database (mainly on the relational databases), or on the cilent side (Guid.NewGuid()). * IsAutoGenerated is null by default, meaning that the provider decides whether to turn on autogeneration by default, based on the property type. This means that GUID properties are always auto-generated by default (if the key property is left unassigned), and int/long properties are auto-generated on relational providers. This aligns with the behavior in Entity Framework. * All providers support GUID generation, by doing it on the client before sending the record (Guid.NewGuid()). Some providers have special behavior, e.g. PG generates UUIDv7 and SQL Server generates sequential IDs in the database (much better for indexing). Universal GUID auto-generation makes things much simpler for upper layers, specifically Microsoft.Extensions.DataIngestion. Closes #11485

.Net: [MEVD] Cosmos NoSQL provider work on keys, partition keys and point reads (#13550) This is a sizable cleanup/redo of large chunks of the Cosmos NoSQL provider; I've broken it down to three commits for easier reviewing. * Cosmos NoSQL collections can now have either string/Guid as their key, or CosmosNoSqlKey. In the former case, this configures the partition key to be the same as the string/Guid document key - this is a sensible default that's common (and encouraged) in Cosmos. All other scenarios require CosmosNoSqlKey: * A single partition key that isn't the document ID * A hierarchical partition key (more than one property) * No partition key (legacy/discouraged) * Note that CosmosNoSqlKey is only passed to GetAsync and DeleteAsync; on the user's .NET record type, the properties are there as usual (e.g. a string document Id property, an int partition key property). This introduces a discrepancy for the first time between the collection's TKey (CosmosNoSqlKey) and its actual key property on the .NET type (a string); this made some things trickier, but is the correct way to do things for composite key cases (we'd do the same if we implement composite keys for relational providers). * Added support for Cosmos hierarchical partition keys (2-3 properties instead of just one). * GetAsync no longer does a SQL query with a WHERE clause, but rather an efficient point read via ReadItem. The multiple-key overload does ReadManyItemsAsync, which again is far more efficient. * UpdateAsync and DeleteAsync now correctly supply the partition key in the call's request options, again making it much more efficient. * Various other cleanup and fixes all around.

.Net: Deduplicate embedding generation management (#13615) Closes #12508

.Net: [MEVD] Ensure identifiers are properly quoted (#13470) Closes #12161

.Net: [MEVD] Support .Any(x => x.Contains(...)) in filters (#13471) ~**NOTE: This PR is based on top of #13470, review 2nd commit only**~ Closes #13152

.Net: [MEVD] Support DateTime/DateTimeOffset/DateOnly/TimeOnly across providers (#13569) See https://github.com/microsoft/semantic-kernel/issues/11286#issuecomment-3904027209 for some design notes. Closes #11286 Closes #11086 --------- Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>

.Net: Final test cleanup (#13328) I think this completes the cleanup work I had planned on the tests.