Thursday, April 02, 2009

DataObjects.Net v4.0: status update

Briefly, we're still fighting with schema evolution and LINQ support, but the end is near.

LINQ support

Currently we have the following set of issues that must be implemented to consider LINQ provider implementation project is finished:

PriorityCategoryImplementation levelTask
1Type mappingLINQ to RSE translatorEnum
1Type mappingLINQ to RSE translatorNullable vs NULL ternary logic in SQL
1Type mappingRSE to SQL translatorImplicit numeric casts
1Type castLINQ to RSE translator"is" operator
1LINQ methodsBothConcat LINQ method
1LINQ methodsBothUnion LINQ method
1LINQ methodsBothExcept LINQ method
1LINQ methodsBothIntersect LINQ method
1LINQ methodsLINQ to RSE translatorGroupBy with aggreagates
1LINQ methodsLINQ to RSE translatorGroupJoin
1LINQ methodsLINQ to RSE translatorFirst, Single, Last, *OrDefault as subquery result (including .Select(x => new {x, x.Customers.First}) )
1IssueLINQ to RSE translatorEntitySet.Count property & EntitySet.Contains method
1IssueLINQ to RSE translatorEntitySet as IQueryable
1IssueLINQ to RSE translatorCorrect column order for OrderBy
2Type mappingLINQ to RSE translatorByte array
2Type castLINQ to RSE translatorOfType LINQ method
2Type castLINQ to RSE translatorCast LINQ method
2Type castLINQ to RSE translator"as", (T) cast operators for type downcasting
2Type castLINQ to RSE translator"as" cast operator for type upcasting
2Type castLINQ to RSE translator(T) cast operator for type upcasting must lead to an error on query translation
2OptimizationRSE optimizerSQL APPLY rewriting
2OptimizationRSE optimizerRedundant join remover
2OptimizationLINQ to RSE translatorJoin first selected subquery returning * in .Select projections (including .Select(x => new {x, x.Customers}) )
2OptimizationRSE optimizerIndex access optimization: pi.Filter(...) -> si1.Range(...).Union(si2.Range(...)) .Union(...).Join(pi, ...) .Select(...).Filter(...)
2IssueLINQ to RSE translatorTake, Skip after OrderBy
3LINQ methodsLINQ to RSE translatorContains, Any, All for parameters of collection (IEnumerable) type

Most part of tasks of priority 1 are in development now, as well as Index access optimization (priority is 2). We're planning to finish with priority 1 queue during the next week. Priority 2 & 3 queue will require 1.5-2 more weeks, but must of these features aren't really necessary now. We're working on all of them mainly to get all our LINQ tests passing.

Btw, most likely it's a very important question: how do we test our LINQ implementation? You may find that now there are ~ 470 different tests for LINQ. Some of them are written by us, but a significant part of them was imported from the following projects:
- IQToolkit (The LINQ IQueryable Toolkit)
- ADO.NET Team's 101 LINQ Samples.

That's why we're pretty sure we'll cover all aspects of LINQ in the end.

Schema evolution

The progress here is not as good as with LINQ - from the point of end users nothing works here yet. But we already implemented the following parts of the framework:
- Generic modelling framework (Xtensive.Modelling) - a set of abstract base classes allowing to describe any models similar to SQL metadata.
- Update actions support for models. Basically, we can track & pass only the changes made to some model, rather then model itself to a remote server.
- Differential comparison support for models. Allows to create hierarchical difference structure taking into account comparison hits (such as node rename), as well as convert it to a set of update actions. This part isn't fully finished yet (some bugs still exist).
- Xtensive.Storage.Indexing.Model - an implementation of indexign storage model based on Xtensive.Modelling
- Converter of SQL DOM model to Indexing model (partially done: some bugs exist)
- Converter of Xtensive.Storage.Model model (Storage model) to Indexing model (in development).

What's left:
- Translation of differences to SQL update actions (for SQL storages)
- Translation of differences to index structure update actions (for indexing storages)
- Integration of all this stuff into Domain.Build pipeline.

Hopefully first version of our schema evolution layer will be available by the end of the next week.