Tigraine
Daniel Hoelbling talks about .NET

NHibernate removes items from Many-To-Many upon Update of Entity due to Model Binding

June 19th, 2010 . by Daniel Hölbling

Imagine the following scenario:

nh

Townships has two m:n collections mapped to Region. My Controller has special actions for updating these collections, while there is a generic Edit method that takes care of updating normal properties on Township. The code in question looks quite innocent:

[HttpPost]
public virtual ActionResult Edit([Bind]T item)
{
    if (!ModelState.IsValid) return View(item);
    using (var trans = session.BeginTransaction())
    {
        session.Update(item);
        trans.Commit();
    }
    return RedirectToAction("List");
}

Well, the problem is quickly found using NHProf:

image

Whenever I updated the Township entity all it’s associated Regions where cleared.

Turns out, the problem lies with the ModelBinder in MVC2: Since it reconstructs a new Township item and populates it with values from the request, there is no way for MVC to fill the WinterRegions and SummerRegions collection. So NHibernate got empty collections and assumed I removed all items from them and decided to persist that removal to the database, resulting in a DELETE.

There are two solutions to the problem: a) turn off Cascade.All b) Fill the collections before the update.

Since I already used the Cascade Behavior in other places I decided to go with b and select the entity prior to updating it. The resulting code looks like this:

[HttpPost]
public override ActionResult Edit([Bind]Township item)
{
    using (var trans = session.BeginTransaction())
    {
        var township = session.Get<Township>(item.Id);
        session.Evict(township);
        item.WinterRegions = township.WinterRegions;
        item.SummerRegions = township.SummerRegions;
        session.Update(item);
        trans.Commit();
    }
    return RedirectToAction("List");
}

Notice that it is important to first evict the fetched entity from the session, otherwise you’ll get an Exception stating that the same identified is already associated with this session cache.

To be honest: I don’t feel particularly fond of this solution, if anyone can point out a better solution please leave a comment or email me. While at it, it would be nice to be able to change the cascade behavior of entities for one session (like FetchMode for one criteria).


Storing binary data in NHibernate / ActiveRecord

August 19th, 2009 . by Daniel Hölbling

I believe the simplest way to store binary data is to just put in the database. Whenever I’ve agreed to throw data to a disk I’ve had issues with deployment, administration or disaster recovery.

Simply put: Once you have a dependency from your database to your file system, you no longer have the luxury of only thinking about recovering the database. You now need to keep two pieces of your system “safe”, both requiring a completely different toolset than the other.

Besides the obvious second point of headache for backup/recovery, you also bring yourself into a world of hurt for deployment / maintenance scenarios.
Filesystem access rights can be a huge pain in the ass, and having to set them right (and keep them that way) is usually a time-bomb waiting to go off.

So, storing your binary data in the db solves many problems, but some new ones arise. Mostly implementation details, but I’d like to show you some things to keep in mind when writing binary data to db.

NHibernate supports no lazy loading of instance fields

image While with conventional ADO.NET I’d just put the binary data as a column inside the table it belongs to, NHibernate requires you to do things different. If you map your data like that NHibernate will fetch it whenever you read objects from that table, meaning that you’ll be querying large binary data fields for no reason, causing you application performance to significantly degrade over time.

 

What you want is to have NHibernate fetch that field only if it is accessed (lazy load it), and that’s not possible for fields inside a class, but it is possible for references. So your database schema should look like this:

image

And your mapping will look similar to this (I’ll use ActiveRecord for easier understanding):

[ActiveRecord]
public class Invoice : ActiveRecordBase<Invoice>
{
    [PrimaryKey]
    public int Id { get; set; }

    [BelongsTo(Lazy = FetchWhen.OnInvoke, Cascade = CascadeEnum.SaveUpdate)]
    public BinaryData ScannedInvoice { get; set; }
}

[ActiveRecord]
public class BinaryData : ActiveRecordBase<BinaryData>
{
    [PrimaryKey]
    public int Id { get; set; }

    [Property(ColumnType = "BinaryBlob", SqlType = "IMAGE", NotNull = true)]
    public byte[] Data { get; set; }
}

Now whenever your Invoice is saved/inserted NHibernate will also check if BinaryData has to be updated/inserted, while only loading the binary field if you actually access the Invoices.ScannedInvoice field.


The fairy tale of binary blob fields

August 17th, 2009 . by Daniel Hölbling

One of the main advantages students get from being members of imagineClub is that they get access to uploaded course materials through the website. Naturally, the new site has to support file upload and download somehow, and yesterday I started implementation of that feature.

In theory this sounds really simple, especially since the file upload in MonoRail is so trivial I figured it wouldn’t be a problem to implement.

One major thing to consider when designing a file upload feature is the question: Save to disk or save to database? Let’s look at the two options:

Save to disk:

Pro: Very easy
Con: Requires metadata to be kept in the database. Could go out of sync with the db. Requires backup. Requires special permissions.

Save to db:

Pro: Zero setup. Data all in one place, backup hugely simplified. Enforces data integrity
Con: Non-trivial implementation.

Now, I naturally went with the db option. Deployment is hugely facilitated if you don’t need to look at file permissions, and most hosters have databases backed up anyway. So things go south, the only thing I need to recover the site would be the database file.

Some searching revealed that binary data could be mapped to the database through AR quite easily:

[Property(ColumnType = "BinaryBlob", SqlType = "varbinary(MAX)")]
public byte[] BinaryData { get; set; }

Problem with that is that it crashed ALL of my database dependant unit-tests:

------------ System.Data.SQLite.SQLiteException : SQLite error

near "MAX": syntax error

Apparently SqlLite can’t figure out that MAX thing and will crash. Since it would accept a numeric value instead I looked at the SqlServer 2008 documentation for varbinary to find out what MAX would be. Turns out it’s exaclty 2147483647 (2^31-1), so my natural reaction was to change the SqlType to be exactly varbinary(2147483647) instead of MAX. Now SqlLite can interpret it and all tests run great again, but creating the schema on SqlServer isn’t possible any more due to the following (odd) error:

The size (2147483647) given to the column 'BinaryData' exceeds the maximum allowed for any data type (8000).

So, what we just saw is a leaky abstraction inside the ORM. But NHibernate never claimed to abstract the DB completely away from me, so we’ll not use that against it. NHibernate explicitly supports these scenarios and in a real NHibernate scenario it’s just a matter of having two different mapping files, one mapping to the appropriate SQLite datatype and the other mapping to the Sql2008 datatype that would be varbinary(MAX).

But, I’m not using NHibernate here, I’m using ActiveRecord that handles mapping through attributes on the data classes, and I’ve no intention of using #ifdef statements anywhere around my code.

The problem here is mainly that whenever you are trying to use two different RDBMS at once you are limiting yourself to the least common denominator, and you have to deal with that.

I won’t be able to use advanced Sql2008 features, and I also won’t be able to use anything fancy inside SQLite either.

The least common denominator in this case is the datatype IMAGE, something that Microsoft is discouraging people to do in their documentation:

image

This puts me in a delicate position since the imagineClub website is hosted on a server I don’t control. So I could just wake up one morning and seeing the iC website down because the hosting company decided to upgrade all users to 2010 (or whatever version the next SQL Server will have).

And I know, usually providers send out warning for stuff like this, but I doubt that through all the structural changes with imagineClub lately they even know where to send those warnings to.

So: Long story short, use image over varbinary(MAX) if you plan on doing in-memory SQLite testing, just keep in mind that your app will break when you upgrade to a newer version of SqlServer.

Update: Looks like Krzysztof Kozmic had the same issues and found a quite clever solution for that. I’m not totally clear on how to do this with ActiveRecord, but it’s a very pragmatic approach to a problem that seems to not have a perfect solution anyway.


ActiveRecord gotchas when testing with an in memory database.

July 15th, 2009 . by Daniel Hölbling

If the title sounds familiar to you, it’s intentional. After having to deal with this in pure NHibernate it came around to also bite me with ActiveRecord.

In short: in-memory SQLite will drop the schema whenever you close the NHibernate ISession object (since ActiveRecord uses NHibernate behind the scenes this poses a problem to us).

So, assuming you have setup ActiveRecord using an InPlaceConfiguratonSource similar to this:

IDictionary<string, string> properties = new Dictionary<string, string>();
properties.Add("connection.driver_class", "NHibernate.Driver.SQLite20Driver");
properties.Add("dialect", "NHibernate.Dialect.SQLiteDialect");
properties.Add("connection.provider",                "NHibernate.Connection.DriverConnectionProvider");
properties.Add("connection.connection_string", "Data Source=:memory:;Version=3;New=True;");
properties.Add("show_sql", "true");
properties.Add("proxyfactory.factory_class",                "NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle");

var source = new InPlaceConfigurationSource();
source.Add(typeof (ActiveRecordBase), properties);

ActiveRecordStarter.Initialize(source, typeof (Member).Assembly.GetTypes());
ActiveRecordStarter.CreateSchema();

You will not be able to run any queries against it because there is no schema present. In fact the code will consistently blow up with a SQLiteException stating “no such table: ….”. Unfortunately it’s not really possible to change the SessionFactory implementation here because that code is inside ActiveRecord.

Thank god I found this handy guide that suggested subclassing the DriverConnectionProvider class to replace the CloseConnection call with a fake like this:

public class SqLiteInMemoryTestingConnectionProvider : NHibernate.Connection.DriverConnectionProvider
{     public static System.Data.IDbConnection Connection = null;     public override System.Data.IDbConnection GetConnection()     {         if (Connection == null)             Connection = base.GetConnection();         return Connection;     }     public override void CloseConnection(System.Data.IDbConnection conn)     {     }
}

I then had to pass that new ConnectionProvider into NH through the configuration and all was well:

properties.Add("connection.provider",                "ImagineClub.Tests.SqLiteInMemoryTestingConnectionProvider, ImagineClub.Tests");

Only catch is, this doing the AR Initialization is painfully slow so I wrote a baseclass for my xUnit tests that makes sure AR Init is only run once and that the ActiveRecordStarter.CreateSchema() is run before every test (xUnit runs the testclass constructor before each test):

The final implementation for all my tests looks like this:

public class ActiveRecordInMemoryTestBase
{     public ActiveRecordInMemoryTestBase()     {         if (!ActiveRecordStarter.IsInitialized)             Initialize();         ActiveRecordStarter.CreateSchema();     }     private static void Initialize()     {         IDictionary<string, string> properties = new Dictionary<string, string>();         properties.Add("connection.driver_class", "NHibernate.Driver.SQLite20Driver");         properties.Add("dialect", "NHibernate.Dialect.SQLiteDialect");         properties.Add("connection.provider",                        "ImagineClub.Tests.SqLiteInMemoryTestingConnectionProvider, ImagineClub.Tests");         properties.Add("connection.connection_string", "Data Source=:memory:;Version=3;New=True;");         properties.Add("show_sql", "true");         properties.Add("proxyfactory.factory_class",                        "NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle");         var source = new InPlaceConfigurationSource();         source.Add(typeof (ActiveRecordBase), properties);         ActiveRecordStarter.Initialize(source, typeof (Member).Assembly.GetTypes());         ActiveRecordStarter.CreateSchema();     }
}

public class SqLiteInMemoryTestingConnectionProvider : NHibernate.Connection.DriverConnectionProvider
{     public static System.Data.IDbConnection Connection = null;     public override System.Data.IDbConnection GetConnection()     {         if (Connection == null)             Connection = base.GetConnection();         return Connection;     }     public override void CloseConnection(System.Data.IDbConnection conn)     {     }
}

Caution: If you copy/paste the above make sure to change the namespace and assemblyname for your SqLiteInMemoryTestingConnectionProvider to match yours.

So a database dependant unit test would look like this:

public class DatabaseTest : ActiveRecordInMemoryTestBase
{     [Fact]     public void DatabaseIsEmpty()     {         Assert.Equal(0, Member.FindAll().Length);     }
}

Initial runs are quite slow due to the AR Init, but the whole test-suite should run quite fast since only the schema creation is run before each test.


Running from trunks can be tough

May 30th, 2009 . by Daniel Hölbling

I just spent almost 10 hours running in circles collecting different releases of NHibernate, Fluent NHibernate and castle, in a rather futile attempt to make them work together.

NHibernate is currently in 2.1 Beta1 at revision 2.1.0.2001 referencing the latest Castle.DynamicProxy2 at revision 2.1.0.0.

Fluent NHibernate’s current trunk compiles against NHibernate 2.1.0.1001 that in turn is compiled against Castle.DynamicProxy2 2.0.3 thus breaking.

So, while the NHibernate Project and the Castle project managed to match their versions pretty well, the Fluent NHibernate trunk was not.

So, the obvious choice would be to just take all the assemblies that are packed with Fluent NHibernate and work from that. But by doing so I then lack Calst MicroKernel and Windsor, two libraries that are not packed with Fluent NHibernate.

At that point I gave up and simply recompiled the Fluent NHibernate trunk with the latest NHibernate trunk.

In case you need Fluent NHibernate with Castle Windsor i packed my results for you: FluentNHibernateWithCastle.7z

Hope this helps.


Fluent NHibernate gotchas when testing with an in memory database.

May 29th, 2009 . by Daniel Hölbling

What I love most about programmatic configuration is that it’s close to the test.
While we were carrying dozens of XML files around for testing before, now with DSL based configuration everywhere the configuration is usually pretty near to the test fixture, instead of residing in some arbitrary XML that only insiders can associate with the test.

The standard sample for using SqlLite and Fluent NHibernate usually looks like this:

return
    Fluently
        .Configure()
        .Database(SQLiteConfiguration.Standard.UsingFile("mydb.db3").ShowSql())
        .Mappings(m => m.FluentMappings.AddFromAssemblyOf<SessionFactory>())
        .ExposeConfiguration(SaveSchema)
        .BuildSessionFactory();

Where SaveSchema is a method that does a database rebuild.

Now, Fluent Nhibernate has in-memory databases built into the API. Just remove the UsingFile directive and you replace it with:

.Database(SQLiteConfiguration.Standard.InMemory().ShowSql())

Charming isn’t it? Now the only problem is that you won’t be able to do anything with that DB since there is no schema present.

The in-memory database exists per session, so once you close the ISession the db is gone. Since the schema export from most samples operates in it’s own ISession the subsequent queries will still hit a blank database, and you’ll get an error stating there is no such table.

So my SessionFactory implementation had to change, since I needed to keep the configuration around for doing the schema export:

public class SessionFactory
{
    public static ISessionFactory CreateSessionFactory()
    {
        return
            Fluently
                .Configure()
                .Database(SQLiteConfiguration.Standard.InMemory().ShowSql())
                .Mappings(m => m.FluentMappings.AddFromAssemblyOf<SessionFactory>())
                .ExposeConfiguration((c) =>  SavedConfig = c)
                .BuildSessionFactory();
    }

    private static Configuration SavedConfig;

    public static void BuildSchema(ISession session)
    {
        var export = new SchemaExport(SavedConfig);
        export.Execute(true, true, false, false, session.Connection, null);
    }
}

And my tests then use a another factory method to construct the ISession object:

public static ISession CreateSession()
{
    var factory = SessionFactory.CreateSessionFactory();
    var session = factory.OpenSession();
    SessionFactory.BuildSchema(session);

    return session;
}

Hope this helps, quite an annoying problem and imo a far from perfect solution. Someone on the FNH mailing list suggested looking at the OneToManyIntegrationTester class but I couldn’t really extract any terribly useful information from there.


Comparing Castle ActiveRecord to Fluent NHibernate

May 27th, 2009 . by Daniel Hölbling

I’ve used my fair share of ORM tools lately and I keep coming back to NHibernate more and more. But sometimes it feels like “too much”. Going for a dedicated data access layer often feels like a bit of overkill and isn’t all that exciting to do, so for smaller projects I sometimes took the easy way and used LinQ to Sql.
Usually LinQ to Sql would then come around to bite me at some point and I’d hate myself for using it. So when I lately tried Castle ActiveRecord (watch this wonderful presentation by Ayende and Hammet at InfoQ: Painless Persistence with Castle ActiveRecord) I was blown away: A simpler way to build stuff with NHibernate.
No complete abstraction of NHibernate, but a layer ontop that makes life easier. (You can access the Session manually and do your crazy NHibernate stuff if you want to).

Since ActiveRecord was created to avoid some of the crazy configuration stuff that was in NHibernate at that time, I thought it might be interesting to compare ActiveRecord to the current state of NHibernate configuration, namely: Fluent NHibernate.

This is by no means a matchup, what you use hugely depends on your needs. And since both are essentially the same it boils down to taste. I haven’t used either for any real application (only old NHibernate with XML), so this is my personal rundown on the samples and spikes I did lately.

Let’s assume the typical Blog sample model:

model

I’ll only focus on the BlogPost class since it is on both ends of a parent/child relationship.

Mapping

First, how does the the ActiveRecord mapping look like:

[ActiveRecord]
public class BlogPost : ActiveRecordBase<BlogPost>
{
    [PrimaryKey]
    public int Id { get; set; }
    [Property]
    public string Headline { get; set; }
    [Property]
    public string PostText { get; set; }
    [Property]
    public DateTime PostedOn { get; set; }
    [BelongsTo]
    public User Poster { get; set; }
    [HasMany]
    public IList<Comment> Comments { get; set; }
}

Now the NHibernate mappings:

public class BlogPost
{
    public virtual int Id { get; set; }
    public virtual string Headline { get; set; }
    public virtual User Poster { get; set; }
    public virtual IList<Comment> Comments { get; set; }
    public virtual string PostText { get; set; }
    public virtual DateTime PostedOn { get; set; }
}
public class BlogPostMap : ClassMap<BlogPost>
{
    public BlogPostMap()
    {
        Id(m => m.Id);
        Map(m => m.Headline);
        Map(m => m.PostText);
        Map(m => m.PostedOn);
        HasMany(m => m.Comments);
        References(m => m.Poster);
    }
}

Cool, both APIs look good. Very readable and no XML.

NHibernate has the ability to have the Model separated from the mappings, a feature that I like when doing enterprise applications. But for something simpler, not really relevant. Where Fluent NHibernate really blows AR away is the auto mapper feature Ayende is so happy with.

Initialization

ActiveRecord hasn’t departed from XML yet, but there are three ways to load a configuration. The simplest would probably be the inplace configuration source, but it’s more or less a in-memory mirror of the XML config:

var properties = new Hashtable();
properties.Add("hibernate.connection.driver_class", "NHibernate.Driver.SQLite20Driver");
properties.Add("hibernate.dialect", "NHibernate.Dialect.SQLiteDialect");
properties.Add("hibernate.connection.provider", "NHibernate.Connection.DriverConnectionProvider");
properties.Add("hibernate.connection.connection_string", "Data Source=:memory:;Version=3;New=True;");
var source = new InPlaceConfigurationSource();
source.Add(typeof (ActiveRecordBase), (IDictionary<string, string>) properties);
ActiveRecordStarter.Initialize(typeof(BlogPost).Assembly, source);
ActiveRecordStarter.CreateSchema();

The Fluent NHibernate configuration looks way better since you don’t have to worry about connectionstrings or other properties that nobody remembers:

public static ISessionFactory CreateSessionFactory()
{
    return
        Fluently
            .Configure()
            .Database(SQLiteConfiguration.Standard.InMemory())
            .Mappings(m => m.FluentMappings.AddFromAssemblyOf<BlogPost>())
            .ExposeConfiguration(BuildSchema)
            .BuildSessionFactory();
}

private static void BuildSchema(Configuration config)
{
    new SchemaExport(config)
        .Create(false, true);
}

I’d vote for Fluent NHibernate on this one, but having the configuration externalized in XML is something you’d probably do anyway.

Where the differences really begin are the querying:

Querying with ActiveRecord is mostly done through static methods, so ease of use is king here:

var post = BlogPost.Find(1);

While NHibernate requires me to retrieve a ISession before giving me access to anything.

var mySession = factory.OpenSession();
var post = mySession.Get<BlogPost>((object)1);

The one is more OO, the other is simpler. I have to say I prefer the added complexity since it makes it easier to test that your code interacts with the session object. (Although I’d not overuse that, going against an in-memory db and verifying the results should be the better way to create durable tests).

Since both AR and NH use the NH Criteria API I won’t compare complex queries.

So, how to conclude on the usability? Not having to think about how to pass the session around is quite nice, but when using NHibernate you could easily just write your session to some global and use it from there.

Validation

Now, what would a ORM be without a good validation framework.

ActiveRecord comes prepacked with the Castle.Components.Validator assembly that allows a rather cool attribute based validation API:

[Property, ValidateEmail, ValidateNonEmpty]
public string Email { get; set; }
var user = new User();
if (user.IsValid())
    user.Save();

While NHibernate doesn’t have a built-in Validation framework, there is the NHibernate Validator (documentation) project that also allows a very clean attribute based validation.

[NotNull]
public static string blacklistedZipCode;

But as before, it’s a bit “heavier” than AR and allows you to have external ValidatorClasses if you need them. Also cool, but something I don’t really care about is that NHV is also XML based, so you could allow your users/admins to tweak the validation rules after deployment.

I guess that’s enough for now, use what you find suits your project better.  :)


Mapping large text or binary values with NHibernate

December 19th, 2008 . by Daniel Hölbling

I encountered the following error when trying to map a large String to my SQL2005 database:

SQL Error Code -2146232060: “String or binary data would be truncated”

The issue here is that Nhibernate maps all string values by default as nvarchar(255) and so inserting something bigger to a field causes this nasty sql error. My mapping declaration looked like this:

<property name="Comments" />

After some searching I found Ayende’s post on NHibernate and large text fields gotchas that almost solved the issue, except for one thing, I didn’t know where to put the sql-type attribute. Turns out it’s defined in chapter 15 of the NHibernate doc (while mapping files are chapter 5).  
The sql-type=”NTEXT” attribute can only reside on the <column node beneath the <property node. So the correct mapping looks like this:

<property name="Comments" type="StringClob">
  <column name="Comments" sql-type="NTEXT"/>
</property>

If you don’t define the sql-type attribute even the StringClob field will be created as a nvarchar(255) by NHibernate (but it can map to a NText field if the schema exists).


Virtual member call in constructor and NHibernate

November 7th, 2008 . by Daniel Hölbling

As you may have noticed, I’ve been doing some NHibernate work lately and really had a great time with this absolutely amazing ORM. Especially now that Microsoft abandoned Linq to SQL I really feel good about having made the step towards NHibernate.

Yesterday was one of those tricky days when something breaks and you have no clue why.

I have a table called “Orders” and there are “OrderItems”.

image

By writing the tests upfront, I found out that I’d like to be able to just say Repository.Add(Order) instead of persisting the Order and afterwards looping through the OrderItems and persisting them too.

To achieve this I changed the mapping to something like this:

<set name="OrderItems" cascade="all">
  <key column="OrderId" />
  <one-to-many class="OrderItem"/>
</set>

The cascade=all statement is what I searched for initially. When the order gets persisted, all OrderItems in the collection get persisted too, and everything is well.

But, since my POCO object looks like this:

public class Order
{
    public virtual long Id { get; set; }
    public virtual ISet<OrderItem> OrderItems { get; set; }
}

I got NullReferenceException whenever I tried to access OrderItems on new Order objects. And I thought, hey.. kinda sucks newing up the collection in my business layer, why not init it in it’s constructor:

    public Order()
    {
        OrderItems = new HashedSet<OrderItem>();
    }

So I could just do Order.OrderItems.Add(OrderItem) without having to instantiate a HashedSet anywhere.

I got a bit cautious when Visual Studio underlined OrderItems and made the cryptic announcement: “Virtual member call in constructor”.

Totally unaware of what this means, I just went on and ran my tests.

Imagine my face when almost all my tests failed due to an omnious nHibernateException:

NHibernate.HibernateException: Illegal attempt to associate a collection with two open sessions

I didn’t figure this out until today, but it had to do with the “Virtual member call in constructor” warning. I discovered this post by Brad Abrams that explains the topic.

Looks like if you set the collection in the constructor of the POCO object NHibernate will break with the above Exception.

Solution to avoid this? Init the collection from calling-code instead of within the object.

What would have helped the issue would be to not have concrete POCO objects but rather use interfaces instead of virtual members (Read Fabio Maulo’s post entity-name in action: Entity Abstraction on that topic).