LINQ to SQL and Serialization
One of the important that things that you will likely have to do at one point or another with entity objects is serialize them. This could be for Web Services most commonly but could be for any sort of storage or transport scenario.
There are a number of pitfalls with this scenario however, primarily because LINQ to SQL will very likely generate circular references into your entity model from your data and XML Serialization will fail outright in that scenario. For example, say you have a customer and projects table and if you let LINQ generate the one to many relationship it will create Customer entity with a Projects property and a Project entity with a Customer property.
For code scenarios this is probably a good thing - you want to be able to see all Projects and filter that list in your code.
Unfortunately in a serialization scenario this doesn't work because you essentially have a circular reference - Customer ->Projects -> Customer. So we have a 1 -> Many and a 1 - 1 relationship going back to the original object here. To demonstrate you can actually do whacky stuff like this:
CustomerEntity entity = Customer.Context.CustomerEntities.Single(c => c.Pk == 1);string address = entity.Projects.ToArray()[0].Customer.Address;And while that's a whacky example, this sort of setup makes sense. When you look at an Project you will liklely want to know about the Customer that is associated with the Project. And when you're looking at a Customer you'll want to know about all the Projects associated with the customer. A classic circular reference scenario.
At an abstract level that's fine but when you want to serialize this arrangement there are problems because the serializer doesn't know what do to with the circular references. So if you run code like this:
CustomerEntity entity = Customer.Context.CustomerEntities.Single(c => c.Pk == 1);MemoryStream ms = new MemoryStream();
XmlTextWriter writer = new XmlTextWriter(ms, new UTF8Encoding());
XmlSerializer serializer =new XmlSerializer( typeof( CustomerEntity ) );
writer.Formatting = Formatting.Indented;writer.IndentChar = ' ';writer.Indentation = 3;
serializer.Serialize(writer, entity);
byte[] Result = new byte[ms.Length];
ms.Position = 0;
ms.Read(Result, 0, (int)ms.Length);string XmlResultString = Encoding.UTF8.GetString(Result, 0, (int)ms.Length);
You'll get:
A circular reference was detected while serializing an object of type TimeTrakker.CustomerEntity.
There are a couple of ways to get around this but they're not exactly pretty as they involving changing the model.
So when you think about Customers and Projects relationship for example, is it really appropriate to have a Projects property in your CustomerEntity? By itself this expression doesn't really represent anything useful, especially if the system grows and there are lots of projects. So for serialization purposes you really wouldn't actually like to see this relationship.
So you can actually hide this relationship for serialization by marking the child property as internal. On the other the ProjectEntity probably should have a CustomerEntity and that always should be visible because it's a 1-1 relationship and you'll likely need that data and it would likely be OK to serialize so that relationship stays public:
You can also set the ChildProperty option to false altogether to completely cut off the relationship through the entity. Using Internal still makes the relationship available in the middle tier (if you use one that is), but hides it for XmlSerialization (which works off public properties only).
As I said this is not a great workaround because by removing this relationship you're also removing the ability to run entity relation ship queries through LINQ. So by doing the above this sort of thing will no longer work:
CustomerEntity entity = Customer.Context.CustomerEntities.Single(c => c.Pk == 1); var query2 = from c in Customer.Context.CustomerEntities
from p in c.Projects
where p.CustomerPk == 1select new { c.Company, p.ProjectName};
this.gdList.DataSource = query2;this.gdList.DataBind();Remove the relationship or set it to internal (outside of the bus object) you loose the ability to the implicit relationship that is implied and you'd have to explicitly define it like this:
var query2 = from c in Customer.Context.CustomerEntities
join p in Customer.Context.ProjectEntities on c.Pk equals p.CustomerPk
select new { c.Company, p.ProjectName};
this.gdList.DataSource = query2;this.gdList.DataBind();return;So the above workaround of turning off the relationship mapping might get you by in SOME scenarios. The internal flagging allows you to hack around the problem by making the property essentially invisible for XmlSerialization. However, that's not going to work for Binary Serialization and is also going to be a problem for WCF based Web Services that don't use the basicHttpBinding - all other formats use the lower level variations of binary serialization that work of field state rather than property values.
There's a solution for WCF however - the Entity designer has a Serialization option for the generating a WCF [DataContract] and [DataMembers] on the entities generated:
This option controls only whether the entities are generated with a WCF [DataContract] attribute in the classes:
[Table(Name="dbo.Customers")]
[DataContract()]public partial class CustomerEntity : INotifyPropertyChanging, INotifyPropertyChanged
and for each property:
[Column(Storage="_LastName", DbType="NVarChar(50)", UpdateCheck=UpdateCheck.Never)]
[DataMember(Order=4)]public string LastName
If you specify a non-public flag (Internal, Protected or Private) for the relationship (or any field for that matter), the DataMember property is not generated. This gives you pretty good control over WCF serialization as well as XML Serialization and ASMX Web Services.
For Binary Serialization that's not going to do anything however - but then again there shouldn't be a lot of need for binary serialization anymore with WCF filling that niche in most places.
.Serialization - a Special Scenario
All of this also got me to thinking about these object relationships. In code having an Projects property is useful because you can apply LINQ filter expressions against it. But when you persist via serialization you don't get that flexibility. So if you were to serialize you would get ALL of the child entities serialized which is probably not at all what you want. For example, in my simple time tracking app I'm using as my playground I have a customer entity with individual entries. I really don't want to serialize ALL entries. I MIGHT want to serialize a handful of new entries, but certainly not ALL of them once the application has been running for a few months.
The problem now is that we have a logical AND a physical relationship in this scenario. As non parameterized view the Projects property doesn't make any sense in a Customer object. However, a Entries property on an Invoice object might make perfect sense. In some cases it makes perfect sense to expose the relationship directly.But if you decide not to express the relationship explicitly between say Customer and Projects you loose some of the cool functionality that LINQ provides by automatically understanding relationships.
I'm not sure there's a solution to this. I think this same problem also exists with the ADO.NET Entity Framework (which uses a very similar approach although it's even more implicit about how relationships are defined by explicitly removing key fields).
<shrug>
Maybe some of the folks who've been doing OR/M development for some time with other tools like nHibernate can chime in. This is a thorny problem.
It seems to me that LINQ to SQL could actually solve these issues quite easily by providing a few more properties to set on entities and relationships. Such as a Serialized property on relations, and explicit serialization attributes on the entities. But even as it is you can bend LINQ to SQL to do as you need it seems.
The Voices of Reason
# re: LINQ to SQL and Serialization
It's almost like a second set of configurations for the relations would be needed.
I haven't thought this all the way through - I need to play with live data to see - but I think for now I'd rather stick with 'loose' relations that aren't visible as relations on the object (ie. Customers -> Projects) because in reality you'd rarely need to see Customers.Projects in an object graph. It's just about as easy to set up the relation or directly query projects for whatever operations you'd perfom on projects:
var q = context.CustomerEntities.All(); foreach(CustomerEntity cust in q) { foreach(Project p in context.ProjectEntities.Where( p.CustomerPk == cust.Pk) { // ... do whatever with projects } }
This code isn't really any less workable than if there was a relation - it's basically just an extra .Where() to enforce the relationship here.
It matters only for persistence; if you pass the object somewhere or if you serialize. Again I'm not sure if that's a realistic - or even desirable - object structure that you'd want to pass around because once there are a few projects, that relationship without a filter is pretty much meaningless for persistence or object passing.
# re: LINQ to SQL and Serialization
We use Castle ActiveRecord attributes to declare only the relationships we're interested in having in our model code. To be honest, I don't see this as a drawback, since in our case it has always been clearly one side that owns the relationship (if not, we revisited our model), and so we put our declarations on the side that owns it.
I'd love a counter-example for which this does not work well though, if you have one though.
I also prefer this approach since it forces one to think hard about how you will be using your entity objects and when they will be loaded. Having everything automagic is nice and RAD until you realize you're accidentally eager loading from a table with 10 million rows :)
(Culprits shall remain unnamed).
# re: LINQ to SQL and Serialization
# re: LINQ to SQL and Serialization
Well I'd think that would be somewhat common. Invoice -> LineItems -> Invoice. A lineitem might need information about the invoice. OTOH, the logical business object probably manages the invoice which logically also deals with the lineitems.
Not sure offhand but I suppose I am just brainstorming and trying to think ahead a bit for scenarios that might affect later dead ends.
@Ryan - yeah I suppose you can map additional objects that represent just the right relationship structure, but I think you'd want to avoid that as much as possible in order to minmize pollution your entity model with redundant types. But it may be the only way to do this. I think I'm with nexus on this though - being more conservative and using a minimalist approach with relations for the base model seems a prudent approach. With LINQ to SQL at least you do have the Internal option which precludes the relation from showing up during serialization but can still be used in the middle tier which potentially gives you the best of both worlds.
# re: LINQ to SQL and Serialization
My table hierarchy is Photos->Tags
and when i serialized my photo object , i took:
System.InvalidOperationException: There was an error generating the XML document. ---> System.InvalidOperationException: The type System.Collections.Generic.List`1[[Photo, App_Code.irdi89y2, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]] may not be used in this context.
I ve overcome this problem by using an partial class implementation.
public partial class Photo { public Photo EmptifyTags() { this.Tags= null; return this; } } usage: PhotoDBDataContext photoDB = new PhotoDBDataContext(); var templates = (from photo in photoDB.Photos where photo.IsTemplate.Equals(true) select photo.EmptifyTags()).Skip(start-1).Take(numVisible*2);
# re: LINQ to SQL and Serialization
# re: LINQ to SQL and Serialization
http://www.codeproject.com/KB/dotnet/linqsqlserialization.aspx
# re: LINQ to SQL and Serialization
CustomerEntity entity = new CustomerEntity{property1="test", ...}</code>) the properties within the entity representing the relational information like Projects in your example are not initialized and are null. By setting the relational properties in the entity to null before returning the entity object in a webservice (or using XmlSerialer) the entity serializes just fine. In your example if Projects is the only relationship to customers defined in your Linq to SQL classes you can make customers serialize like this: <code lang="c#">CustomerEntity entity = Customer.Context.CustomerEntities.Single(c => c.Pk == 1); entity.Projects = null;
Now, the only thing is that the Projects property is still serialized as an empty tag. Something kind of like this:
<CustomerEntity>
<Projects />
...
</CustomerEntity>
If this is not a problem then setting Projects to null is a fast solution. For my entity classes I added a member function called MakeSerializable to my partial class that sets all the properties to null and I just call that before returning the object.
# re: LINQ to SQL and Serialization
Thanks for all the articles on these topics they've been very helpful.
I tried this using a WCF service with BasicHTTPBinding. I managed to get the Parent Property to appear on the client side but it is always null. I have checked while debugging that the property is populated prior to passing up from the service so the contents is getting lost on the way.
Did you manage to get your example actually working with WCF.
Thanks
# re: LINQ to SQL and Serialization
# re: LINQ to SQL and Serialization
I have applied your article in my project and it worked superb.
# re: LINQ to SQL and Serialization
select new
{
ID = p.ID,
Title = p.Title,
Subcategory = p.Subcategory,
AuthorName = p.Author.Name, //NOTE THAT THIS POINTS AT A MEMBER OF A REFERENCE
Summary = p.Summary
};
The point is, how often in a webmethod are you really just interested in maybe a field or two from one of the object's circular references? I eliminated my issue by simply grabbing what I needed, creating a new generic type and serializing that instead. You can keep the ID if you need to reference it later.
Maybe I'm off base here, thoughts?
# re: LINQ to SQL and Serialization
The real problem, I think, is actually a combination of the basic problems with XML [it's a tree, not a graph] and the way the XmlSerializer works [which is to serialise objects on the presumption that they're a tree]. It would actually be possible to do this OK by writing out entity ref's and simply referring to these later on in the XML (which is why entity ref's were added to XML), and by doing this it's possible to serialise any arbitrary graph. Unfortunately, XmlSerialiser fails to do so. Just to demonstrate that it's perfectly possible, try this:
[Serializable]
class A
{
private A _sibling;
public A Sibling
{
get { return _sibling; }
set { _sibling = value; }
}
}
static void Main(string[] args)
{
A a = new A();
a.Sibling = a; // now that really *is* circular!
SoapFormatter sf = new SoapFormatter();
using (MemoryStream ms = new MemoryStream())
{
sf.Serialize(ms, a);
Console.WriteLine(Encoding.Default.GetString(ms.GetBuffer(), 0, (int)ms.Length));
}
Console.WriteLine("Hit the any key...");
Console.Read();
}
Note that the SoapFormatter is essentially useless now, though, because it doesn't support generics!
# re: LINQ to SQL and Serialization
www.signumframework.com
You can see some videos explaning how to make a client - server application here:
http://www.signumframework.com/VideoTutorials.ashx
Finally, in order to make the resrialization we use NetDataContractSerializer, that enables old remoting-style serialization over WCF services. This way it uses SharedTypes between client and server, enabling validation and change tracking in both sides. It also enables sending full objects graphs (not just trees).
http://www.signumframework.com/ServerContracts.ashx
Hope you find the framework as usefull as we do.
Disclaimer: I'm the lead developer of Signum Framework, but nothing I've said is an exageration :)
# re: LINQ to SQL and Serialization
public static void AddIgnoreForLinqForeignKeys(this XmlAttributeOverrides overrides, System.Type type) { var ignoreAttributes = new XmlAttributes(); ignoreAttributes.XmlIgnore = true; foreach (var propInfo in type.GetProperties()) { foreach (var attr in propInfo.GetCustomAttributes(false)) { var associationAttr = attr as System.Data.Linq.Mapping.AssociationAttribute; if (associationAttr != null && associationAttr.IsForeignKey) { overrides.Add(type, propInfo.Name, ignoreAttributes); } } } }
Then i need only add my linq types and i'm done:
XmlAttributeOverrides overrides = new XmlAttributeOverrides(); overrides.AddIgnoreForLinqForeignKeys(typeof(Data.DiscussionAnnotation)); overrides.AddIgnoreForLinqForeignKeys(typeof(Data.UserDiscussionAnnotationState)); overrides.AddIgnoreForLinqForeignKeys(typeof(Data.Notification)); overrides.AddIgnoreForLinqForeignKeys(typeof(Data.UserReadDiscussion)); overrides.AddIgnoreForLinqForeignKeys(typeof(Data.Task)); var serializer = new XmlSerializer(typeof(DiscussionDocument), overrides);
# re: LINQ to SQL and Serialization
For purely dynamic situations assigning null to references also works to prevent circular references and excessive serialization data, but it gets to be a pain to do this every time and this is where your approach might work better.
I suppose DataMember attributes though are the ones that you really need to worry about too.
# re: LINQ to SQL and Serialization
Your blog is quite useful. I pieced together information on LINQ to SQL Serialization and WF that you and your readers may find useful.
http://justgeeks.blogspot.com/2009/04/linq-to-sql-cant-be-serialized-when.html
I spent a lot of time looking for a good solution, I hope this helps others.
Thank you for your very useful blog.
Brent
# re: LINQ to SQL and Serialization
but I know this is not useful when we are working with many tables.
Nice blog, see yeah!
# Per-property overriding of XML attributes
using System.Xml.Serialization; using System.Reflection; using System.ComponentModel; using System.ComponentModel.DataAnnotations; public static class XsLinq { public static void SetupXmlAttributes(this XmlAttributeOverrides overrides, System.Type type) { var metadataAttrib = type.GetCustomAttributes(typeof(MetadataTypeAttribute), true).OfType<MetadataTypeAttribute>().FirstOrDefault(); var buddyClassOrModelClass = metadataAttrib != null ? metadataAttrib.MetadataClassType : instance.GetType(); var buddyClassProperties = buddyClassOrModelClass.GetProperties(); var properties = from buddyProp in buddyClassProperties join modelProp in type.GetProperties() on buddyProp.Name equals modelProp.Name; foreach (PropertyInfo prop in properties) { overrides.Add(type, propInfo.Name, new XmlAttributes(prop)); } } }
We setup the runtime Linq class as follows:
[MetadataType(typeof(User_Meta))] public partial class User { public class User_Meta { [XmlAttribute("first-name")] public object FirstName { get; set; } [XmlIgnore] public object Friend { get; set; } } }
And finally:
XmlAttributeOverrides overrides = new XmlAttributeOverrides(); overrides.SetupXmlAttributes(typeof(User)); var serializer = new XmlSerializer(typeof(User), overrides);
# Correction to my previous post
public static class XsLinq { public static void SetupXmlAttributes(this XmlAttributeOverrides overrides, System.Type type) { var metadataAttrib = type.GetCustomAttributes(typeof(MetadataTypeAttribute), true).OfType<MetadataTypeAttribute>().FirstOrDefault(); if ( metadataAttrib != null ) { var buddyClassProperties = metadataAttrib.MetadataClassType.GetProperties(); foreach (PropertyInfo prop in buddyClassProperties) { overrides.Add(type, propInfo.Name, new XmlAttributes(prop)); } } } }
# re: LINQ to SQL and Serialization
With this setup you could define serialization slices across your entity model without having to change it's structure.
An example:
[MetadataType(typeof(ProductMetadata))] public partial class Product { } public class ProductMetadata { [IgnoreForSerializationGroup("CategoryGroup")] Object Categories; } [MetadataType(typeof(CategoryMetadata))] public partial class Category { } public class CategoryMetadata { [IgnoreForSerializationGroup("ProductGroup")] Object Products; }
then a call to GroupableSerializer i imagine to be like this:
var productXML = GroupableSerializer(Product,"ProductGroup") var categoryXML = GroupableSerializer(Category,"CategoryGroup")
# re: LINQ to SQL and Serialization
# re: LINQ to SQL and Serialization
My problem involves Json Serializer of asp.net MVC framework, what would you suggest to this ? since the error is still the same.
IVAPM.Models.IVAPMEntities context = new IVAPM.Models.IVAPMEntities();
int pageIndex = Convert.ToInt32(page) - 1;
int pageSize = rows;
int totalRecords = context.MappUser.Count();
int totalPages = (int)Math.Ceiling((float)totalRecords / (float)pageSize);
var mapusers = from ob in context.MappUser.OrderBy("it." + sidx + " " + sord).Skip(pageIndex * pageSize).Take(pageSize)
select ob;
var result = new JsonResult();
result.Data=new {
total = totalPages,
page = page,
records = totalRecords,
rows = mapusers
};
return result;
# re: LINQ to SQL and Serialization
# re: LINQ to SQL and Serialization
# re: LINQ to SQL and Serialization
JavaScriptSerializer json = new JavaScriptSerializer();
json.RecursionLimit = 1;
json.Serialize(myLinqEntityInstance);
Option #2 is to use a 3rd party json library like JSON.Net (http://json.codeplex.com/), which gives you options on how to handle circular references:
JsonSerializerSettings jsSettings = new JsonSerializerSettings();
jsSettings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
JsonConvert.SerializeObject(myLinqEntityInstance, Formatting.None, jsSettings)
# re: LINQ to SQL and Serialization
It can be found at http://wcfmetal.codeplex.com/
# re: LINQ to SQL and Serialization
I have a problem with LINQ to SQL and C#, I try to change the Association Properties but this are atributes, how can i do? For example, just for one WebMethod I want to set the Table1-Table2 Association Child Property Access as Internal.
Thanks!
This is my code,
[WebMethod]
public List<Perfil> SE_ListarPerfilesXIdSistema (int IdSistema)
{
SeguridadDataContext seguridad = new SeguridadDataContext();
List<Perfil> perfil = (from p in seguridad.Perfil
where p.Sist_Id == IdSistema
select p).ToList();
return perfil;
}
# re: LINQ to SQL and Serialization
Can any one guide me?
# re: LINQ to SQL and Serialization
- Added [System.Xml.Serialization.XmlIgnore] to all EntitySet<t>
- Set all child associations to Internal
I double-checked that I got ALL of them. In fact I wrote a little parser to go through and parse and update the items, because it's about a thousand lines of xml.
Neither of these approaches worked. I see some suggestions that look interesting but would require a hundred hours of refactoring my code to implement; not an option for me.
It's driving me crazy that I've seen lots of posts saying that 1 or both of the above items worked for them, yet it doesn't work for me. I've triple checked, read lots of articles, recompiled, tried variations on different suggestions. Nothing.
Anything I might have missed?
# re: LINQ to SQL and Serialization