There are several ways you can write an object into Azure Table Storage. You can inherit from TableEntity class, you can implement ITableEntity interface, you can convert your entity into a DynamicTableEntity object or you can use an Adapter pattern that takes your original entity as inner object and itself implements ITableEntity interface.
Having said all that, none of these patterns really help if you want to write complex objects to Azure Table Storage. By complex objects I mean objects with nested properties which themselves may be complex objects with their own nested properties and so on.
And that is the exact challenge I had in one of the projects I was working on, writing and reading complex objects from Azure Table Storage. In addition, these complex objects could be of different types.
This is why I really like working on enterprise level projects, the unique problems they challenge you with, force you to invent and innovate.
To solve this problem, in a most elegant and generic way I have implemented an API which allows complex objects to be written and read from Azure Table Storage. I uploaded the package to Nuget.org:
https://www.nuget.org/packages/ObjectFlattenerRecomposer/
https://www.nuget.org/packages/ObjectFlattenerRecomposer.Core/
V2.0.0 and IEnumerable Property Support
With version 2.0.0 of the ObjectFlattenerRecomposer Api in Nuget, I have implemented support for any property type that implements IEnumerable interface. That is a big change and it covers as we know most commonly known collection types like List, Array, Dictionary, their generic versions and other interfaces that inherit from IEnumerable like ICollection, IReadOnlyCollection etc. With this change these properties would be automatically converted to json strings when flattened by the api, when reading the entity back from azure, the api will recompose the original property with the correct type and correct values all transparently.
So if you need full complex object support and ability to write/read IEnumerable properties to table storage I would recommend you to get the latest version (2.0.0) of the ObjectFlattenerRecomposer Api from Nuget. Try it and hope you enjoy it 🙂 Of course all feedback is welcome 🙂
So How Does Object Flattener Recomposer API Work?
As the name hints, there are 2 main functionality of this API.
First It flattens the complex object and turns it into a Dictionary <string, EntityProperty>. The clients (callers of the API) can at this stage assign this Dictionary to a DynamicTableEntity, set its PartitionKey and RowKey and happily write the DynamicTableEntity to AzureTableStorage.
To flatten the complex object, the API traverses the entire object graph using recursive reflection. Creates a string key by appending the names of the properties visited during the traversal from root object to the end node property (including the name of the end node property) delimited by ‘_’. And the Value is the EntityProperty object created using the actual value of the end node property. The API then adds this (key, value) pair into the Dictionary <string, EntityProperty>. It continues this until it traverses the entire object graph of the root object and all end node properties are saved into the Dictionary<string, EntityProperty>.
(If you want to learn a bit more about its implementation and object traversal by recursive reflection, I recently published a post about it: https://wordpress.com/post/doguarslan.wordpress.com/152)
As an example, imagine object A has 2 properties P1 and P2. P1 is a class and it has a value type property P3 with value V3. P2 is a struct and it has a value type property P4 with value V4.
So object A looks like: (P1(P3:V3), P2(P4:V4))
This object is not writable to Azure Table Storage and none of the methods that I mentioned in the beginning of the post can help writing this to Table Storage, there is no existing API that can write this to Table Storage either.
“P2_P4”, EntityProperty(V4)
Magic:) So now the only thing client needs to do, is new up a DynamicTableEntity object, assign this EntityProperty dictionary to it, set PartitionKey and RowKey and you guessed it, write it to Azure Table Storage.
Moving on to the next part of the puzzle, now we wrote our flattened entity to table, we want to read it back. When we read it back, we want to pass it back to the client not in the form of a flattened DynamicTableEntity but exactly in the form of the complex object that the client code passed to our API to write it to Azure Table Storage.
This is the ConvertBack method in the Object Flattener Recomposer API. This method takes in the flattened EntityProperty Dictionary as input and recomposes the complex object of type T, which was originally passed to the Flatten method to be written to Table Storage in a complete generic and transparent way.
The ConvertBack method instantiates an instance of type T and all of its properties, recomposes its original state using the flattened EntityProperty Dictionary. How does it do this? I will leave it for you to find out by downloading the package from Nuget:)
There you go, complex objects get written and read from Azure Table Storage in a complete generic, schema-less, version tolerant and queryable way. Your complex objects do not need to implement any interface nor do they need to inherit from a specific base class (like TableEntity), you do not need to implement separate Adapters, you do not need to define any schema or version, in fact your complex objects do not even need to be complex:) This API works perfectly even if the objects just have value type properties and no nested complex properties.
The properties need to have both Getter and Setter. The Getter and Setter do not need to be public, they can be protected or private.
The other requirement is that there should not be a recursive (cyclic) reference in the object graph. The Flatten method detects a recursive reference in any part of the object graph and terminates the flattening operation.
There may be some use cases where you may want to exclude certain properties from being flattened ie. you have some Get only properties and you do not want to add even a private Setter or the property has a recursive reference and you want to exclude it in that case all you need to do is to add [IgnorePropertyAttribute] attribute to the top of the properties you want to exclude.
In case of recursive referenced objects, v1.1.3 and above versions of the Object Flattener Recomposer API detects the recursive references anywhere along the object graph and exits from flattening operation, returning a null result. This avoids the flattener from going into an infinite loop traversing inside the recursive path in the object graph.
Yes. From version 1.1.4.0 and on, you can specify a custom property name delimiter as part of the newly added EntityPropertyConverterOptions parameter, by setting its PropertyNameDelimiter property.
https://www.nuget.org/packages/ObjectFlattenerRecomposer/
using ObjectFlattenerRecomposer;
//Flatten object and convert it to EntityProperty Dictionary.
Dictionary<string, EntityProperty> flattenedProperties = EntityPropertyConverter.Flatten(complexObject);
// Create a DynamicTableEntity, set its Partition Key and Row Key and assign the flattenedProperties of the complex object to this DynamicTableEntity.
DynamicTableEntity dynamicTableEntity = new DynamicTableEntity(partitionKey, rowKey);
dynamicTableEntity.Properties = flattenedProperties;
// Write the DynamicTableEntity to Azure Table Storage using client SDK.
//Read the entity back from AzureTableStorage as DynamicTableEntity using the same PK and RK.
DynamicTableEntity entity = [Read from Azure using the PK and RK];
//Convert the DynamicTableEntity back to original complex object.
Imagine original complexObject was of type Order.
Order order = EntityPropertyConverter.ConvertBack(entity.Properties);
That’s all:)
doguarslan 