Persistent variables for method scope only - Appendix

Thanks to Martin, who took some time to view my blog entries, he suggested me to use ConditionalWeakTable<TKey, TValue> to get rid of the object disposing notification and to shorten the Dictionary-cascade to store values with the usage of parenthesis for dictionary key.

One more word to ConditionalWeakTable<TKey, TValue>: This class from compiler services was unknown to me, until its use for the mentioned improvement for the persistent variables for method scope only stuff. However, explained in short words, when the garbage collector checks for object removal, he ignores the key reference in ConditionalWeakTable instances, and with the removal of the specific object, the entry in in the weak table loses references as well and will be collected by garbage collector as well  sooner or later (except your code has any other references to TValue, of course). This provides a nice way to mount values on other objects while having an insurance, that these values will die together with the target object ...

Okay, here's the Persistent class again, with the Stack Frame Offset approach:

public static class Persistent<T> where T : struct
{
    #region StaticValues
    static readonly Dictionary<int, T[]> staticValues;
 
    public static ref T Static(T value)
    {
        var stackFrameOffset = new StackFrame(1, false).GetNativeOffset();
 
        if (!staticValues.ContainsKey(stackFrameOffset))
            staticValues.Add(stackFrameOffset, new T[] { value });
 
        return ref staticValues[stackFrameOffset][0];
 
    }
    #endregion
 
    #region NonStaticValues
    static readonly ConditionalWeakTable<object, Dictionary<int, T[]>> 
        nonStaticValues;
 
    public static ref T Local(T value, object callerInstance)
    {
        var stackFrameOffset = new StackFrame(1, false).GetNativeOffset();
 
        if (!nonStaticValues.TryGetValue(callerInstance, out var storage))
        {
            storage = new Dictionary<int, T[]>
            {
                { stackFrameOffset, new T[] {value} }
            };
            nonStaticValues.Add(callerInstance, storage);
        }
        else if (!storage.ContainsKey(stackFrameOffset))
        {
            storage.Add(stackFrameOffset, new T[] { value });
        }
        return ref storage[stackFrameOffset][0];
 
    }
    #endregion
 
    static Persistent()
    {
        staticValues = new Dictionary<int, T[]>();
        nonStaticValues = new ConditionalWeakTable<object, 
            Dictionary<int, T[]>>();
    }
}

And here with the default compiler arguments approach:

public static class Persistent<T> where T : struct
{
    #region StaticValues
    static readonly Dictionary<(string callerFilePath, string callerMemberName, 
        int callerLineNumber), T[]> staticValues;
 
    public static ref T Static(T value, 
        [CallerFilePath] string callerFilePath = default,
            [CallerMemberName] string callerMemberName = default,
                [CallerLineNumber] int callerLineNumber = default)
    {
        var complierKeys = (callerFilePath, callerMemberName, callerLineNumber);
 
        if (!staticValues.ContainsKey(complierKeys))
            staticValues.Add(complierKeys, new T[] { value });
 
        return ref staticValues[complierKeys][0];
 
    }
    #endregion
 
    #region NonStaticValues
    static readonly ConditionalWeakTable<object, 
        Dictionary<(string callerFilePath, string callerMemberName, 
            int callerLineNumber),
                T[]>> nonStaticValues;
 
    public static ref T Local(T value, object callerInstance,
        [CallerFilePath] string callerFilePath = default,
            [CallerMemberName] string callerMemberName = default,
                [CallerLineNumber] int callerLineNumber = default)
    {
        var complierKeys = (callerFilePath, callerMemberName, callerLineNumber);
 
        if (!nonStaticValues.TryGetValue(callerInstance, out var storage))
        {
            storage = new Dictionary<(string callerFilePath, 
                string callerMemberName, int callerFileNumber), T[]>
            {
                { complierKeys, new T[] {value} }
            };
            nonStaticValues.Add(callerInstance, storage);
        }
        else if (!storage.ContainsKey(complierKeys))
        {
            storage.Add(complierKeys, new T[] { value });
        }
        return ref storage[complierKeys][0];
    }
    #endregion
 
    static Persistent()
    {
        staticValues = new Dictionary<(string callerFilePath, 
            string callerMemberName, int callerLineNumber), T[]>();
        nonStaticValues = new ConditionalWeakTable<object, 
            Dictionary<(string callerFilePath, string callerMemberName, 
                int callerLineNumber), T[]>>();
    }
}

The usage hasn't changed except you don't need to implement any interface anymore when using non-static persistent values in your methods.

And still, bricolage ;)

Persistent variables for method scope only (V) - Conlusion

After elaborating all the stuff shown in previous articles, we can state, that it works somehow, even quite secure with the use of StackFrame, but so slowly that you cannot use it in production code. Furthermore the need of implement INotifyDispose for the use of non-static persistent values blows up code and impure implementation or missing object disposal leads to memory leakage. And the correct use of the ref syntax in consumption code is an additional obstacle to take care of.

For my personal claims too many drawbacks, so I decided for "Hands off!". Let's use such a festure, if it's baked in native C# and until then, use member or static class fields ...

But it was quite fun to noodle around the topic and I've learned a little bit more about the ref syntax I didn't use yet so much.

One more disclaimer: If you ever gone use the posted code from the previous articles, please be aware that this code is not yet thread safe! You can make it thread safe with corresponding locking techniques. Please keep this in mind!

And if you ever have some idea how to simplify or improve the posted solution attempt, please let me know in your comment. I'm eager to hear it :)

Persistent variables for method scope only (IV) - Speed issue

As you can remember, we have a complete solution for persistent variables, but it is painfully slow. And that's because we use StackFrame class which is not designed to be used in normal code, so its performance is rather low. On the other hand, StackFrame would provide more information than just the Native Offset we're reading from, and to elaborate this other information, it costs time which we could spare. So we could use our own implementation of a light-weighted StackFrame class. Its code is visible with every cheap IL de-compiler. Yes, you could, but the content is complex and if something changes on the IL framework, the StackFrame class will be changed accordingly, but will your implementation changed accordingly as well? Probably not. Rather your application will compile but chaos rising at runtime ...

So let's see what other possibility we have. We could use these pre-compiler driven attributes called CallerFilePath, CallerMemberName, CallerLineNumber. Using these three attributes, our code will work as fast as any simple code (in comparison, the use of StackFrame costs about 20 times more time to run!).

So when we change our Persistent class to that technique it would look like this:

public static class Persistent<T> where T : struct
{
    #region StaticValues
 
    static readonly Dictionary<string, Dictionary<string, Dictionary<int, T[]>>> staticValues;
 
    public static ref T Static(T value, [CallerFilePath] string callerFilePath = default,
        [CallerMemberName] string callerMemberName = default,
            [CallerLineNumber] int callerLineNumber = default)
    {
        if (!staticValues.ContainsKey(callerFilePath))
            staticValues.Add(callerFilePath, new Dictionary<string, Dictionary<int, T[]>>());
        if (!staticValues[callerFilePath].ContainsKey(callerMemberName))
            staticValues[callerFilePath].Add(callerMemberName, new Dictionary<int, T[]>());
        if (!staticValues[callerFilePath][callerMemberName].ContainsKey(callerLineNumber))
            staticValues[callerFilePath][callerMemberName].Add(callerLineNumber, new T[] 
            { value });

 
        return ref staticValues[callerFilePath][callerMemberName][callerLineNumber][0];
 
    }
    #endregion
 
    #region NonStaticValues
    static readonly Dictionary<string, Dictionary<string, 
        Dictionary<int, Dictionary<INotifyDisposed, T[]>>>> nonStaticValues;
 
    public static ref T Local(T value, INotifyDisposed callerInstance, 
        [CallerFilePath] string callerFilePath = default,
            [CallerMemberName] string callerMemberName = default,
                [CallerLineNumber] int callerLineNumber = default)
    {
        if (!nonStaticValues.ContainsKey(callerFilePath))
            nonStaticValues.Add(callerFilePath, new Dictionary<string, 
                Dictionary<int, Dictionary<INotifyDisposed, T[]>>>());
        if (!nonStaticValues[callerFilePath].ContainsKey(callerMemberName))
            nonStaticValues[callerFilePath].Add(callerMemberName, 
                new Dictionary<int, Dictionary<INotifyDisposed, T[]>>());
        if (!nonStaticValues[callerFilePath][callerMemberName].ContainsKey(callerLineNumber))
            nonStaticValues[callerFilePath][callerMemberName].Add(callerLineNumber, 
                new Dictionary<INotifyDisposed, T[]>());
        if (!nonStaticValues[callerFilePath][callerMemberName][callerLineNumber]
            .ContainsKey(callerInstance))
        {
            callerInstance.Disposed += (sender, e) =>
            {
                if (nonStaticValues[callerFilePath][callerMemberName][callerLineNumber]
                    .Remove(callerInstance))
                {
                    if (nonStaticValues[callerFilePath][callerMemberName][callerLineNumber]
                        .Keys.Count == 0)
                    {
                        nonStaticValues[callerFilePath][callerMemberName]
                            .Remove(callerLineNumber);
 
                        if (nonStaticValues[callerFilePath][callerMemberName].Keys.Count == 0)
                        {
                            nonStaticValues[callerFilePath].Remove(callerMemberName);
                            if (nonStaticValues[callerFilePath].Remove(callerMemberName))
                            {
                                if (nonStaticValues[callerFilePath].Keys.Count == 0)
                                {
                                    nonStaticValues.Remove(callerFilePath);
                                }
                            }
                        }
                    }
                }
                return;
            };
            nonStaticValues[callerFilePath][callerMemberName][callerLineNumber]
                .Add(callerInstance, new T[] { value });
        }
        
        return ref nonStaticValues[callerFilePath][callerMemberName][callerLineNumber]
            [callerInstance][0];
    }
    #endregion
 
 
    static Persistent()
    {
        staticValues = new Dictionary<string, Dictionary<string, Dictionary<int, T[]>>>();
        nonStaticValues = new Dictionary<string, Dictionary<string, Dictionary<int, 
            Dictionary<INotifyDisposed, T[]>>>>();
    }
}

First, we see, that the code is quite grown, but this is the smallest drawback of our change. But let's first check if the consumption has changed in any way:

class Test: INotifyDisposed
{
    public void Method1()
    {
        ref int myInt = ref Persistent<int>.Local(0, this);
        myInt++;
        Console.WriteLine($"myInt for this object instance is now {myInt}");
 
        ref int myIntStatic = ref Persistent<int>.Static(0);
        myIntStatic++;
        Console.WriteLine($"myIntStatic object independent is now {myIntStatic}");
 
    }
 
    #region INotifyDisposed & IDisposable implementation
 
    public event EventHandler Disposed;
 
    private bool disposedValue;
 
    protected virtual void Dispose(bool disposing)
    {
        if (!disposedValue)
        {
            if (disposing)
            {
                Disposed?.Invoke(this, new EventArgs());
            }
            disposedValue = true;
        }
    }
    public void Dispose()
    {
        Dispose(disposing: true);
        GC.SuppressFinalize(this);
    }
 
    #endregion
}

No, the code for the consumer is still the same. And all works fine and super fast with our sample Test class. So, where's the problem then?

The answers are called 'Optional parameters' and 'Uniqueness'.

You have sincerely noticed the optional parameters in the methods Local() and Static() on Persistent class. A developer could try to assign values to callerFilePath, callerMemberName and callerLineNumber arguments, either because he don't know how to use it exactly or to thickle out some values which are not supposed to be read and/or modified by his/other scope. So these optional arguments can lead either to chaos in your application or even leaks sensitive data. You wanna agree, this solution begins to smell, right?

But there is even another very bad side effects when using the pre-compiler attributes to help building a unique identifier for memory's key. It is not unique. The callerFilePath value is built when compiling. So imagine your colleague is using our feature as well, he builds a class library and has a code file called tests.cs which will lead in built phase to c:\a\test.cs. And now you are gonna use this library but having also a test.cs which also leads to c:\a\test.cs in the built phase. Vóilà! Your and your collegue's code have now big potential to interfere each other's code when using persistent values in methods. You can state: "That's almost impossible!" Maybe, but one time is the first time and to figure out what's going wrong with your application in such a situation can be very annoying. And besides the callerFilePath, what about the callerLineNumber? Let's imagine you write such a code:

public void Method1()
{
    ref int myIntStatic = ref Persistent<int>.Static(0); ref int myIntStatic2 = ref Persistent<int>.Static(0);
    myIntStatic++;
    myIntStatic2++;
    Console.WriteLine($"hey! myIntSatic ({myIntStatic}) does affect myIntStatic2 {myIntStatic2} and vice versa, " +
        $"what's going on here?");
 
}

Yes, the line number is not enough to identify your persistent value. All use of Persistent.Static (or Local) on the same line with the same type will share the same memory. This does not just smells, now it definitely stinks!

Next and final chapter: The conclusion

Persistent variables for method scope only (III) - Non-Static values

As shown in the previous article, we have possibility to use static values for method scopes only. Now we want to achieve the same for non-static values. We could say for member values, but as member indicates to object's scope, we rather call it here non-static, or later in the code just local.

When we provide values for life time of an object, we need to ensure the provided values dies with the object. So the question is: How to get notified when an object dies? Actually this is impossible, but we can make use of the disposable pattern. When the consumer class can inform about its disposal, we can get rid of the values.

But as you can see, this is now a breaking change: We need to prepare the class where we want to use our feature, which stinks, because the feature is now class depending, not just method depending as before. However, I couldn't find another solution, so lets start with that approach.

First, we need a notification implementation for these consumer classes. A really easy extension of IDisposable will do the job:

public interface INotifyDisposed : IDisposable
{
    event EventHandler Disposed;
}

Then let's extend the Persistent class which will be able now, to keep object-life-time values from method scopes as well:

public static class Persistent<T> where T : struct
{
    #region StaticValues
 
    static readonly Dictionary<int, T[]> staticValues;
 
    public static ref T Static(T value)
    {
        var key = new StackFrame(1, false).GetNativeOffset();
 
        if (!staticValues.ContainsKey(key))
        {
            staticValues.Add(key, new T[] { value });
        }
 
        return ref staticValues[key][0];
    }
    #endregion
 
    #region NonStaticValues
    static readonly Dictionary<int, Dictionary<INotifyDisposed, T[]>> nonStaticValues;
 
    public static ref T Local(T value, INotifyDisposed owner)
    {
        var key = new StackFrame(1, false).GetNativeOffset();
 
        if (!nonStaticValues.ContainsKey(key))
        {
            nonStaticValues.Add(key, new Dictionary<INotifyDisposed, T[]>());
        }
        if (!nonStaticValues[key].ContainsKey(owner))
        {
            nonStaticValues[key].Add(owner,new T[] { value });
 
            owner.Disposed += (sender, e) =>
            {
                if (nonStaticValues[key].Remove(owner))
                {
                    if (nonStaticValues[key].Keys.Count == 0)
                    {
                        nonStaticValues.Remove(key);
                    }
                }
                return;
            };
        }
        return ref nonStaticValues[key][owner][0];
    }
    #endregion
 
 
    static Persistent()
    {
        staticValues = new Dictionary<int, T[]>();
        nonStaticValues = new Dictionary<int, Dictionary<INotifyDisposed, T[]>>();

    }
}

See, that the region NonStaticValues has been added. Basically it works the same as for static values, but you need to provide an owner, type of INotifyDisposed. When the Disposed event is called, the memorized values from it's reference will be destroyed, and with it also the references to the owner object itself. To get rid of the owner's reference is very important, otherwise the garbage collector will never release the consumer object, because we still have references to it.

Let's see how the consumption now looks like:

class Test: INotifyDisposed
{
    public void Method1()
    {
        ref int myInt = ref Persistent<int>.Local(0, this);
        myInt++;
        Console.WriteLine($"myInt is now {myInt}");
    }
 
    #region INotifyDisposed & IDisposable implementation
 
    public event EventHandler Disposed;
 
    private bool disposedValue;
 
    protected virtual void Dispose(bool disposing)
    {
        if (!disposedValue)
        {
            if (disposing)
            {
                Disposed?.Invoke(this, new EventArgs());
            }
            disposedValue = true;
        }
    }
    public void Dispose()
    {
        Dispose(disposing: true);
        GC.SuppressFinalize(this);
    }
 
    #endregion 

} 

The correct implementation of INotifyDisposed and the correct object disposal are elemental. Not doing so will lead to catastrophic memory leakage! Yes, I can read your mind: "Uh - another really ugly pit you need to be aware of!". But let me state a conclusion in further chapter. Let's finish here first.

There's one more thing: Even all works fine with correct implementation and disposing objects, the access of the persistent values is painfully slow. Next chapter will explain why and what we can do here.

Persistent variables for method scope only (II) - Static values

As mentioned in the previous article, we have to think about how we can address the memory and how we can achieve a good user experience for using these persistent variables.

When we start with static values we're talking about memory existing once each method and thread, application domain or process (depending on which framework you compile against and how you structure it).

So in the end we want to have something like this:

public void Method1()
{
    static int myInt;
    myInt++;
    Console.WriteLine($"myInt is now {myInt}");
}

Problem 1: The addressing

As a first approach we could easily solve this with a static class providing memory for every static variable, when we just give a identifier. It could look like this:

public void Method1()
{
    int myInt = Persistent.Static("Method1");
    myInt++;
    Console.WriteLine($"myInt is now {myInt}");
}

Persistent would represent a static class maintaining the values of the static scope values and the Static method would provide us the value. But you would have to provide a key (here the same as the method's  name "Method1") which is quite ugly because of two reasons:

• You cannot trust the uniqueness of your key. Maybe your colleague has used the feature elsewhere too and used also the "Method1" key. You can imagine what will happen: Loss of memory isolation and furthermore your application will behave just crazy.
  
• On the other hand you want to access your values quick and elegant, to provide a key is not comfy at all.

We are able to solve this by auto generate a key given by the source which requests the value. An accurate method to do this is to use the native offset from the stack frame. Like this:

public static class Persistent<T> where T : struct
{
    static readonly Dictionary<int, T> staticValues;
    static Persistent()
    {
        staticValues = new Dictionary<int, T>();
    }
 
    public static T Static(T value)
    {
        var key = new StackFrame(1, false).GetNativeOffset();
 
        if (!staticValues.ContainsKey(key))
        {
            staticValues.Add(key, value);
        }
 
        return staticValues[key];
    }
}

And we can consume it now like this:

public void Method1()
{
    int myInt = Persistent<int>.Static(0);
    myInt++;
    Console.WriteLine($"myInt is now {myInt}"); 

} 

You surely have noticed that we have now a generic type parameter on the Persistent class as well as a argument to the Static method.

The generic type parameter is used to ensure type safety and the value argument on Static is used as initial value. Imagine that your variable would start with any value instead of its default value. And for reference types like objects we need a non-null value anyway, because we cannot predict how to construct such a value you want to use as local static value.

Ok, fine job. But if you test it, you will see, that myInt is always 1. Why is this? This is because the myInt is just a copy of the stored value. Any modification on it will not be saved back to the dictionary in Persistent class.

Problem 2: Accessing the value in comfy way

To enforce write-back mechanism when changing myInt, we could use a class with a typed property instead of providing a typed value directly:

public static class Persistent<T> where T : struct
{
    public class Memory
    {
        public T Value { get; set; }
    }
 
    static readonly Dictionary<int, Memory> staticValues;
    static Persistent()
    {
        staticValues = new Dictionary<int, Memory>();
    }
 
    public static Memory Static(T value)
    {
        var key = new StackFrame(1, false).GetNativeOffset();
 
        if (!staticValues.ContainsKey(key))
        {
            staticValues.Add(key, new Memory() { Value = value });
        }
 
        return staticValues[key];
    }
}

Consuming now like this:

 

public void Method1()
{
    var myInt = Persistent<int>.Static(0);
    myInt.Value++;
    Console.WriteLine($"myInt is now {myInt.Value}");
}

Ok, that works now, the increment is remembered. But you would agree, this looks ugly when using a class object as a proxy to access the stored value, right?

There's another approach: instead to use a class object we're gone use the ref feature to ensure write-back functionality:

public static class Persistent<T> where T : struct
{
    static readonly Dictionary<int, T[]> staticValues;
    static Persistent()
    {
        staticValues = new Dictionary<int, T[]>();
    }
 
    public static ref T Static(T value)
    {
        var key = new StackFrame(1, false).GetNativeOffset();
 
        if (!staticValues.ContainsKey(key))
        {
            staticValues.Add(key, new T[] { value });
        }
 
        return ref staticValues[key][0];
    }
}

See that the Static method has now upgraded with ref keywords. In order to be able to return values by reference we use now an array declaration of the target type instead of the dictionary's value. Why this? Because a dictionary's value property does not gives us the reference to it's memory, but some other, temporary memory area, in other words: another copy. The array object can give us reference of it's stored memory. As we only store one value each dictionary entry, we just populate and maintain the first entry (key 0) from that array. Looks a little bit confusing, right? I agree, but it works quite well.

But let's see how the consumption now looks like:

public void Method1()
{
    ref int myInt = ref Persistent<int>.Static(0);
    myInt++;
    Console.WriteLine($"myInt is now {myInt}");
}

Take notice, that we have here twice the ref keyword as well. This is needed since we really want a reference to the stored values, not just a copy. Just adding one ref is not enough: either the compiler will not swallow it or the result is not what you wanted. But on the other hand you can again operate with just a variable myInt and any modification on it will be stored persistent.

Cool, we have now to ability to keep static values which are only accessible in Method1. Let's see in next chapter how we can extend the functionality to achieve this also for non-static values, meaning, keep values per object instance.

Persistent variables for method scope only (I) - Introduction

Hello and welcome back for another series of articles here. This time a real C# relevant topic, not just .NET in general. So let's start ....

Have you ever wished to keep information only in a method and can re-access to it when the method is called again?

I barely remembered something, that Visual Basic provided back in the days: The static variable declaration in a method:

Sub Method1()
 
    Static myInt As Integer
 
    myInt = myInt + 1
 
    MsgBox("myInt is now" & myInt)
 
End Sub

The myInt is incremented every time Method1 is called. This feature is pretty cool if you want to encapsulate some information which needs only be accessed by exactly one method. However this feature also exists for VB.NET but not for C#. 

The main reason people often tell you is:

• C# is an object oriented language and therefore methods should not have state, only objects should have.

On the other hand, some people say, the feature would be hard to implement. Well, I doubt that a little bit, but I'm not expert in here ...

And most people stated, that you can use a variable on class level (member or static), and then just use it in one method. Of course I could do this, but in a world where new code becomes legacy codes in just some weeks, I don't like to ponder whether I should use the variable also in other methods or not, when modifying code afterwards - especially if the original code was not by me.

So I combed the internet and found some interesting articles on StackOverflow. But the materials there did not satisfy me, so I took it and tried to refine and tickle a nice workaround to achieve a home-brew feature - that was the plan. Thanks to all StackOberflow contributers on the mentioned article!

As a first attempt I tried to solve the static persistent variables for method scopes only. Later on, we will try to solve the persistent variables in methods scopes only for non-static values as well.

Read the next article when we have to think about the following Questions:

• What serves as identifiers to keep theses persistent values, while the method is not running?
• How can we access these values through normal variables, so it feels comfy?