Long Lost Eric

What is an environment agnostic test?
This is a test which will execute successfully no matter how the environment is configured assuming that the product which you intend to test is available on the machine.

What are some possible environment variables?
Operating system, platform, OS language, product language, product sku, product settings, additional applications, etc.

As seen here there are many variables and one thing that my team expects is that our tests will execute the exact same way under all circumstances providing that the test is executing in a supported scenario. This leads to a rather large automation matrix that must be tested against before we ship (this will be another post), but it also leads to a very well defined and efficient test infrastructure.

The above expectation sparked a discussion because we were executing a joint run that included both VB IDE and C# IDE tests, which resulted in some of the C# tests failing. They were failing because in order to execute a joint run with both VB and C# tests, all tests must execute under the VB profile. All tests must execute under this profile, because the VB tests are not environment agnostic (yet!) and all of the C# tests are (well should be).

We will be discussing this topic more in our team meeting on Monday in order to assure we are all on the same page and I will make it one of my missions over the next few months to ensure my teams tests will execute flawlessly under any supported circumstances.

My questions for you are what kind of tests do you execute? What expectations do you have for your environment? Have you had similar discussions at your company?

The Live Mesh team has announced their technology preview. I've been using this service for well over a month now and it's pretty slick. Today is basically has parity with another entity called Folder Share, but let me promise you that much more will be coming soon.

A question that I was recently asked on a trip was what source control system we use and what it looks like. I've spent quite a bit of time in this space for our team and will try to answer the question with this post.

What source control provider does your team use?
During the Visual Studio 2008 product cycle the primary tool for source control was an internal tool called Source Depot. While not all teams used Source Depot, yes some used TFS, it was the primary source control provider and it was what most of the internal tools were built against and we didn't have time to rebuild them to completely target TFS. Any important branch, Main, Beta1, Beta2, RTM  was in Source Depot. The move had started though... we are beginning to dogfood TFS and tools where getting ported.

Once 2008 was shipped the entire division focused on moving to TFS. This is now the only supported source control provider moving forward for new and future work. We will continue to support Source Depot for a while as our 2003, 2005 and 2008 source code lives there and we must continue to support those products.

What does your source control structure look like?
Generally our source code tree is about 3 levels deep. The top level root branch is referred to as Main, child branches of Main are referred to as Product Unit (PU) branches and child branches of PU branches are referred to as feature branches. Some teams have one PU branch and one feature branch. These teams generally just use the PU branch as a staging area when taking integrations up or down. Other teams will have multiple feature branches that will regularly integrate up to the PU when their work is complete. It's often interesting for individuals to see that each PU within the division functions as their own entity handling check-ins, integrations, builds and the number of branches differently. While most of the teams function in a similar fashion almost every team has some quirk in their process. My team as long as I have been a part of it follows the single feature branch and single PU branch model.

How many integration\merge conflicts do you normally have to deal with?
This really depends on how the branch structure is setup and how much your team needs to change shared code vs. team specific code. Generally I would say the number of conflicts my team sees on average is around 30 - 40. Though the other day when I was doing an integration of QA code only I had around 5,000 conflicts. Fortunately I didn't have to resolve each of these by hand, because the owner of the files just said they wanted to take what ever was in the destination branch and to ignore the incoming changes. This was a blessing.

Is your test code under source control and if so where do the sources live?
Yes all of our test code is under source control. It is treated very similar to the product code and is branched with the product code. While this was probably the answer that you expected the test code has not always lived with the product nor was it branched accordingly. This was a recent change in the last three years. I feel this was the correct change, though I still feel some in the division might not agree. When we added the test sources to the product source control I believe it almost double the size of the depot in size and tripled the number of files.

How long does it take to sync all of Main?
I don't have any data on this. I will work on getting some and make another post out of it in the future.

Yesterday I was bitten by a bug in our test infrastructure where the code was attempting to dispose of an object in the finalize method. It took me a bit longer then I had hoped to track down the issues, but in the end I did track it down and fixed it.

Here is some example code where the bug exists:

class Logger : IDisposable
{
    private StreamWriter Log { get; set; }

    public Logger(FileInfo file)
    {
        Log = File.CreateText(file.FullName);
    }

    public void Write(string msg)
    {
        Log.WriteLine(msg);
    }

    ~Logger()
    {
        Dispose();
    }

    public void Dispose()
    {
        if (Log != null)
        {
            Log.Dispose();
            Log = null;
        }
        GC.SuppressFinalize(this);
    }
}

You may be looking at this code and asking what is wrong even after you glance over the MSDN article, Overriding the Finalize Method. The first sentence of the article is, "A Finalize method acts as a safeguard to clean up resources in the event that your Dispose method is not called." At a quick glance one might think the following. A dispose method is designed to be called multiple times without adversely affecting the object and the framework has a notion of a Finalize method that will always be called, so why don't I just call the Dispose method from Finalize?

The reason that you never want to call a Dispose method from a Finalize method is that the Finalize method was implemented to ensure that all unmanaged resources are cleaned up since the GC cannot handle these. Since the GC is allowed to clean up/finalize managed resources in which ever order it wishes if you run the above code enough you will notice that the StreamWriter object is finalized before the Logger object. At this point when the Logger object is finalized it attempts to invoke it's Dispose method, which in turn attempts to invoke Dispose on StreamWriter which throws because the object has been cleaned up.

To fix this code I removed the Finalize method, since it should have never been implemented as we are never allocating any unmanaged resources.

Here are some other resources on the topic.

• C# Destructor Syntax
• Implementing a Dispose Method