Software. Efficiency. Scalability.

I posted this as a comment here:

http://10rem.net/blog/2010/11/16/windows-forms-developers-tell-me-about-your-applications

and figured I’ll re-post it in my blog.

I have a lot of experience designing WinForms apps. Here is a screenshot from my the latest app I designed: http://execqview.com/images/facility-age-gender.gif.

This is a very simple screen. There are quite a few of more complex screens with hundreds of controls and a lot of data binding/data entry.

I had an opportunity to move to WPF, however I ran into some problems that prevented me from doing so:

1) Lack of business controls. WinForms doesn’t have a lot of good controls built in, but there are mature 3rd party libraries like DevExpress. For instance, take one of the most common controls in business apps: data grid. The grid from my screenshot up above comes with bands, multi sorting, totals, column selection, grouping, reordering, filtering, incremental search, custom cell rendering, cell controls, export to excel, and many other features. It can hold 100,000 records without any consequences to performance whatsoever. This is just one example. Tree view, tree list, scheduling controls, layouts, menu controls, ribbon controls, tab controls, charts, etc. All these things are readily available for WinForms development and provide an amazing productivity boost. The market for WPF controls is a lot weaker. Even established vendors like DevExpress have rather weak libraries for WPF.

2) Incredibly complicated design system. In WinForms you have controls that you can anchor to other controls- a very simple concept. In WPF you have different layouts that are cumbersome and yet inferior to anchoring in terms of real work. On top of it, you have access to myriads of properties such as gradients, layouts, panes, paths, resources, templates, 3D, timelines, and many other things. It’s just crazy complicated and almost assumes that you have to have a designer on the team. Setting up even a simple application requires a lot more work with WPF which results in lower productivity.

3) As a developer, you can’t leverage almost anything from your WinForms experience. Literally, you have to throw what you know away, clear your mind, and embrace all kinds of new concepts such as MVVM, new data binding, new control structure and design, resources, etc. You have to learn new tools such as Blend. It takes a while to get productive using all these new concepts and tools. We are talking about a serious learning curve here, a curve that is not easily justified at the moment.

4) Most WPF applications don’t have a good feel to them. I am sorry, but it’s true. They are slow, clunky, and non-native looking. You CAN make them look slick and flashy, but it requires a lot of effort and a set of strong artistic skills. Unfortunately, most GUI applications are written by businesses for businesses. Teams don’t have access to designers and have very strict time lines. Not a lot of mainstream developers have a luxury of turning their apps into a piece of art.

Essentially, it takes a lot more effort and time to write WPF applications and unless you need a very flashy app there are no clear benefits that you gain. There is

If you need to troubleshoot a poorly performing ASP.NET application here is a couple tips for you.

GC Performance Counters

GC is the lungs of the application. It needs to breathe freely to perform well. The rhythm is as important as oxygen intake. First of all, look at the list of the available performance counters for .NET GC:

http://msdn.microsoft.com/en-us/library/x2tyfybc.aspx

The counters you want to watch for are: # Gen 1 Collections, # Gen 2 Collections, % Time in GC, # Bytes in all Heaps, and Large Object Heap size

The best way to monitor a .NET application memory performance is to run a performance counter with the maximum resolution (1 sec) over 2-5 minutes. It’s a bit like getting an EKG: you want a detailed diagram of a short period, not an approximation over a longer period.

Here is what you watch for:

1. Normal % Time in GC is around 10-12%. Over 30% is where it starts getting bad. Over 50% is really bad. That means 50% of the processing time is spent managing memory. By design, .NET memory manager will stop all active threads for garbage collection (GC), and no requests will be processed at the time of collection. All new requests will have to wait in the queue.
2. # Gen 2 Collections should be 10 times or so lower than # Gen 1 Collections. Gen1 collections are relatively cheap. Gen2 collections are expensive. If the ratio is much lower than 10, it can cause potential latency delays. A ration of 1-to-1 or 1-to-2 indicates severe issues with memory management.
3. Large Object Heap size. LOH is never compacted so it can get fragmented easily. It is also collected as a part of Gen2 collection, which is expensive.

W3SCV_W3WP counters

Another set of counters to look at are called W3SVC_W3WP counters. These performance counters allow you to monitor IIS worker processes for application pools. Worker process instance counters are named PID_APPPOOLNAME. You can get all instances using *_APPPOOLNAME for a specific app pool.

The counters to watch for are Active Threads Count, Total Threads Count, and Maximum Threads Count.

Each request requires a thread to execute. Thread Pool is a mechanism in .NET that manages worker threads. It can dynamically grow the number of threads up to the Maximum Threads Count. If GC is the lungs of your app, Thread Pool is the heart. If it stops beating your app will freeze.

Total Threads Count indicates the currently utilized number of threads. If this number gets close to Maximum Threads Count, it means that the pool has at some point expanded to its limit. It’s a thing to watch for, but it’s not very critical.

Active Threads Count indicates how many threads are currently busy. If this number is close or equal to Maximum Threads Count it means the Thread Pool is reaching its capacity. The lack of available threads in the Thread Pool can cause requests to be delayed or denied (you will likely to see 500 HTTP error with ‘too busy’ in details).

Long running requests can cause Thread Pool starvation. It’s a situation where all (or most) worker threads are busy waiting for a response from a web service or long running database query. The utilization of the server is very low (close to 0%) but no new requests can be processed. All new requests will either get denied or delayed until one of the worker threads gets available for processing. Imagine all lanes in a supermarket waiting for a manager at the same time. Everybody’s busy and no work gets done at the same time.

ASP.NET Counters

The last things I am going to mention are general ASP.NET counters.

Take a look at ASP.NET\Request Execution Time and ASP.NET\Request Wait Time counters. They show execution time and wait time, respectively, for the last request. While these numbers are arbitrary, you can quickly estimate the max throughput of your Thread Pool. For instance, if your execution time is 5 seconds and your Maximum Thread Count is 50 you will not be able to process more than 10 requests a second.

Separate application pools have separate thread pools. Collecting performance counter data during peak hours can help understand any potential performance bottlenecks in .NET applications.

Conclusion

By no means these are the only things you should look at, but it’s a good place to start.

If you want to understand how .NET GC works (and it’s a pretty sophisticated GC), Doug Strewart compiled a great index to Maoni Stephens’s GC blog:

http://blogs.msdn.com/b/dougste/archive/2010/02/18/an-index-to-maoni-s-blog-posts-about-the-gc.aspx

I run Windows 7 on my workstation with SSD + RAID1 setup. The SSD contains Windows, Visual Studio, and some other tools that I use. The mirrored HDDs contain all data files and a backup copy of the SSD. This setup has been working fairly well for me because it gives an outstanding performance of the SSD drive (Windows 7 boots up in under 8 sec) and a stability of RAID1 which prevents any significant data loss.

The only disappointing issue so far has been Windows mirror fail once in a while. It’s hard to say why, but it seems that sometimes the mirror just breaks and needs to be restored. It takes a LONG time. By that I mean it’s been running for over 30 hours now and it’s 48% complete. No data has been lost, but it’s still rather annoying when it happens. Last time I blamed a power failure but this time there was none. The mirror just broke while my workstation was running. I am not sure what’s the deal with it as the HDDs are new and checkdisk doesn’t seem to find any issues.

As promised,here is  a little example of how to use SyncQueue with TaskGroup when you need to queue multiple requests and wait until all of them come back. This is a fairly common situation that is not addressed by the standard BackgroundWorker control.

Here is a piece of working code from one of the projects I’ve been working on. It searches and fetches all health care claims that belong to a specific hospital visit and displays the visit once all the claims are fetched:

private void FetchVisitData() {
  // create new SyncTaskGroup in the constructor
  Global.TaskQueue.Append( new SyncTask( this, new SyncTaskGroup() ) {
    Action = task => {
      // find all claims that belong to the visit
      var claims = Global.Claims.Search( Data.Claims.ClaimSearchAction.VisitId,
                      VisitId, 500, Data.Claims.ClaimSearchKind.Equals );
      ListData list = new ListData();
      foreach( Data.Claims.ClaimSearchResult info in claims.Items ) {
        // when we queue a task, pass the group from the parent task as the first argument
        Global.TaskQueue.Append( task.Group, new SyncTask( this ) {
          Context = info,
          Action = task2 => {
            // fetch claim data from the server
            MapData map = GetClaimData( (Data.Claims.ClaimSearchResult)task2.Context );
            // add claim to the list, lock as this happens concurrently
            lock( list ) { list.Add( map ); }
          },
          Error = Global.ReportTaskError,
          // completion of the child task notifies the group
          Complete = task.Group.TaskComplete
        } );
      }
      task.Result = list;
      // in a concurrent environment (especially multi-CPU) the first group task can
      // complete BEFORE the 2nd task is queued, therefore triggering a premature
      // completion of the whole group. To address this a task group needs to be manually
      // released when all child tasks are queued
      task.Group.Release();
    },
    Error = Global.ReportTaskError,
    Complete = task => {
      // we will only get here when all the claims have been fetched and the group task is completed
      Visit = new MapData( "Claims", (ListData)task.Result );
      RenderVisit();
    }
  });
}

As promised, here is a little library that allows for background queuing of long running synchronous operations.  Unlike the standard BackgroundWorker component, it’s not a visual component. Additionally,  it allows a little bit more control over what’s going on.  Also, I personally prefer lambdas over events, because  it keeps all the code relevant to the background process in the same place.

Here is the code:
http://code.google.com/p/syncqueue/source/browse/syncqueue.cs

Here is how you can use it:

// create a new queue with 4 worker threads
TaskQueue queue = new TaskQueue( 4 );

// 'this' must inherited from System.Windows.Forms.Control
SyncTask task = new SyncTask( this )  {

    Action = task => {
       // task action, execute your background task here
       // executed on a worker thread
       // use task.Result to return values
    },

    Complete = task => {
      // task complete without errors, called on the main thread
    },

    Error = task => {
      // an error occurred during execution, called on the main thread
      // exception is stored in task.Exception
      // if an exception if thrown in this method it will be ignored
    },

    Finally = task => {
      // if exists, always called at the end on the main thread
    }

};

// en-queue the task for execution
queue.Append( task );

The library also allows for spawning sub-tasks using SyncTaskGroup class. I will illustrate how it can be used in the next post.

After an extended period of silence I am back. A lof has happened but I am back in business of blogging.

To compensate for the missing posts, I will publish a little but handy C# library for proper handling of the background operations.  The built-in BackgroundWorker component is rather weak. I just need to put proper comments and figure out a free license for its distribution.

Stay tuned!