Having your InternalPreserveStackTrace and eating it

This post stems from a discussion of stack trace problems at the CLR team blog.

The problem

When an existing exception is thrown in the normal way with throw e, any stack trace that was recorded in it is overwritten and destroyed. This complicates debugging and logging — the stack trace seen by a top-level handler (which, in a long-running application, must log it and somehow restore the application to operation) is practically useless. Throwing existing exceptions — ones which were previously caught and stored or serialized — is a necessity when doing custom cross-thread invoke, e.g. a custom thread pool. Custom remote call solutions also suffer from this problem.

The known hacksolution

Microsoft's Remoting team encountered the same problem, but they had the advantage of being able to modify the CLR. They introduced the internal Exception._remoteStackTraceString field, which is not overwritten by CLR when an exception is thrown. Exception.StackTrace prepends the contents of this field to the normal stack trace. They also introduced two internal methods on Exception, PrepForRemoting and InternalPreserveStackTrace, which squirrel away the existing stack trace into this field. However, all these members are internal, so they cannot be reliably called by third-party code with similar needs.
It seems that Chris Taylor was the first to discover these internal members. He published a hack which preserves stack trace in an exception by accessing _remoteStackTraceString with Reflection. A more mature version of this hack by Fabrice Marguerie calls InternalPreserveStackTrace (again using Reflection). Later, Brad Wilson ranted on this subject. Brad also mentions that the Reflection team did not use stack trace preservation, but instead introduced the pesky TargetInvocationException (which most everyone has to unwrap and throw the inner exception ASAP to propagate the original exception).

Back to the present

When I mentioned this hack in the discussion at the CLR team blog, CLR team's Mike Magruder pointed out its essential brittleness/hackiness. Mike is, of course, right; I am sure no-one who uses this hack is happy about messing with mscorlib's internals; but the problem has to be dealt with. Mike's criticism prodded me into looking for a more portable solution.

It

My solution exploits the fact that cross-AppDomain calls need to preserve stack traces of exceptions propagating across the AppDomain boundary. Cross-AppDomain calls seem to use the serialization infrastructure to get non-trivial data across, so when Exception's SetObjectData constructor sees the CrossAppDomain flag in the supplied SerializationContext, it prepares the exception for subsequent throwing — by setting the crucial _remoteStackTraceString field in essentially the same way as InternalPreserveStackTrace, although SetObjectData forgets to insert a newline after the old stack trace. It remains, then, to call an exception's GetObjectData and SetObjectData, tricking it into believing that it is being serialized across the AppDomain boundary.
The primitive version of my solution relied on BinaryFormatter to do the heavy lifting:

static Exception WithPreservedStackTrace (Exception e)
{
    var context   = new StreamingContext (StreamingContextStates.CrossAppDomain) ;
    var formatter = new BinaryFormatter  (null, context) ;
    formatter.FilterLevel = TypeFilterLevel.Full ;

    using (var stream = new MemoryStream ())
    {
        formatter.Serialize (memory, e) ;
        memory.Position = 0 ; // rewind stream

        return (Exception) formatter.Deserialize (memory) ;
    }
}

This works like a charm, but all the unnecessary extra work done by BinaryFormatter galled me, so I poked around RedBits code some more and evolved the following version, which uses the arcane ObjectManager class:

static void PreserveStackTrace (Exception e)
{
    var context = new StreamingContext  (StreamingContextStates.CrossAppDomain) ;
    var manager = new ObjectManager     (null, context) ;
    var serinfo = new SerializationInfo (e.GetType (), new FormatterConverter ()) ;

    e.GetObjectData  (serinfo, context) ;
    manager.RegisterObject (e, 1, serinfo) ; // prepare for SetObjectData

    manager.DoFixups () ;                    // ObjectManager calls SetObjectData for us

    // voila, e is unmodified save for _remoteStackTraceString
}

This still wastes a lot of cycles compared to InternalPreserveStackTrace, but has the advantage of relying only on public functionality. Purists who really want to avoid calling InternalPreserveStackTrace can use this workaround :3

Update: usage samples which I posted on StackOverflow:

// usage (A): cross-thread invoke, messaging, custom task schedulers etc.
catch (Exception e)
{
    PreserveStackTrace (e) ;

    // store exception to be re-thrown later,
    // possibly in a different thread
    operationResult.Exception = e ;
}

// usage (B): after calling MethodInfo.Invoke() and the like
catch (TargetInvocationException tiex)
{
    PreserveStackTrace (tiex.InnerException) ;

    // unwrap TargetInvocationException, so that typed catch clauses 
    // in library/3rd-party code can work correctly;
    // new stack trace is appended to existing one
    throw tiex.InnerException ;
}