Bootstrapper package for Windows Imaging Component

Developers who target .NET 4 and choose to create a proper bootstrapper for their installer have been bit repeatedly by installation failures on Windows XP SP2 and Windows Server 2003. .NET 4 fails to install on these admittedly outdated, but still widespread OSes because it depends on the Windows Imaging Component, which appears in XP SP3, Vista, 7 and 2008 Server and is installed with .NET 3.5 SP1. These OSes increasingly being the majority, Microsoft decided to leave WIC out of the .NET 4 installer in order to reduce the download size, a laudable intention. Since they actually documented this, there is no cause to complain; but why not also provide a bootstrapper package for WIC in the Windows SDK since it began to include packages for .NET 4? Developers usually direct the bootstrapper to download Microsoft files from the home site, so setup size would not increase appreciably.
Anyway. I created a WIC bootstrapper package and tested it on XP SP2, XP SP3 and 7. I also added a dependence on WIC to the .NET 4 packages. No warranty; use at own risk but comments welcome!

How to use SSL3 instead of TLS in a particular HttpWebRequest

My application has to talk to different hosts over https, and the default setting of ServicePointManager.SecurityProtocol = TLS served me well. The other day, though, I had some NetWare hosts which (as System.Net trace log shows) don't answer the initial TLS handshake message but keep the underlying connection open until it times out, throwing a timeout exception. It seems that Netware's policy regarding unrecognized/invalid requests is not to respond or give any error messages, presumably to reduce attack surface. Very understandable, but this behaviour does not give .NET's built-in TLS-to-SSL3 fallback mechanism a chance to kick in.
I really didn't want to have to degrade the security protocol setting to SSL3 in the whole application for the sake of a few musty Netware hosts, but this ServicePointManager setting is global and there is no way to force a downgrade through HttpWebRequest. Luckily, 'global' has more than one meaning in the .NET world; ServicePointManager settings are actually per-appdomain. This enabled me to work around the problem by creating a separate appdomain set up to use only SSL3, making my data collection object MarshalByRefObject (WebClient and WebRequest are marshal-by-ref too, but better to reduce the number of cross-appdomain calls and avoid marshaling anything more complicated than a string) and creating it there. Worked perfectly combined with a timeout-based detection scheme.

Having your InternalPreserveStackTrace and eating it

This post stems from a discussion of stack trace problems at the CLR team blog.

The problem

When an existing exception is thrown in the normal way with throw e, any stack trace that was recorded in it is overwritten and destroyed. This complicates debugging and logging — the stack trace seen by a top-level handler (which, in a long-running application, must log it and somehow restore the application to operation) is practically useless. Throwing existing exceptions — ones which were previously caught and stored or serialized — is a necessity when doing custom cross-thread invoke, e.g. a custom thread pool. Custom remote call solutions also suffer from this problem.

The known hacksolution

Microsoft's Remoting team encountered the same problem, but they had the advantage of being able to modify the CLR. They introduced the internal Exception._remoteStackTraceString field, which is not overwritten by CLR when an exception is thrown. Exception.StackTrace prepends the contents of this field to the normal stack trace. They also introduced two internal methods on Exception, PrepForRemoting and InternalPreserveStackTrace, which squirrel away the existing stack trace into this field. However, all these members are internal, so they cannot be reliably called by third-party code with similar needs.
It seems that Chris Taylor was the first to discover these internal members. He published a hack which preserves stack trace in an exception by accessing _remoteStackTraceString with Reflection. A more mature version of this hack by Fabrice Marguerie calls InternalPreserveStackTrace (again using Reflection). Later, Brad Wilson ranted on this subject. Brad also mentions that the Reflection team did not use stack trace preservation, but instead introduced the pesky TargetInvocationException (which most everyone has to unwrap and throw the inner exception ASAP to propagate the original exception).

Back to the present

When I mentioned this hack in the discussion at the CLR team blog, CLR team's Mike Magruder pointed out its essential brittleness/hackiness. Mike is, of course, right; I am sure no-one who uses this hack is happy about messing with mscorlib's internals; but the problem has to be dealt with. Mike's criticism prodded me into looking for a more portable solution.

It

My solution exploits the fact that cross-AppDomain calls need to preserve stack traces of exceptions propagating across the AppDomain boundary. Cross-AppDomain calls seem to use the serialization infrastructure to get non-trivial data across, so when Exception's SetObjectData constructor sees the CrossAppDomain flag in the supplied SerializationContext, it prepares the exception for subsequent throwing — by setting the crucial _remoteStackTraceString field in essentially the same way as InternalPreserveStackTrace, although SetObjectData forgets to insert a newline after the old stack trace. It remains, then, to call an exception's GetObjectData and SetObjectData, tricking it into believing that it is being serialized across the AppDomain boundary.
The primitive version of my solution relied on BinaryFormatter to do the heavy lifting:

static Exception WithPreservedStackTrace (Exception e)
{
    var context   = new StreamingContext (StreamingContextStates.CrossAppDomain) ;
    var formatter = new BinaryFormatter  (null, context) ;
    formatter.FilterLevel = TypeFilterLevel.Full ;

    using (var stream = new MemoryStream ())
    {
        formatter.Serialize (memory, e) ;
        memory.Position = 0 ; // rewind stream

        return (Exception) formatter.Deserialize (memory) ;
    }
}

This works like a charm, but all the unnecessary extra work done by BinaryFormatter galled me, so I poked around RedBits code some more and evolved the following version, which uses the arcane ObjectManager class:

static void PreserveStackTrace (Exception e)
{
    var context = new StreamingContext  (StreamingContextStates.CrossAppDomain) ;
    var manager = new ObjectManager     (null, context) ;
    var serinfo = new SerializationInfo (e.GetType (), new FormatterConverter ()) ;

    e.GetObjectData  (serinfo, context) ;
    manager.RegisterObject (e, 1, serinfo) ; // prepare for SetObjectData

    manager.DoFixups () ;                    // ObjectManager calls SetObjectData for us

    // voila, e is unmodified save for _remoteStackTraceString
}

This still wastes a lot of cycles compared to InternalPreserveStackTrace, but has the advantage of relying only on public functionality. Purists who really want to avoid calling InternalPreserveStackTrace can use this workaround :3

Update: usage samples which I posted on StackOverflow:

// usage (A): cross-thread invoke, messaging, custom task schedulers etc.
catch (Exception e)
{
    PreserveStackTrace (e) ;

    // store exception to be re-thrown later,
    // possibly in a different thread
    operationResult.Exception = e ;
}

// usage (B): after calling MethodInfo.Invoke() and the like
catch (TargetInvocationException tiex)
{
    PreserveStackTrace (tiex.InnerException) ;

    // unwrap TargetInvocationException, so that typed catch clauses 
    // in library/3rd-party code can work correctly;
    // new stack trace is appended to existing one
    throw tiex.InnerException ;
}

Dynamic method drop

Dynamic methods drove me crazy for the last two days. A completely innocent-looking, verified IL which created delegates from dynamic methods sometimes blew up with all kinds of exceptions: null-reference exceptions in weird places, stack overflows (or hang-ups if not running under Developer Studio) and the enigmatic FatalExecutionEngineError. Other times the code executed without a problem. It was difficult to establish any pattern in these failures. windbg+SoS revealed nothing beyond the fact that the IL was being generated correctly, and excavations of RedBits/Rotor code did not help much either. I went over the whole mess in my mind while listening to a jazz performance, and realized that the flaw was in my delegate-creation IL code, which went like this:

push target object
ldftn dynamic method
newobj Procedure..ctor(object, native int)
store new delegate

This code creates a working delegate, but the delegate's internal _methodBase field is initially null. The bald function pointer does not work as a GC reference, and if there are no other references to the dynamic method, it is liable to be garbage-collected, and the delegate containing the stale function pointer naturally but silently becomes a nest of nasal demons.

Delegate.CreateDelegate overloads don't fill _methodBase either. DynamicMethod.CreateDelegate does fill it explicitly, but it is impossible to get the DynamicMethod object from its token without calling mscorlib's internal methods. I understand the security reasons behind this decision, but it's damned uncomfortable.

The only solution to the problem I have found so far is to call the new delegate's get_Method() function to fill _methodBase and establish a GC-visible reference to the dynamic method. Whew!

Update: reported this issue to Microsoft; they decided to close it as a 'known limitation'. Well, it is known — now :3

Detroit Public School Repository

Superb photos, somewhat reminiscent of Prypiat schools and kindergartens.

WM_WINDOWPOSCHANGED.NET

A list view control is handy, but sometimes it needs a little black magic to work properly.

SetStyle (ControlStyles.OptimizedDoubleBuffer, true) ;

in the constructor takes care of the general flicker problem (in Win32, you need to request 6.0 common controls with a manifest and set the LVS_EX_DOUBLEBUFFER style). It seems that suppressing background painting in addition to double-buffer is overkill. Message sniffing and disassembly of comctl32.dll indicates that the background processing is different in 6.0 common controls with «double-buffer» vs. earlier versions. I quote «double-buffer», because I am not sure how much double-buffering is actually involved.

However, if you use the list view in report mode and want to have your columns adjust to the list view's width, you need more magic. The simple solution — setting column width(s) in OnResize — leads to nasty horizontal scrollbar glitching when you shrink the list view. Shuffling the column-resizing code around the various resizing handlers did no good whatsoever. A couple of hours of tedious debugging uncovered the proximate source of this glitch: after the list view gets resized and receives the WM_WINDOWPOSCHANGED message, it passes the message to the list view's original window procedure in comctl32.dll, which faithfully paints the offending horizontal scrollbar, because the columns are now too wide and the OnResize handler wasn't called yet. After this, another, nested WM_WINDOWPOSCHANGED occurs, and this one finally calls UpdateBounds, which calls the OnResize handler. And then the handler of the original WM_WINDOWPOSCHANGED calls UpdateBounds, and thus your handler, again! Why does it work this way? It is beyond me. Conclusion: it seems to be impossible to resize columns properly in response to any resize event.

The remedy is simple enough: run the column-resizing code before the list view is resized if the columns need to be narrower, and after it is resized if the columns need to be wider. The place to do this is the SetBoundsCore function, which is, luckily, virtual:

protected override void SetBoundsCore (int x, int y, int width, int height, BoundsSpecified specified)
{
  int newClientWidth = EstimateNewClientWidth (width, height) ;
  if (newClientWidth < ClientSize.Width)
  {
    // Reduce column width first to prevent horizontal scrollbar flicker.
    FixColumnWidths    (newcx) ;
    base.SetBoundsCore (x, y, width, height, specified) ;
  }
  else
  {
    base.SetBoundsCore (x, y, width, height, specified) ;
    FixColumnWidths    (ClientSize.Width) ; // the new client width!
  }
}

The remaining tricks are in the EstimateNewClientWidth function, which has to guess whether there was a scrollbar before the resize and whether there will be one after it:

private int EstimateNewClientWidth (int width, int height)
{
  if (Items.Count == 0) return ClientSize.Width ;

  // Estimate the new client size from the old one and the window size change.
  int oldcx = ClientSize.Width  ;
  int oldcy = ClientSize.Height ;
  int newcx = oldcx + width  - Bounds.Width  ;
  int newcy = oldcy + height - Bounds.Height ;
  int hdrcy = GetHeaderHeight () ;

  Rectangle rect0 = GetItemRect (0,               ItemBoundsPortion.Entire) ;
  Rectangle rectN = GetItemRect (Items.Count - 1, ItemBoundsPortion.Entire) ;          

  bool needScrollBar = rect0.Top < hdrcy || rectN.Bottom > newcy ;
  bool haveScrollBar = rect0.Top < hdrcy || rectN.Bottom > oldcy ;

  // Correct the new client size for vertical scrollbar changes.
  if (haveScrollBar)
  {
    if (!needScrollBar) newcx += SystemInformation.VerticalScrollBarWidth ;
  }
  else
  {
    if ( needScrollBar) newcx -= SystemInformation.VerticalScrollBarWidth ;
  }

  return newcx ;
}

I found no pure .NET method to get the current height of the list view's header control, so I have to resort to P/Invoke:

[StructLayout (LayoutKind.Sequential)]
struct RECT
{
  public int left   ;
  public int top    ;
  public int right  ;
  public int bottom ;
}

[DllImport ("user32.dll", EntryPoint = "GetWindowRect")]
static extern int GetWindowRect (IntPtr hWnd, ref RECT rect) ;

[DllImport ("user32.dll")]
static extern IntPtr SendMessageW (IntPtr hWnd, int message, IntPtr wParam, IntPtr lParam) ;

private int GetHeaderHeight ()
{
  RECT rect = new RECT () ;
  GetWindowRect        (SendMessageW (Handle, 0x101F /*LVM_GETHEADER*/, (IntPtr)0, (IntPtr)0), ref rect) ;

  return PointToClient (new System.Drawing.Point (rect.left, rect.bottom)).Y ;
}

And that's it for the .NET case ^.^

In Win32, there is a more elegant solution for EstimateNewClientWidth, which does not work under .NET: use SetWindowPos with SWP_NOREDRAW to do a fake resize on the list view, measure its new client width, and fake-resize it back. And instead of the SetBoundsCore, the column-resizing code goes into the parent window's WM_WINDOWPOSCHANGED handler, which is where you resize the list view anyway.

"ee"

In 1947, Orwell wrote in «The English People» that the English language gravitates towards simpler grammar and syntax. If this trend continues, he wrote, English would have more in common with isolating languages of East Asia than with Indo-European languages. I will write about one of the steps in this direction, and have some fun in the process.

The other day I was reading something somewhere, and bam! the word attackee (800) leapt out of the page (NB: where available, I give Google hit counts, excluding obvious misspellings, in parenthesized small numerals). I recalled that in a contract I had recently translated there was a payee (1,3M), but it did not strike me then as anything unusual. After all, programmers commonly use callee (61K) and sometimes pointee (36K), personifying functions and objects. But attackee somehow seems a peculiarly ugly word, its sole raison'd'etre being that «X being attacked» and «X under attack» are too long while «defender» has somewhat different meaning. attackee tells us that the -ee suffix, having originally appeared in the French loanword employee (138M), has firmly established itself as an independent lexical unit. If attackee, I thought, why not more? Let's have more fun with -ee! Me and my sister almost laughed our backsides off today thinking up all sorts of -ee words and Googling them.

Fun I



Picrelated: the huntee is not visible at all.

Violent transitive verbs seem to eeify particularly well. Some examples: programmers without respect for language sometimes use trapee; sandbaggee (3) is at least funny; kickee, rapee (the Urban Dictionary has it) and killee are too contaminated with accidental foreign matches to give accurate hit counts; murderee (6K) featured in the Independent (I'm not kidding you!); then there's fuckee (21K) with its gross and sometimes misspelt retinue; and the king of them all, pwnee (3K) — I'll be using this one! «pwnee detected» might even have a chance on the various chans.

Fun II

Besides the venerable employee, economists have payee (1,3M) and mortgagee (1M) and such gems as buyee (10K) and bankruptee (1K), while lawyers hold their own with the likes of contractee (73K), insuree (8K), harassee (4K) and slanderee (70). creditee is too confused with a declension of the French crediter to ascertain its usage; other -ee words suffer from French grammar as well. In the political sphere there is electee (10K), and jokers have invented votee. Generally speaking, just about any economic or legal term with the -er suffix can sprout an -ee sibling and, sooner or later, usually does, for the greater benefit of terminological uniformity.

Fun III

On a more (or rather less) forgiving note, some terminally deaf bonehead thought up the abominable words forgivee (2K) and even lovee — they say 1913 Webster had it! I can't imagine using these words for anyone close or important to me. Their natural habitat is probably restricted to badly written self-help books and similar trash.

Fun IV

Finally, for some random fun try postee, bloggee, googlee, trollee, quittee and decidee (hello George), but don't try firee or dumpee (5K)! Remember, I warned you!

Conclusion

eeified transitive verbs surely add to the vibrancy of the English language, and may sometimes help with terminology; but please do remember that not all of them sound and feel equally good.