I’ve written a multi-threaded windows service in C#. For some reason, csc.exe is being launched each time a thread is spawned. I doubt it’s related to threading per se, but the fact that it is occurring on a per-thread basis, and that these threads are short-lived, makes the problem very visible: lots of csc.exe processes constantly starting and stopping.
Performance is still pretty good, but I expect it would improve if I could eliminate this. However, what concerns me even more is that McAfee is attempting to scan the csc.exe instances and eventually kills the service, apparently when one the instances exits in mid-scan. I need to deploy this service commercially, so changing McAfee settings is not a solution.
I assume that something in my code is triggering dynamic compilation, but I’m not sure what. Anyone else encounter this problem? Any ideas for resolving it?
Update 1:
After further research based on the suggestion and links from @sixlettervariables, the problem appears to stem from the implementation of XML serialization, as indicated in Microsoft’s documentation on XmlSerializer:
To increase performance, the XML serialization infrastructure dynamically generates assemblies to serialize and deserialize specified types.
Microsoft notes an optimization further on in the same doc:
The infrastructure finds and reuses those assemblies. This behavior occurs only when using the following constructors:
XmlSerializer.XmlSerializer(Type)
XmlSerializer.XmlSerializer(Type, String)
which appears to indicate that the codegen and compilation would occur only once, at first use, as long as one of the two specified constructors are used. However, I don’t benefit from this optimization because I am using another form of the constructor, specifically:
public XmlSerializer(Type type, Type[] extraTypes)
Reading a bit further, it turns out that this also happens to be a likely explanation for a memory leak that I have been observing when my code executes. Again, from the same doc:
If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, which results in a memory leak and poor performance. The easiest solution is to use one of the previously mentioned two constructors. Otherwise, you must cache the assemblies in a Hashtable.
The two workarounds that Microsoft suggests above are a last resort for me. Going to another form of the constructor is not preferred (I am using the "extratypes" form for serialization of derived classes, which is a supported use per Microsoft’s docs), and I’m not sure I like the idea of managing a cache of assemblies for use across multiple threads.
So, I have sgen‘d, and see the resulting assembly of serializers for my types produced as expected, but when my code executes the sgen-produced assembly is not loaded (per observation in the fusion log viewer and process monitor). I’m currently exploring why this is the case.
Update 2:
The sgen’d assembly loads fine when I use one of the two "friendlier" XmlSerializer constructors (see Update 1, above). When I use XmlSerializer(Type), for example, the sgen’d assembly loads and no run-time codegen/compilation is performed. However, when I use XmlSerializer(Type, Type[]), the assembly does not load. Can’t find any reasonable explanation for this.
So I’m reverting to using one of the supported constructors and sgen’ing. This combination eliminates my original problem (the launching of csc.exe), plus one other related problem (the XmlSerializer-induced memory leak mentioned in Update 1 above). It does mean, however, that I have to revert to a less optimal form of of serialization for derived types (the use of XmlInclude on the base type) until something changes in the framework to address this situation.
Psychic debugging:
If this is the case you can build these XML Serializer Assemblies a-priori.