Question

We have a legacy VB6 application that uses Crystal Reports XI to generate printed reports. We’ve found through experience that the Crystal Reports print engine crashes if it picks up the wrong version of usp10.dll (the Windows Uniscribe library).

One customer is consistently having printing issues on their Windows 7 machines (running Windows 7 Enterprise, 32-bit). However, we have a few other customers running various editions of Windows 7 that are not having any problems.

On one of the machines that was having printing issues, I noticed that there was an older version of usp10.dll (one incompatible with Crystal Reports XI) in the folder C:\Program Files\Common Files\Microsoft Shared\Office10\. I’m not sure what application installed these files, because the customer doesn’t have Office 2002 installed (so I assume another application installed them). However, I temporarily renamed the file and our application was able to print correctly, so it seems that our application was loading that version of the file originally, which was causing the crashes.

The crash only happens the moment the user tries to print a report. Our application has direct dependencies on craxdrt.dll (the Crystal Reports ActiveX Designer Runtime library) and crviewer.dll (the Crystal ActiveX Report Viewer library), and the crash happens whether we print directly through craxdrt.dll or through the Report Viewer control.

In the past, we have resolved this issue by copying a known good version of usp10.dll into our application’s directory and creating an .local file to enable DLL redirection. At the customer site, I tried this, and also tried the alternate approach of creating a .local folder for our EXE and placing the usp10.dll in there, but neither approach worked on the machine I was connected to.

I did notice that usp10.dll is a "known" DLL in Windows (it has an entry in HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\KnownDLLs), but I tested our application on another Windows 7 machine (running Professional Edition, 32-bit) here that also had the DLL listed as a known DLL in the registry, and by using Dependency Walker, I could see that the redirection was working on that computer. This is somewhat confusing, since the Microsoft documentation states that known DLL’s cannot be redirected. Also, as I implied in the question title, our main EXE does not use a manifest file (the Microsoft documentation states that the presence of a manifest, embedded or standalone, disables DLL redirection).

So, my question is, is there any other reason why DLL redirection would work on some machines and not others, and does this have anything to do with differences between Windows 7 and Windows XP? I had considered deleting everything in the KnownDLLs registry key, but since the redirection was working on a machine here that had the same set of KnownDLLs, I’m not sure that would actually resolve the issue, and I don’t want to delete that key if I don’t need to. I haven’t yet had a chance to connect to the customer’s machine again to run Dependency Walker, but I’m not sure I would be able to interpret its logs anyway (even on the machine where it was working, I saw a lot of LoadLibrary calls for usp10.dll pointing to a folder other than the redirected folder, but some calls were apparently redirected, so I’m not sure what that means either).

---

EDIT: I should have also mentioned that every computer we’ve checked also has another copy of usp10.dll in the System32 folder. Looking at Chris’s answer and this blog post by Larry Osterman explaining a bit more about how known DLLs work, I realized that that probably doesn’t factor into the problem at all, since our program isn’t loading the copy of usp10.dll that is in the System32 folder.

---

EDIT #2: After playing around with Dependency Walker some more on my VB6 development machine (Windows XP SP3), where the printing has always worked, I was able to glean some information. I profiled our application in Dependency Walker and set it to log full path names, and it looks like one of the Crystal Reports dependencies (another Crystal Reports DLL) tries to load usp10.dll from multiple (hard-coded) paths before giving up and just asking for it by filename. It turns out that it tries to load it from the Crystal Reports bin folder first, then tries to load it from from C:\Program Files\Common Files\Microsoft Shared\Office10\usp10.dll. If it can’t find it at either location, it just asks Windows for usp10.dll (which will grab the one in System32). But even this isn’t consistent. Sometimes it asks for the file in the Office10 folder, and then appears to ignore the fact that it couldn’t find the file, while other times there are a series of LoadLibrary calls where it looks like the Crystal Reports code is actively looking for alternate copies of the file in different locations. Even more confusing is that at least one of the Crystal Reports components looks like it actually has a load-time dependency on usp10.dll, so that component always seems to get the copy in System32.

I’m still not 100% clear why the .local redirection wouldn’t work in this situation on this customer’s computers, but I think that partly explains why this particular customer is having problems, since all of the computers with the problem have an Office10 folder with an apparently incompatible version of usp10.dll in it.

But, once again, I’m still left with the basic question: if these components are looking for this file in so many different places, how can I guarantee that they will all use the same copy?

Answer 1

The solution was actually pretty simple, but it took me a while to figure out why it worked.

On the customer machine, I copied usp10.dll from C:\Windows\System32 (which is known-good version) into the folder C:\Program Files\Common Files\Business Objects\3.0\bin (where most of the Crystal components are installed). I then ran a crdeploy.reg file that was already present in in the bin folder: this file adds a HKEY_LOCAL_MACHINE\SOFTWARE\Business Objects\Suite 11.0\Crystal Reports key to the Registry and sets the value CommonFiles to C:\Program Files\Common Files\Business Objects\3.0\bin.

Since I couldn’t connect to the customer’s machine earlier today, I did some more testing of the issue on a Windows 7 virtual machine. Like I mentioned in one of my edits, on this computer Crystal Reports never looked in the C:\Program Files\Common Files\Business Objects\3.0\bin directory for usp10.dll, so it would immediately try to load the copy in the C:\Program Files\Common Files\Microsoft Shared\Office10 folder.

It turns out the when Crystal Reports calls LoadLibrary, it checks the following folders for usp10.dll:

• If HKEY_LOCAL_MACHINE\SOFTWARE\Business Objects\Suite 11.0\Crystal Reports\CommonFiles is present in the Registry, it calls LoadLibrary using that path.

• If the registry key is not present, or usp10.dll doesn’t exist in that folder, Crystal Reports will call LoadLibrary with C:\Program Files\Common Files\Microsoft Shared\Office10\usp10.dll as the path.

• If the file isn’t found in the Office10 folder, it passes just the filename (usp10.dll) to LoadLibrary, which then causes Windows to load the copy in System32.

So, on my test Windows 7 machine, I didn’t have the CommonFiles registry key set, so Crystal Reports always loaded the version of usp10.dll that was in the Office10 folder, even after putting a copy of usp10.dll in C:\Program Files\Common Files\Business Objects\3.0\bin. Once I set the registry key to point to the right place, Crystal Reports loaded the correct version of the file.

On the customer’s machine, the registry already had the CommonFiles path set to the right folder, but our application’s setup program wasn’t installing usp10.dll to that folder, so it was still picking up the copy in the Office10 folder.

The final workaround given to the customer was stupidly simple:

1. Copy the version of usp10.dll from the System32 into C:\Program Files\Common Files\Business Objects\3.0\bin.

2. Run the crdeploy.reg file in the bin folder to ensure that the CommonFiles registry key exists and is pointing to C:\Program Files\Common Files\Business Objects\3.0\bin.

I had originally thought putting a copy of usp10.dll into the bin folder would fix the problem on the customer’s machines, but like I said, this didn’t work on my Windows 7 test machine because I was missing the CommonFiles registry key, so I was hesitant to consider the issued fixed.

Also, in case it helps anyone else experiencing this problem, the versions of usp10.dll involved were:

• 1.405.2416.1: This is the version in the Office10 folder, and the one that causes Crystal Reports to crash. When you print a report, an access violation occurs when Crystal Reports calls one of the functions in usp10.dll (I don’t have the original stacktrace, but I think it was the ScriptApplyDigitSubstitution function).

• 1.626.7600.16385: This is a known good version that works correctly with Crystal Reports. This version seems to be the one installed in Windows 7 by default.

There are other versions, such as installed by default in Windows XP in the System32 folder that also work fine with Crystal Reports.