Question

I am currently writing a keyboard driver for Windows, that provides an additional modifier and allows to directly type some additional characters. For example real quotation marks (“, ”, ‘, ’, …) and some mathematic characters (ℕ, ℝ, ℚ, ℤ, √, …).

Now I want to provide the arrow keys just like that. (alt_gr + e = up; alt_gr + s = left; alt_gr + d = down; alt_gr + f = right)

So my question is, if there are Unicode control characters for the arrow keys, that I could use in my aVkToWcharTable or maybe another solution. This would be really helpful.

Answer 1

I don’t have any prior experience with Windows driver programming, so if anyone notices something wrong, please correct me.

However I am using the NEO layout (or the AdNW variant to be precise) and was curious, so I downloaded the DDK to see what I can learn from the documentation. I don’t really have a solution for you and I think you may already know some of this stuff, but I thought that writing down what I have found may still be of some use, if only to clear things up a bit.

Overview

As far as I can tell, there are two paths that keyboard data can originate from. There are PS/2 devices and then there are USB keyboards. Both take a different path (USB data has its source in the HID infrastructure and PS/2 data is produced by the I8042prt driver), but end up at the so-called “Keyboard class driver”.

From there on the data is either passed to DirectInput or the user32 legacy interface. It is not entirely clear, where the layout comes into play, but I think it is implemented on top of the Keyboard class driver.

Layout

Keyboard layouts basically seem to be a bunch of data structures, conforming to the KBDTABLES structure defined in kbd.h. Basically the low-level drivers produce a ScanCode and interpretation of these codes is done with this structure.

The first important ones are pusVSCtoVK, pVSCtoVK_E0 and pVSCtoVK_E1, which map the raw scan codes (and extended scan codes) of the driver to the hardware-independent virtual key codes. These key codes describe the key. The character mapping comes later. So the left cursor key produces VK_LEFT, and the S key produces VK_S, even if the layout turns this to something else later.

Next there is pCharModifiers, which specifies which virtual keys are modifiers and what combinations produce the different shift states.

Lastly there is pVkToWcharTable which maps all the virtual keys to a Unicode character, depending on the shift state. This is where most of the actual layout resides. There are some special values for dead keys and ligatures, but let’s take that aside for now.

Applications

Now what do applications get at the end? Obviously they can obtain the virtual key code and the scan code from the usual window messages, such as WM_INPUT, WM_KEYDOWN, …. They can also query the layout for a Unicode character, using functions such as ToUnicodeEx. The virtual key codes and the Unicode characters are provided by the layout.

For usual characters, everything is fine. The application will receive the input event, ask the layout for a character and there you go. The cursor keys are different though, because the mapping from virtual key to character is neither needed, nor meaningful. The application cares for the key, but not for a character. So it will base the action on the virtual key code. If it is VK_LEFT, it moves the cursor left. If it is VK_RIGHT, the cursor moves right and so on.

This leaves us with a dilemma: if we want to make ModX + S move the cursor left, the modifier-aware pVkToWcharTable won’t help us, because the applications never ask for a character, but for a key. And clearly S is not the same key as the left key. Producing a control character (that doesn’t exist as far as I can tell) will most likely don’t help either, because the applications wouldn’t ask for it and don’t interpret it.

So what you actually want is for the S key to produce a different virtual key code if the proper modifier key is pressed, but the data structure for a layout won’t allow such a mapping. It is not executing code after all, only a data structure.

What about Numpad?

Numpad seems to do what is required. The scan code of the keys stay the same, but depending on the Numlock state they produce different virtual key codes. I think, however, that windows does some special handling here. Those keys have the KBDNUMPAD flag set in pusVSCtoVK. It seems that windows will detect that and exchange the virtual keys depending on the Numlock state.

I have at least found some evidence for that. Try searching for “xxxNumpadCursor” in Google. It turns up with some seemingly leaked windows code. There is a check for KBDNUMPAD and the code that seems to replace the virtual key, depending on Shift and Numlock. I doubt that this can be exploited for this purpose.

Is it even possible?

So if the layout can’t do this, who can? Obviously some lower-level driver that produces raw scan codes. There is a sample for a so-called filter driver in the DDK, which is called “kbfiltr”. Basically it seems that such a filter driver can hook up on top of the existing drivers and rewrite the data that comes from the driver, so it should be possible to rewrite the scan code of the S key to Left, if another key is pressed.

Note however that this would be a driver, not a layout. I’m not sure if that is a practical solution. It is as far as I can see unaware of the selected layout, so the changes would apply for all layouts. (It might be possible though to implement the whole layout here, complete with a key to toggle it off.) I’m also not sure how complex installation of such a driver would be and if you need to sign the driver, which would make things worse.

I don’t really have enough experience with driver programming (which is to say no experience at all), to recommend for or against this approach. It might be worth some experimenting (I would prefer a virtual machine here), but it seems a lot more challenging than the simple layout replacement that is currently done. The Autohotkey solution is probably still the best solution (with the drawback that it doesn’t work for all applications).

Update

I did some more research. I think it is possible to do this with a filter driver. This driver would basically reside between the HID infrastructure or I8042prt driver and the Keyboard class driver. It can then modify the data before it arrives at the class driver and replace/remove/add key events before they arrive there. In the end you would have to use a special layout (for the additional unicode characters) as well as the driver, which can be installed for an individual keyboard via the device manager. This solution should then work for all applications.

The problem here is the driver signature. As a kernel-mode driver it has to be signed in order to load it without ugly hacks in 64 Bit Vista/Win7. And its not cheap to sign a driver, so you’re basically out of luck with that approach. I thought that one could avoid that by writing a user-mode driver (which isn’t required to be signed), but the problem is, that the filter must be installed below the Keyboard class driver. Sadly it seems, that a user-mode filter driver can’t have a kernel-mode driver above it. So this is not an option.

So I think you’re basically stuck with AutoHotkey. Unless you are willing and able to write a kernel-mode driver and live with the signing problem.