Files
reactos/base/system
Hermès Bélusca-Maïto 8c9039bbd4 [WINLOGON] Workaround buggy 3rd-party DLLs that use a wrong notification handler calling convention (#8640)
CORE-20279

PRELIMINARY REMARK: The described bug and code workaround only applies
for x86 32-bit builds.

----

While the Winlogon notification handlers[^1] actually use a `STDCALL`
calling convention, which can be trivially verified by debugging the
official Windows <= 2003 winlogon.exe and its notification extensions,
there exist 3rd-party Winlogon notification DLLs, like the `Ati2evxx.dll`
one from AMD/ATI XP video drivers, that use a `CDECL` calling convention,
or an invalid number (zero) of parameters.

I think the reason why this happens is as follows.
The official documentation[^1] indicates that the handlers have the
following prototype:
```c
void Event_Handler_Function_Name(
  _In_ PWLX_NOTIFICATION_INFO pInfo
);
```
The documentation (and possibly the internal header Windows is using for
Winlogon) is sloppy, because it doesn't tell whether the convention is
`STDCALL` or `CDECL`. When compiling routines with such a signature, the
compiler will employ whatever default convention it is set to use.

Windows code is typically compiled with `STDCALL` convention as the default
(see e.g. how the Windows Development Kit is set up), thus, such a
function signature would default to `STDCALL`. Observation (with debugger)
shows that it is what Windows' winlogon.exe is indeed expecting.

However, 3rd-party code using a different development environment, could
set the compiler to use `CDECL` as the default calling convention. As a
result, the function signature from above would use `CDECL` instead.

The difference between the `STDCALL` and `CDECL` conventions is how the
function parameters are passed on the stack and how the stack is cleaned
at the end (`STDCALL`: the function unwinds the stack; `CDECL`: the caller
does it). A calling convention mismatch would therefore corrupt the stack,
and this is exactly what happens with the `Ati2evxx.dll` from the AMD/ATI
drivers, see CORE-20279.

The ReactOS Winlogon crashes from the `_RTC_Failure()` handler just after
the 3rd-party handler returns, since we compile our code with runtime checks
enabled. Windows' winlogon.exe doesn't apparently crash, because neither
in Release nor in Checked/Debug mode did they compile winlogon.exe with
RTC enabled. However, its stack would become more corrupt with time.

In order to alleviate this in ReactOS' winlogon.exe, I decided to use
a "generic" workaround, manually calling the handler with inline ASM
(which is OK since the problem and solution is x86-specific only).
It does something similar to what the RTC support does: it checks the
stack pointer after the call and restores it if needed.
An informative message is then emitted in the debugger telling which DLL
is buggy and needs to be fixed.

[^1]: https://learn.microsoft.com/en-us/windows/win32/secauthn/event-handler-function-prototype
2026-01-29 21:04:44 +01:00
..
2021-07-30 17:14:57 +03:00