INTEL Pintools
It is possible to detect hooks by using Intel’s Pintools
Pintools enables you to split the code into blocks, something very similar to what IDA does. It also enables you to determine to which binary or DLL this code block belongs to.
MS Detours
When you are examining a security solution it’s always a good idea to read its documentation. The product that I was analysing had specifically mentioned that it uses MS Detours hooks to function. However, it did not mention anything regarding the application control implemented in kernel space and also did not mention which DLL it used for hooking.
Unfortunately there is no single (special) Windows API that would do the hooking. Instead it uses multiple APIs to do its job. I wanted to find a rare API or a sequence of APIs that I could use as some sort of signature. I found one API that is quite special and rarely used (unless you want to do hooking): “FlushInstructionCache”.
As the documentation says:
“Applications should call FlushInstructionCache if they generate or modify code in memory. The CPU cannot detect the change, and may execute the old code it cached.”
So if the MS Detours code wants its new jump instruction to be executed it needs to call FlushInstructionCache API. In summary what MS Detours needs to do when installing the hook is to:
- Allocate memory for the trampoline function;
- Change the access of the code of the target function to make it writable;
- Copy the instructions from the beginning of the target function (the ones that it’s going to replace) to previously allocated space; and make changes there so that the trampoline function ends up executing your code;
- Replace the beginning of the target function with a jump instruction to trampoline function;
- Change the access of the code of the target function back to the original access;
- Flush the instruction cache.
https://blog.nviso.eu/2022/08/05/finding-hooks-with-windbg/