Hooking is a computer programming term that refers to a collection of techniques employed to change how applications or operating systems behave. Hooking involves the interception of function calls, system events, or messages, and the code snippets that perform these interceptions are called hooks.
Hooking has a variety of uses, and it’s commonly used for debugging or extending the functionality of particular applications. Hooking can also be employed to monitor behavior, as is the case in performance benchmarking software, where hooks are used to provide the inputs and outputs for system component temperatures, voltages, clock speeds, or frame rates in video games.
However, hooking can also be used by malicious code, as is the case with rootkits, a form of malware that frequently relies on hooking to fabricate API calls so that it can remain hidden within a system.
Conversely, similar to how benchmarking software employs hooking techniques to monitor a system, computer security researchers also use hooking techniques to monitor malware within virtual sandbox analysis environments.
How Hooking Works
As a testament to its flexibility as a cybersecurity tool, both malware authors and security researchers make use of hooking.
Malware generally uses hooking techniques in two principal ways. First, malware uses hooking techniques to steal sensitive information like passwords or banking credentials. This is done by inserting hooks that monitor user-inputs from the keyboard at critical times or with hooks that intercept sensitive information before it can be transmitted across networks. Some advanced forms of malware also use hooking techniques to alter the returned results of file-system or registry API calls to obfuscate the malware’s existence within a victim’s system, from the system itself.
Second, hooking techniques are also used by security researchers primarily as part of a dynamic malware analysis toolset. This involves inserting user-mode or kernel-level driver hooks into a sandbox environment to intercept API and system calls made by malware to better understand its behavior and the threats it may pose. However, using hooking in this way is much less prevalent because increasingly evasive malware types are able to detect the inserted hooks in sandbox environments, making them more difficult to reliably study using this method alone.
