How to Prevent DLL Hijacking in C

Dynamic-link library (DLL) hijacking and library injection occur when an application loads a malicious DLL from an untrusted location instead of the intended system or application library. In C programming on Windows, preventing these vulnerabilities requires developers to bypass insecure default search orders, sanitize dynamic loading calls, and restrict the process-wide library search path. By enforcing absolute paths, utilizing secure API flags with LoadLibraryEx, stripping the current working directory from the search order, and validating digital signatures, C applications can effectively mitigate DLL hijacking risks.

Remove the Current Working Directory from the Search Order

By default, Windows applications may include the current working directory (CWD) in the library search order. If an attacker places a malicious DLL in a directory containing documents that the application opens, the application might load the attacker's DLL.

To eliminate the current working directory from the search path at runtime, call SetDllDirectory with an empty string during application startup:

#include <windows.h>

void SecureSearchPath(void) {
    // Passing an empty string removes the current directory from the DLL search path
    if (!SetDllDirectoryA("")) {
        // Handle error
    }
}

Enforce Modern Default Search Paths

For Windows 8, Windows Server 2012, and newer systems (or older systems with KB2533623), the Windows API provides SetDefaultDllDirectories. This function sets a strict search order across the entire process:

#include <windows.h>

void SetStrictDllSearchOrder(void) {
    // Restrict DLL lookups to System32, the application directory, and user-specified directories
    DWORD flags = LOAD_LIBRARY_SEARCH_SYSTEM32 | LOAD_LIBRARY_SEARCH_APPLICATION_DIR;
    
    if (!SetDefaultDllDirectories(flags)) {
        // Handle initialization failure
    }
}

By setting these flags, the application ignores the current working directory and the system PATH environment variable unless explicitly allowed.

Use LoadLibraryEx Instead of LoadLibrary

Avoid using standard LoadLibrary or LoadLibraryA/LoadLibraryW when dynamically loading libraries, as these functions rely on the legacy DLL search order. Instead, use LoadLibraryEx with specific search-order flags.

1. Loading System Libraries

When loading built-in operating system libraries, force the loader to check only the Windows system directory:

#include <windows.h>

HMODULE LoadSystemModule(LPCWSTR dllName) {
    return LoadLibraryExW(dllName, NULL, LOAD_LIBRARY_SEARCH_SYSTEM32);
}

2. Using Fully Qualified Absolute Paths

When loading custom application modules, compute the full, canonical path programmatically using the application's executable directory rather than relying on relative paths:

#include <windows.h>
#include <strsafe.h>

HMODULE LoadApplicationModule(LPCWSTR dllRelativeName) {
    WCHAR fullPath[MAX_PATH];
    DWORD length = GetModuleFileNameW(NULL, fullPath, MAX_PATH);

    if (length == 0 || length >= MAX_PATH) {
        return NULL;
    }

    // Strip the executable name to obtain the application directory
    WCHAR *lastBackslash = wcsrchr(fullPath, L'\\');
    if (!lastBackslash) {
        return NULL;
    }
    *(lastBackslash + 1) = L'\0';

    // Append the DLL name
    if (FAILED(StringCchCatW(fullPath, MAX_PATH, dllRelativeName))) {
        return NULL;
    }

    // Load explicitly from the verified absolute path
    return LoadLibraryExW(fullPath, NULL, LOAD_WITH_ALTERED_SEARCH_PATH);
}

Secure Third-Party Dependency Directories

If an application depends on libraries located in dedicated subdirectories, explicitly register those directories using AddDllDirectory instead of modifying the system PATH:

#include <windows.h>

DLL_DIRECTORY_COOKIE RegisterSecureDirectory(PCWSTR dirPath) {
    // Must be used after SetDefaultDllDirectories has been called
    return AddDllDirectory(dirPath);
}

Validate Digital Signatures Before Execution

For environments requiring high security, verify the authenticity of a dynamic library before calling LoadLibraryEx. Windows provides the WinVerifyTrust API to check whether a binary has been signed with a trusted certificate:

#include <windows.h>
#include <softpub.h>
#include <wincrypt.h>
#include <wintrust.h>

#pragma comment(lib, "wintrust.lib")

BOOL VerifyEmbeddedSignature(LPCWSTR filePath) {
    WINTRUST_FILE_INFO fileData = { 0 };
    fileData.cbStruct = sizeof(WINTRUST_FILE_INFO);
    fileData.pcwszFilePath = filePath;

    WINTRUST_DATA trustData = { 0 };
    trustData.cbStruct = sizeof(WINTRUST_DATA);
    trustData.dwUIChoice = WTD_UI_NONE;
    trustData.fdwRevocationChecks = WTD_REVOKE_WHOLECHAIN;
    trustData.dwUnionChoice = WTD_CHOICE_FILE;
    trustData.pFile = &fileData;
    trustData.dwStateAction = WTD_STATEACTION_VERIFY;

    GUID policyGuid = WINTRUST_ACTION_GENERIC_VERIFY_V2;
    LONG status = WinVerifyTrust(NULL, &policyGuid, &trustData);

    // Clean up state
    trustData.dwStateAction = WTD_STATEACTION_CLOSE;
    WinVerifyTrust(NULL, &policyGuid, &trustData);

    return (status == ERROR_SUCCESS);
}

Summary of Implementation Best Practices

1. Invoke SetDefaultDllDirectories at the very beginning of the main or WinMain function.
2. Replace all occurrences of LoadLibrary with LoadLibraryEx paired with LOAD_LIBRARY_SEARCH_SYSTEM32 or absolute path controls.
3. Avoid relative paths when referencing dependencies in runtime configurations.
4. Ensure build manifests do not declare weak load dependencies.