Memory-Mapped Files for Sharing Data in C

This article provides an overview of memory-mapped files (MMFs) in C#, detailing how they facilitate high-performance inter-process communication (IPC) by sharing memory between independent processes. You will learn the difference between persisted and non-persisted memory-mapped files, how to implement a writer and a reader using the System.IO.MemoryMappedFiles namespace, and how to safely synchronize access to shared memory segments using system-wide primitives.

Understanding Memory-Mapped Files

A memory-mapped file maps the contents of a file or a block of system paging memory directly into a process's virtual address space. In .NET, memory-mapped files are managed through the System.IO.MemoryMappedFiles.MemoryMappedFile class.

When two or more processes map the same physical memory space, changes made by one process are instantly visible to the others without the overhead of context switching, network stacks, or serialization required by protocols such as sockets or named pipes. This makes memory-mapped files one of the fastest IPC mechanisms available in Windows and Linux environments.

There are two primary types of memory-mapped files:

• Persisted Memory-Mapped Files: Backed by an actual source file stored on disk. When data is written, it is committed to both the memory map and the underlying physical file.
• Non-Persisted Memory-Mapped Files: Backed by system memory (the OS paging file) rather than a disk file. These are ideal for transient inter-process data exchange and disappear when all referencing processes close their handles.

Creating and Writing to a Shared Memory Segment

To share data between processes, one process typically creates a named, non-persisted memory-mapped file, and subsequent processes open it using the same name.

using System;
using System.IO.MemoryMappedFiles;
using System.Text;
using System.Threading;

class WriterProcess
{
    static void Main()
    {
        string mapName = "SharedMemoryDemo";
        long capacity = 1024 * 1024; // 1 MB capacity

        // Create the named memory-mapped file
        using var mmf = MemoryMappedFile.CreateNew(mapName, capacity);

        // Create a named mutex to synchronize access
        using var mutex = new Mutex(false, "SharedMemoryDemoMutex");

        Console.WriteLine("Writer started. Writing data to memory...");

        using (var accessor = mmf.CreateViewAccessor(0, capacity))
        {
            string message = "Data transfer via memory-mapped file at " + DateTime.UtcNow;
            byte[] buffer = Encoding.UTF8.GetBytes(message);

            mutex.WaitOne();
            try
            {
                // Write message length followed by the payload
                accessor.Write(0, buffer.Length);
                accessor.WriteArray(4, buffer, 0, buffer.Length);
            }
            finally
            {
                mutex.ReleaseMutex();
            }
        }

        Console.WriteLine("Data written. Press Enter to exit and free resources.");
        Console.ReadLine();
    }
}

Reading from a Shared Memory Segment

A secondary process accesses the shared memory region using MemoryMappedFile.OpenExisting, referencing the unique string identifier defined by the creator.

using System;
using System.IO.MemoryMappedFiles;
using System.Text;
using System.Threading;

class ReaderProcess
{
    static void Main()
    {
        string mapName = "SharedMemoryDemo";

        try
        {
            // Open the existing named memory-mapped file
            using var mmf = MemoryMappedFile.OpenExisting(mapName);
            using var mutex = Mutex.OpenExisting("SharedMemoryDemoMutex");

            using var accessor = mmf.CreateViewAccessor();

            mutex.WaitOne();
            try
            {
                // Read the length header, then read the byte payload
                int length = accessor.ReadInt32(0);
                byte[] buffer = new byte[length];
                accessor.ReadArray(4, buffer, 0, length);

                string message = Encoding.UTF8.GetString(buffer);
                Console.WriteLine($"Received message: {message}");
            }
            finally
            {
                mutex.ReleaseMutex();
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine("The specified memory-mapped file does not exist.");
        }
    }
}

Synchronization and Data Integrity

Memory-mapped files provide no built-in concurrency control. Simultaneous writes or concurrent reads and writes can result in race conditions and corrupted data.

To ensure thread and process safety:

• System-Wide Mutexes: Use a named System.Threading.Mutex across processes to serialize write access or mutually exclusive read/write blocks.
• EventWaitHandles: Use named EventWaitHandle instances (or AutoResetEvent/ManualResetEvent) to signal to other processes when new data is ready to be consumed.
• MemoryMappedViewAccessor vs. Stream: Use MemoryMappedViewAccessor for random memory access (reading and writing primitives/structures directly) and CreateViewStream() when working with sequential stream-based serialization formats like JSON, Protocol Buffers, or MessagePack.