Multiple Threads Writing to the Same UDP Socket

Writing to a single UDP socket simultaneously from multiple threads is generally safe in terms of datagram integrity, as modern operating systems ensure that individual packets are sent atomically without corrupting each other’s payloads. However, doing so introduces non-deterministic packet ordering, kernel-level lock contention that can degrade performance, and challenges related to socket buffer exhaustion.

Datagram Atomicity

At the operating system level, system calls such as send(), sendto(), and write() on a UDP socket are atomic operations. Because UDP is a message-oriented protocol, the kernel packages each call into a distinct, individual datagram. Unlike stream-oriented protocols such as TCP, the payload from one thread will never be interleaved or mixed into the middle of a payload from another thread. Every datagram reaches the network interface as a complete, separate unit.

Non-Deterministic Packet Ordering

While each datagram remains intact, the execution order among concurrent threads is non-deterministic. Operating system thread scheduling dictates which thread acquires the socket resource first. As a result, the sequence in which packets are transmitted over the network and received by the destination cannot be guaranteed. If your application logic relies on sequential data processing, you must implement sequence numbering within the application-layer payload.

Kernel Contention and Performance Overhead

When multiple threads concurrently issue write system calls on the same socket file descriptor, they compete for the socket’s internal kernel lock.

• Lock Contention: The kernel serializes access to the socket’s transmit queue, forcing threads to wait on internal mutexes or spinlocks.
• CPU Cache Invalidation: Frequent context switching and shared resource access across multiple CPU cores reduce caching efficiency, limiting throughput gains from multithreading.

Socket Buffer Exhaustion

Concurrent writes can rapidly saturate the socket’s send buffer (SO_SNDBUF):

• Blocking Sockets: If the send buffer is full, subsequent calling threads will block until space becomes available, causing latency spikes across multiple application threads.
• Non-Blocking Sockets: If the buffer fills up, calls will immediately fail and return EWOULDBLOCK or EAGAIN, requiring explicit retry and backoff mechanisms within the application.

Connected vs. Unconnected UDP Sockets

If multiple threads use sendto() with different destination addresses on the same unconnected socket, routing lookups and address handling occur per call without conflict.

However, if threads attempt to call connect() on a shared socket concurrently to change destinations while other threads are writing with send(), unpredictable behavior or errors will occur. A shared UDP socket should remain either strictly unconnected or permanently connected to a single remote endpoint.

Recommended Alternatives

To avoid lock contention and simplify thread synchronization, consider the following architectures:

1. Socket-per-Thread: Assign a dedicated UDP socket to each worker thread. This eliminates kernel lock sharing and maximizes parallel throughput.
2. Channel or Queue Architecture: Use a lock-free queue or message ring buffer where worker threads push messages, and a single dedicated I/O thread handles all socket write operations.
3. SO_REUSEPORT (Linux): Bind multiple independent sockets to the same port across different threads, allowing the kernel to distribute incoming and outgoing UDP traffic without shared socket locks.