12 Virtual Threads
Virtual threads are lightweight threads that reduce the effort of writing, maintaining, and debugging high-throughput concurrent applications.
Note:
This is a preview feature. A preview feature is a feature whose design, specification, and implementation are complete, but is not permanent. A preview feature may exist in a different form or not at all in future Java SE releases. To compile and run code that contains preview features, you must specify additional command-line options. See Preview Language and VM Features.For background information about virtual threads, see JEP 436.
A thread is the smallest unit of processing that can be scheduled. It runs concurrently with—and largely independently of—other such units. It's an instance of java.lang.Thread. There are two kinds of threads, platform threads and virtual threads.
What is a Platform Thread?
A platform thread is implemented as a thin wrapper around an operating system (OS) thread. A platform thread runs Java code on its underlying OS thread, and the platform thread captures its OS thread for the platform thread's entire lifetime. Consequently, the number of available platform threads is limited to the number of OS threads.
Platform threads typically have a large thread stack and other resources that are maintained by the operating system. Platform threads support thread-local variables.
Platforms threads are suitable for running all types of tasks but may be a limited resource.
What is a Virtual Thread?
Like a platform thread, a virtual thread is also an instance of java.lang.Thread. However, a virtual thread isn't tied to a specific OS thread. A virtual thread still runs code on an OS thread. However, when code running in a virtual thread calls a blocking I/O operation, the Java runtime suspends the virtual thread until it can be resumed. The OS thread associated with the suspended virtual thread is now free to perform operations for other virtual threads.
Virtual threads are implemented in a similar way to virtual memory. To simulate a lot of memory, an operating system maps a large virtual address space to a limited amount of RAM. Similarly, to simulate a lot of threads, the Java runtime maps a large number of virtual threads to a small number of OS threads.
Unlike platform threads, virtual threads typically have a shallow call stack, performing as few as a single HTTP client call or a single JDBC query. Although virtual threads support thread-local variables, you should carefully consider using them because a single JVM might support millions of virtual threads.
Virtual threads are suitable for running tasks that spend most of the time blocked, often waiting for I/O operations to complete. However, they aren't intended for long-running CPU-intensive operations.
Why Use Virtual Threads?
Use virtual threads in high-throughput concurrent applications, especially those that consist of a great number of concurrent tasks that spend much of their time waiting. Server applications are examples of high-throughput applications because they typically handle many client requests that perform blocking I/O operations such as fetching resources.
Virtual threads are not faster threads; they do not run code any faster than platform threads. They exist to provide scale (higher throughput), not speed (lower latency).
Creating and Running a Virtual Thread
The Thread and Thread.Builder APIs provide ways to create both platform and virtual threads. The java.util.concurrent.Executors class also defines methods to create an ExecutorService that starts a new virtual thread for each task.
Creating a Virtual Thread with the Thread Class and the Thread.Builder Interface
Call the Thread.ofVirtual() method to create an instance of Thread.Builder for creating virtual threads.
The following example creates and starts a virtual thread that prints a message. It calls the join method to wait for the virtual thread to terminate. (This enables you to see the printed message before the main thread terminates.)
Thread thread = Thread.ofVirtual().start(() -> System.out.println("Hello"));
thread.join();The Thread.Builder interface lets you create threads with common Thread properties such as the thread's name. The Thread.Builder.OfPlatform subinterface creates platform threads while Thread.Builder.OfVirtual creates virtual threads.
The following example creates a virtual thread named
MyThread with the Thread.Builder
interface:
try {
Thread.Builder builder = Thread.ofVirtual().name("MyThread");
Runnable task = () -> {
System.out.println("Running thread");
};
Thread t = builder.start(task);
System.out.println("Thread t name: " + t.getName());
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}The following example creates and starts two virtual threads with Thread.Builder:
public class CreateNamedThreadsWithBuilders {
public static void main(String[] args) {
try {
Thread.Builder builder =
Thread.ofVirtual().name("worker-", 0);
Runnable task = () -> {
System.out.println("Thread ID: " +
Thread.currentThread().threadId());
};
// name "worker-0"
Thread t1 = builder.start(task);
t1.join();
System.out.println(t1.getName() + " terminated");
// name "worker-1"
Thread t2 = builder.start(task);
t2.join();
System.out.println(t2.getName() + " terminated");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}This example prints output similar to the following:
Thread ID: 21
worker-0 terminated
Thread ID: 24
worker-1 terminatedCreating and Running a Virtual Thread with the Executors.newVirtualThreadPerTaskExecutor() Method
Executors let you to separate thread management and creation from the rest of your application.
The following example creates an ExecutorService with the Executors.newVirtualThreadPerTaskExecutor() method. Whenever ExecutorService.submit(Runnable) is called, a new virtual thread is created and started to run the task. This method returns an instance of Future. Note that the method Future.get() waits for the thread's task to complete. Consequently, this example prints a message once the virtual thread's task is complete.
try (ExecutorService myExecutor =
Executors.newVirtualThreadPerTaskExecutor()) {
Future<?> future =
myExecutor.submit(() -> System.out.println("Running thread"));
future.get();
System.out.println("Task completed");
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
} Multithreaded Client Server Example
The following example consists of two classes. EchoServer is
a server program that listens on a port and starts a new virtual thread for each connection.
EchoClient is a client program that connects to the server and sends
messages entered on the command line.
EchoClient creates a socket, thereby getting a connection to
EchoServer. It reads input from the user on the standard input
stream, and then forwards that text to EchoServer by writing the text
to the socket. EchoServer echoes the input back through the socket to
the EchoClient. EchoClient reads and displays the data
passed back to it from the server. EchoServer can service multiple
clients simultaneously through virtual threads, one thread per each client
connection.
public class EchoServer {
public static void main(String[] args) throws IOException {
if (args.length != 1) {
System.err.println("Usage: java EchoServer <port>");
System.exit(1);
}
int portNumber = Integer.parseInt(args[0]);
try (
ServerSocket serverSocket =
new ServerSocket(Integer.parseInt(args[0]));
) {
while (true) {
Socket clientSocket = serverSocket.accept();
// Accept incoming connections
// Start a service thread
Thread.ofVirtual().start(() -> {
try (
PrintWriter out =
new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(clientSocket.getInputStream()));
) {
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
out.println(inputLine);
}
} catch (IOException e) {
e.printStackTrace();
}
});
}
} catch (IOException e) {
System.out.println("Exception caught when trying to listen on port "
+ portNumber + " or listening for a connection");
System.out.println(e.getMessage());
}
}
}public class EchoClient {
public static void main(String[] args) throws IOException {
if (args.length != 2) {
System.err.println(
"Usage: java EchoClient <hostname> <port>");
System.exit(1);
}
String hostName = args[0];
int portNumber = Integer.parseInt(args[1]);
try (
Socket echoSocket = new Socket(hostName, portNumber);
PrintWriter out =
new PrintWriter(echoSocket.getOutputStream(), true);
BufferedReader in =
new BufferedReader(
new InputStreamReader(echoSocket.getInputStream()));
) {
BufferedReader stdIn =
new BufferedReader(
new InputStreamReader(System.in));
String userInput;
while ((userInput = stdIn.readLine()) != null) {
out.println(userInput);
System.out.println("echo: " + in.readLine());
if (userInput.equals("bye")) break;
}
} catch (UnknownHostException e) {
System.err.println("Don't know about host " + hostName);
System.exit(1);
} catch (IOException e) {
System.err.println("Couldn't get I/O for the connection to " +
hostName);
System.exit(1);
}
}
}Scheduling Virtual Threads and Pinned Virtual Threads
The operating system schedules when a platform thread is run. However, the Java runtime schedules when a virtual thread is run. When the Java runtime schedules a virtual thread, it assigns or mounts the virtual thread on a platform thread, then the operating system schedules that platform thread as usual. This platform thread is called a carrier. After running some code, the virtual thread can unmount from its carrier. This usually happens when the virtual thread performs a blocking I/O operation. After a virtual thread unmounts from its carrier, the carrier is free, which means that the Java runtime scheduler can mount a different virtual thread on it.
A virtual thread cannot be unmounted during blocking operations when it is pinned to its carrier. A virtual thread is pinned in the following situations:
- The virtual thread runs code inside a
synchronizedblock or method - The virtual thread runs a
nativemethod or a foreign function (see Foreign Function and Memory API)
Pinning does not make an application incorrect, but it might hinder its scalability.
Try avoiding frequent and long-lived pinning by revising synchronized
blocks or methods that run frequently and guarding potentially long I/O operations with
java.util.concurrent.locks.ReentrantLock.
Debugging Virtual Threads
Virtual threads are still threads; debuggers can step through them like
platform threads. Java Flight Recorder and the jcmd tool have additional
features to help you observe virtual threads in your applications.
Java Flight Recorder Events for Virtual Threads
Java Flight Recorder (JFR) can emit these events related to virtual threads:
jdk.VirtualThreadStartandjdk.VirtualThreadEndindicate when a virtual thread starts and ends. These events are disabled by default.jdk.VirtualThreadPinnedindicates that a virtual thread was pinned (and its carrier thread wasn’t freed). This event is enabled by default with a threshold of 20 ms.jdk.VirtualThreadSubmitFailedindicates that starting or unparking a virtual thread failed, probably due to a resource issue. Parking a virtual thread releases the underlying carrier thread to do other work, and unparking a virtual thread schedules it to continue. This event is enabled by default.
Enable the events jdk.VirtualThreadStart and
jdk.VirtualThreadEnd through JDK Mission Control or with a custom
JFR configuration as described in Flight Recorder Configurations in Java Platform, Standard Edition Flight Recorder API
Programmer’s Guide.
To print these events, run the following command, where
recording.jfr is the file name of your recording:
jfr print --events
jdk.VirtualThreadStart,jdk.VirtualThreadEnd,jdk.VirtualThreadPinned,jdk.VirtualThreadSubmitFailed
recording.jfr
Viewing Virtual Threads in jcmd Thread Dumps
You can create a thread dump in plain text was well as JSON format:
jcmd <PID> Thread.dump_to_file -format=text <file>
jcmd <PID> Thread.dump_to_file -format=json <file>
The JSON format is ideal for debugging tools that accept this format.
The jcmd thread dump lists virtual threads that are blocked
in network I/O operations and virtual threads that are created by the ExecutorService interface. It does not include object
addresses, locks, JNI statistics, heap statistics, and other information that appears in
traditional thread dumps.
Practical Advice for Virtual Threads
Because virtual threads are inexpensive and plentiful, many programming techniques you would usually use because platform threads are expensive and heavyweight are no longer applicable or recommended.
Don't Pool Virtual Threads
A thread pool is a group of preconstructed platform threads that are reused when they become available. Some thread pools have a fixed number of threads while others create new threads as needed.
Don't pool virtual threads. Create one for every application task. Virtual threads are short-lived and have shallow call stacks. They don't need the additional overhead or the functionality of thread pools.
Use Semaphores for Limited Resources
A semaphore restricts the number of threads that can access a physical or logical resource. Use semaphores (instead of thread pools) if you need to limit concurrency, for example, if you make sure only a specified number of threads can access a limited resource, such as requests to a database.
Avoid Pinning
As mentioned in Scheduling Virtual Threads and Pinned Virtual Threads, pinning might hinder an application's scalability. Avoid frequent
and long-lived pinning by revising synchronized blocks and guard
potentially long I/O operations with java.util.concurrent.locks.ReentrantLock, which controls access to a
shared resource by multiple threads in a similar way as implicit locks in synchronized
code.