Obviously, you will get an OutOfMemory error with this code, on my machine I was able to spawn 40k threads before running out of memory. Sometimes people have to build systems able to handle an enormous number of simultaneous clients. Native threads are inadequate means for doing that due to RAM consumption and context switching costs. Instead, use semaphores to make sure only a specified number of threads are accessing that resource. Virtual threads help in achieving the same high scalability and throughput as the asynchronous APIs with the same hardware configuration, without adding the syntax complexity. Also, we have to adopt a new programming style away from typical loops and conditional statements.

There are other ways to achieve what virtual thread do (such as NIO and the related Reactor pattern). This, however, entails using message loops and callbacks which warp your mind (that’s why so many people hate JavaScript). There are layers of abstractions on top of them making things a bit easier but they also have a cost. Virtual threads give us an ability to run millions of I/O bound tasks simultaneously without changing our mental model. OS threads are more expensive and they are slower to start/stop, have to deal with context switching (magnified by their number) and you are dependent on the OS which might refuse to give you more threads.

Few words on the Java Concurrency model

Project Loom aims to address this issue by splitting threads into two distinct concepts. This could lead to the under-desirable state of an application that has maxed out its thread allocation while still having low CPU utilization. To switch to Virtual threads we don’t have to learn new things, we just have to unlearn a few. Both threads seem to work in User Space and not in Kernel Space as Javas Native Threads do. It is worth noting that Thread.ofVirtual().start(runnable) is equivalent to Thread.startVirtualThread(runnable). A long-form guide on how to make microservices communicate with gRPC in Scala.

The JVM will eventually be implemented so that synchronized methods or blocks no longer lead to pinning. Next, the main thread has continued its execution as if the run() method returned and continued with the while loop. After the second call to continuation’s run method, the JVM restored the state of the main thread to the point where the continuation has yielded and finished the execution.

Use descriptive and meaningful resource names-

For this reason, Java 20 will introduce scoped values, which enable the sharing of immutable data within and across threads. As an example, on my personal web site, I provide demo services for producing random items. If a large number of requests comes at an instant from the same IP address, the hosting company blacklists the IP address.

On the other side, for Virtual Threads, regardless of the number of concurrency of request, almost all of them were processed immediately having a similar number of transactions per second. These might seem like small optimizations and indeed they are insignificant with small applications or servers with low load. These mariadb developers things matter when you need to process millions of requests per day they can be a game changer and drastically increase your throughput in some situations. The code is the same, we just have to use Executors.newVirtualThreadPerTaskExecutor() which creates a new green thread every time a task is submitted to it.

What is a Virtual Thread?

The scheduling of Virtual Threads is handled by default with a java.util.concurrent.ForkJoinPool. It operates on a FIFO model and has a parallel capacity equivalent to the number of available Platform Threads, which itself is based on the number of available processors. Virtual Threads extend java.lang.Thread but run on top of Platform Threads and are not linked to underlying OS threads.

These frameworks also make us give up a number of the runtime features that make developing in Java easier. This programming style is at odds with the Java Platform because the frameworks unit of concurrency — a stage of an asynchronous pipeline — is not the same as the platforms unit of concurrency. Virtual threads, on the other hand, allow us to gain the same throughput benefit without giving up key language and runtime features. The following example illustrates using virtual threads to concurrently fetch two URLs and aggregate their results as part of handling a request. It creates an ExecutorService that runs each task in a new virtual thread, submits two tasks to it, and waits for the results. ExecutorService has been retrofitted to implement AutoCloseable, so it can be used with try-with-resources, and the close method shuts down the executor and waits for tasks to complete.

When not to use Virtual Threads

The Thread class is the way Java gives you access to the OS Thread API. Most of the operations performed in this class make system calls. In production we rarely use the Thread directly, we use the Java concurrency package with the Thread Pools, Locks along with other nice things. And of course, there would have to be some actual I/O or other thread parking for Loom to bring benefits. Project Loom has revisited all areas in the Java runtime libraries that can block and updated the code to yield if the code encounters blocking.

As we said, structural concurrency and scoped values are some of them. This article will help you better understand virtual threads and how to use them. In the beginning, we introduced the reason behind the introduction of virtual threads in the JVM. We made some examples of pinned threads, and finally, we saw how some old best practices are no longer valid when using virtual threads. In this section, we’ll introduce the implementation of continuation in Java virtual threads. We’re not going into too much detail, but we’ll try to give a general idea of how the virtual threads are implemented.

The Scheduler and Cooperative Scheduling

The client requests something, while we fetch the data or do the processing this thread is taken and can’t be used by anyone else. Servers start and allocate a predefined number of threads (for example 200 for Tomcat). Their initial state is called “Parked”, while they are in this state they don’t take CPU resources.

• Note that after using the virtual threads, our application may be able to handle millions of threads, but other systems or platforms handle only a few requests at a time.
• Spring Runtime offers support and binaries for OpenJDK™, Spring, and Apache Tomcat® in one simple subscription.
• Now, calling factory.newThread(myRunnable) creates a new (unstarted) virtual thread.
• In fact, the implementation of the ExecutorService interface present in the
  java.util.concurrent package is a thread pool implementation.
• As we said, both the JEPs are still in the preview/incubation step, so we must enable them in our project.
• And of course, there would have to be some actual I/O or other thread parking for Loom to bring benefits.

When we hit the limit on concurrent threads, the throughput of the thread-per-task model is limited by Littles Law. Virtual threads address this in a graceful way by giving us more concurrent threads rather than asking us to change our programming model. Whenever a virtual thread blocks, it is unmounted, and the platform thread runs another virtual thread.

Modern Java server concurrency problems

In concurrent programming, we should write programs as if they were sequential. In fact, the more straightforward way to write concurrent programs in Java is to create a new thread for every concurrent task. There are other ways of using Thread to spawn virtual threads, like Thread.ofVirtual().start(runnable). To locate uses of thread locals in your app, run with the VM flag jdk.traceVirtualThreadLocals. You will get a stack trace when a virtual thread mutates a thread-local variable. Another common use for thread locals is to provide “implicit” context, such as a database connection, that is properly configured for each task.

Working with Virtual Threads in Spring 6

We can use the Thread.Builder reference to create and start multiple threads. In this example, we ran our continuation and, at some point, decided to stop the processing. Then once we re-ran it, our continuation continued from where it left off. By the output, we see that the run() method was called twice, but the continuation was started once and then continued its execution on the second run from where it left off.

A lot of changes and proposals from the first draft, but it seems to be ready in Java 21. Virtual threads are a preview feature of Java 19, which means that they will be included in an official JDK release within the next 12 months. Originally introduced by Project Loom, Spring 6 release grants developers the option to start experimenting with this awesome feature.