← Back to the summary

ScopedValue Is Not a ThreadLocal Killer — It Fixes What Virtual Threads Break

ThreadLocal is a scheme for isolating variables between threads, also called a thread-local variable table. In Java, each thread owns a variable of type ThreadLocalMap under ThreadLocal, which is used to store ThreadLocal objects defined in the thread. The key of ThreadLocalMap is a weak reference pointing to the corresponding ThreadLocal object.

However, it is worth noting for every Java developer that ThreadLocal variables can cause serious memory leak issues if not remove()'d in time. In JDK 20 Early-Access Build 28, a ScopedValue class was redesigned for the ThreadLocal class.

ScopedValue is a JDK incubator feature that requires manual configuration to use in the released JDK 20 version. The purpose of ScopedValue is to serve as a replacement for ThreadLocal in certain situations. Different code running on the same thread can share immutable values through ScopedValue. ScopedValue is mainly designed to solve some problems that may exist when using ThreadLocal with virtual threads.

In this article, we will introduce several case backgrounds of ThreadLocal in development practice, as well as detail the new concurrency tools proposed in JDK 19 and the ScopedValue class being incubated in JDK 20.

ThreadLocal

Basic Concepts

In the introduction, part of the concept of ThreadLocal has been roughly introduced. The significance of ThreadLocal is not only to achieve isolation but more importantly to solve the problem of object reuse. These ideas are reflected in database connection pool frameworks. However, ThreadLocal can also cause memory leak issues. This is because the ThreadLocal object in ThreadLocalMap is not reclaimed by the JVM in time. To solve this problem, a weak reference WeakReference is used, but if the weakly referenced ThreadLocal is set to null and not cleaned up in time through the remove method, it will also cause a memory leak.

Application Cases

Taking the most common Spring application as an example, ThreadLocal can be fully utilized in these applications.

In the Spring business of some e-commerce projects, thread isolation is required for each request:

@Service
public class ShoppingCartService {
    private ThreadLocal<ShoppingCart> cartHolder = new ThreadLocal<>();

    public ShoppingCart getCurrentCart() {
        ShoppingCart cart = cartHolder.get();
        if (cart == null) {
            cart = new ShoppingCart();
            cartHolder.set(cart);
        }
        return cart;
    }

    public void checkout() {
        // Get the current shopping cart
        ShoppingCart cart = getCurrentCart();
        // Execute checkout operation
        // Clear the shopping cart information in the current thread to prevent memory leaks
        cartHolder.remove();
    }
}

// Shopping cart class
class ShoppingCart {
    private List<Product> products = new ArrayList<>();

    public void addProduct(Product product) {
        products.add(product);
    }

    public List<Product> getProducts() {
        return products;
    }
}

In this code, ShoppingCartService is a Spring Bean used to manage shopping cart information. In this Bean, ThreadLocal<ShoppingCart> is used to save the shopping cart information for each thread. The getCurrentCart method first gets the shopping cart information from ThreadLocal. If the current thread does not have corresponding shopping cart information, a new shopping cart is created and saved to ThreadLocal. The checkout method is used to execute the checkout operation. After checkout is completed, the shopping cart information in the current thread needs to be cleared through cartHolder.remove(); to prevent memory leaks. In this way, even in a multi-threaded environment, each thread has its own independent shopping cart information without affecting each other. This is an application scenario of ThreadLocal in solving the thread safety problem of Spring Beans.

Using ThreadLocal in business logic is a very common method for handling thread-isolated data. Let's consider: what should be done if a series of interfaces all need to perform user authentication first and then operate on this user data? This problem is very simple. Use Spring Security integrated with JWT to parse the Token passed from the front end to get the username and then verify the user. This can be completely encapsulated into an aspect to handle. As for needing this user again in the business, just fetch it from Spring Security. However, some businesses will preprocess user data in their aspects, such as updating the access interface timestamp. Then fetching it from Spring Security again seems inappropriate, because this may lead to expectations inconsistent with the desired object. So what should be done? Obviously, use ThreadLocal to cache this user object, so that this User exists in a single state throughout the entire HTTP session process:

@Aspect
@Component
public class UserConsistencyAspect {
// Enable thread isolation for each UserVo, created upon entering the aspect, and reclaimed by GC after use in business logic
private static final ThreadLocal<UserVo> userHolder = new ThreadLocal<>();

@Pointcut("@annotation(org.nozomi.common.annotation.GetUser)")
public void userAuthPoint() {}

@Around("userAuthPoint()")
public Object injectUserFromRequest(ProceedingJoinPoint joinPoint) throws Throwable {
   Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
   UserVo operator = (UserVo) authentication.getPrincipal();
   if (operator == null) {
    return Response.fail("User does not exist");
   }
   userHolder.set(operator);
   return joinPoint.proceed();
  }

/**  
  * Retrieve the UserVo object in the current thread. These UserVos are isolated following the threads created by HTTP.
  *
  * @return UserVo of the current thread
  */
public static UserVo getUser() {
   return userHolder.get();
  }
 }

Using this aspect and the UserConsistencyAspect .getUser() method in the business allows you to obtain the User object in this HTTP session.

When solving the thread-unsafe problem of Spring Beans, ThreadLocal is used to ensure the thread safety of the Bean:

@Service
public class ProductService {

    private final ThreadLocal<Session> sessionThreadLocal = new ThreadLocal<>();

    public Product getProductById(String id) {
        Session session = getSession();
        return session.get(Product.class, id);
    }

    public void updateProduct(Product product) {
        Session session = getSession();
        session.update(product);
    }

    private Session getSession() {
        Session session = sessionThreadLocal.get();
        if (session == null) {
            session = sessionFactory.openSession();
            sessionThreadLocal.set(session);
        }
        return session;
    }

    public void closeSession() {
        Session session = sessionThreadLocal.get();
        if (session != null) {
            session.close();
            sessionThreadLocal.remove();
        }
    }
}

In many cases, developers use Spring to manage database sessions or transactions, but such Beans are usually thread-unsafe, such as Hibernate's SessionFactory or MyBatis's SqlSessionFactory. The Sessions produced by these factories are thread-unsafe. In e-commerce projects, a common scenario is that multiple interactions with the database may be required during the processing of a single request. At this time, to ensure the same database session (Session) is used within a request, this Session is usually placed in a ThreadLocal. In this way, even in different methods within a thread, the same Session can be obtained. In this example, each thread has its own Session instance, stored in ThreadLocal. When different threads call the getSession() method, they will get their own Session from ThreadLocal. But in fact, these session handlings have already been processed through ThreadLocal in MyBatis or Hibernate, and developers do not need to isolate sessions in the business themselves. The example here is mainly to explain how ThreadLocal works, not a recommended practice in actual development.

StructuredTaskScope

Structured concurrency programming is closely related to Virtual Threads. To understand ScopedValue, one must first understand these two concepts. Since JDK 5, there has been a philosophy: we should not interact with threads directly. The correct pattern is to submit tasks as Runnable or Callable to an ExecutorService or Executor, and then operate on the returned Future. The Loom project has retained this model and added some nice features. The first object to introduce here is the Scope object, the exact type being StructuredTaskScope. We can think of this object as a virtual thread launcher. We submit tasks to it in the form of Callable, we will get a future returned, and this callable will execute in the virtual thread created for us by the scope Scope. This is very similar to Executor. But there are also significant differences between the two.

public static Weather readWeather() throws Exception {
 // try-with-resource
 try(var scope = new StructuredTaskScope<Weather>()) {
  Future<Weather> future = scope.fork(Weather::readWeatherFrom);
  scope.join();
  return future.resultNow();
 }
}

StructuredTaskScope instances are AutoCloseable, so we can use the try-with-resource pattern. Fork a Callable type task through the fork() method. The fork() method returns a Future object. We call the join() method to block the call; it will block the current thread until all tasks submitted (forked) to the StructuredTaskScope are completed. Finally, call the Future's resultNow() to get the result and return it. resultNow() will throw an exception if we call it before the Future completes, so we need to call it after the join() method and return it.

ScopedValue

Basic Concepts

More relevant to structured concurrency should be CompletableFuture proposed in JDK 8, which I will further introduce in the next extra issue. ScopedValue is a feature incubated in JDK 20 based on the concept of structured concurrency. It is obviously not intended to replace ThreadLocal, but to allow virtual threads in structured concurrency to each enjoy external variables. In fact, ThreadLocal can also be used in structured concurrency, but ThreadLocal itself has some significant problems:

ThreadLocal variables are mutable. Any code running in the current thread can modify the value of the variable, easily producing bugs that are difficult to debug.
The lifecycle of ThreadLocal variables can be very long. When using the set method of a ThreadLocal variable to set a value for the current thread, this value will be retained throughout the entire lifecycle of the thread until the remove method is called to delete it. However, the vast majority of developers do not actively call remove to delete it, which may cause memory leaks.
ThreadLocal variables can be inherited. If a child thread inherits ThreadLocal variables from a parent thread, the child thread needs to independently store all the ThreadLocal variables from the parent thread, which can generate significant memory overhead.

The characteristic of virtual threads is their huge number, but each virtual thread has a short lifecycle, so memory leak issues are less likely to occur. However, the memory overhead brought by thread inheritance will be even greater. To solve these problems, ScopedValue was incubated. ScopedValue possesses the core feature of ThreadLocal, which is that each thread has only one value. Unlike ThreadLocal, ScopedValue is immutable and has a definite scope, which is the meaning of "scoped" in the name.

Basic Usage

ScopedValue objects are represented by the ScopedValue class in the jdk.incubator.concurrent package. The first step in using ScopedValue is to create a ScopedValue object, done through the static method newInstance. ScopedValue objects are generally declared as static final. Since ScopedValue is an incubator feature, to use it, you need to create a Java class module-info.java in the same directory as the project's first-level package directory to introduce it into the module:

module dioxide.cn.module {  
 requires jdk.incubator.concurrent;  
}

At the same time, you need to enable the preview feature --enable-preview in the startup parameter VM Option. The next step is to specify the value and scope of the ScopedValue object, done through the static method where. The where method has 3 parameters:

ScopedValue object
The value bound to the ScopedValue object
A Runnable or Callable object, representing the scope of the ScopedValue object

During the execution of the Runnable or Callable object, the code within can use the ScopedValue object's get method to obtain the value bound when the where method was called. This scope is dynamic, depending on the methods called by the Runnable or Callable object, and other methods called by those methods. When the Runnable or Callable object finishes execution, the ScopedValue object loses its binding and the value can no longer be obtained through the get method. Within the current scope, the value of the ScopedValue object is immutable, unless the where method is called again to bind a new value. At this time, a nested scope is created, and the new value is only valid within the nested scope. Using scoped values has the following advantages:

Improve data security: Since scoped values can only be accessed within the current scope, data leakage or malicious modification can be avoided.
Improve data efficiency: Since scoped values are immutable and can be shared between threads, the overhead of data copying or synchronization can be reduced.
Improve code clarity: Since scoped values can only be accessed within the current scope, the use of parameter passing or global variables can be reduced.

Java JEP 429 is a new feature being incubated, which provides a way to share immutable data within and between threads. Currently, Java JEP 429 is still in the incubator stage and has not been formally incorporated into the Java language specification.

public class Main {
// Declared a static, final ScopedValue<String> instance
// ScopedValue is a class that supports passing values within a specific scope (such as a task or thread)
// Its usage is similar to ThreadLocal, but more suitable for structured concurrency
    private static final ScopedValue<String> VALUE = ScopedValue.newInstance();
    
    public static void main(String[] args) throws Exception {
        System.out.println(Arrays.toString(stringScope()));
    }

    public static Object[] stringScope() throws Exception {
        return ScopedValue.where(VALUE, "value", () -> {
            // Use try-with-resource to bind the scope of structured concurrency
            // Used to automatically manage the lifecycle of resources, this is a structured task scope
            // All subtasks created within this scope will be considered part of the scope
            // If any task in the scope fails, all other tasks will be cancelled
            try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
             // Used scope.fork to create two parallel tasks
             // Each task gets the value of VALUE in the execution context and operates on it
                Future<String> user  = scope.fork(VALUE::get);
                Future<Integer> order = scope.fork(() -> VALUE.get().length());
                // join() method waits for all tasks within the scope to complete
                // throwIfFailed() method checks the results of all tasks, and throws an exception if any task fails
                scope.join().throwIfFailed();
                // After all tasks are completed, use the resultNow() method to get the result of each task, and put the results into an object array
                return new Object[]{user.resultNow(), order.resultNow()};
            }
        });
    }
}

This code demonstrates how to use ScopedValue and structured concurrency to create and execute multiple parallel tasks, and safely pass and manipulate values in the task context.

Source Code Analysis

A value that is set once and is then available for reading for a bounded period of execution by a thread. A ScopedValue allows for safely and efficiently sharing data for a bounded period of execution without passing the data as method arguments. ScopedValue defines the where(ScopedValue, Object, Runnable) method to set the value of a ScopedValue for the bouned period of execution by a thread of the runnable's run method. The unfolding execution of the methods executed by run defines a dynamic scope. The scoped value is bound while executing in the dynamic scope, it reverts to being unbound when the run method completes (normally or with an exception). Code executing in the dynamic scope uses the ScopedValue get method to read its value. Like a thread-local variable, a scoped value has multiple incarnations, one per thread. The particular incarnation that is used depends on which thread calls its methods.

Before starting the source code analysis of ScopedValue, let's look at the Java doc introduction: ScopedValue is an object that is set once and is then available for reading for a bounded period of execution by a thread. ScopedValue allows for safely and efficiently sharing data for a bounded period of execution without passing the data as method arguments. ScopedValue defines the where(ScopedValue, Object, Runnable) method to set the value of a ScopedValue for the bounded period of execution by a thread of the runnable's run method. The unfolding execution of the methods executed by run defines a dynamic scope. The scoped value is bound while executing in the dynamic scope, it reverts to being unbound when the run method completes (normally or with an exception). Code executing in the dynamic scope uses the ScopedValue get method to read its value. Like a thread-local variable, a scoped value has multiple incarnations, one per thread. The particular incarnation that is used depends on which thread calls its methods. A typical usage of ScopedValue is to declare it in final and static fields. The accessibility of the field will determine which components can bind or read its value. ScopedValue has 3 inner classes, namely Snapshot, Carrier, and Cache, which play crucial roles in ScopedValue.

Snapshot

An immutable map from ScopedValue to values. Unless otherwise specified, passing a null argument to a constructor or method in this class will cause a NullPointerException to be thrown.

Snapshot is an immutable map from ScopedValue to values. Unless otherwise specified, passing a null argument to a constructor or method in this class will cause a NullPointerException to be thrown. The main purpose of this class is to create an immutable mapping for ScopedValue instances, so that at runtime, no matter how other code modifies the original ScopedValue instance, the values in the Snapshot will not change. It provides a safe way to share values in a multi-threaded environment.

Carrier

A mapping of scoped values, as keys, to values. A Carrier is used to accumlate mappings so that an operation (a Runnable or Callable) can be executed with all scoped values in the mapping bound to values. The following example runs an operation with k1 bound (or rebound) to v1, and k2 bound (or rebound) to v2. ScopedValue.where(k1, v1).where(k2, v2).run(() -> ... ); A Carrier is immutable and thread-safe. The where method returns a new Carrier object, it does not mutate an existing mapping. Unless otherwise specified, passing a null argument to a method in this class will cause a NullPointerException to be thrown.

The Carrier class is used to accumulate mappings so that an operation (Runnable or Callable) can be executed with all scoped values in the mapping bound to values. Carrier is immutable and thread-safe. The where method returns a new Carrier object and does not change the existing mapping. This is a tool for creating and maintaining mapping relationships between ScopedValue instances and corresponding values, so that these mapping relationships can be applied together when executing operations.

Cache

A small fixed-size key-value cache. When a scoped value's get() method is invoked, we record the result of the lookup in this per-thread cache for fast access in future.

Cache is a small fixed-size key-value cache. When a scoped value's get() method is called, we record the result of the lookup in this per-thread cache for fast access in the future. The main purpose of this class is to optimize performance. By caching the results of the get() method, repeated lookup operations can be avoided when obtaining the value of the same ScopedValue multiple times. The cache only needs to be updated when the value of the ScopedValue is changed.

where()

The where() method is the core method and entry point of the ScopedValue class. It receives three parameters. When the operation completes (normally or with an exception), the ScopedValue will revert to an unbound state in the current thread, or revert to the previous value when it was previously bound.

graph TB
    A("ScopedValue.where(key, value, op)")
    A --> B("ScopedValue.Carrier.of(key, value)")
    B --> C("ScopedValue.Carrier.where(key, value, prev)")
    C --> D("Return ScopedValue.Carrier object")

Scoped values are intended to be used in a structured manner. If op has already created a StructuredTaskScope but has not closed it, exiting op will cause each StructuredTaskScope created in the dynamic scope to be closed. This may require blocking until all child threads have completed their subtasks. Closing is done in the reverse order of their creation.

Using ScopedValue.where(key, value, op); is equivalent to using ScopedValue.where(key, value).call(op);

 public static <T, R> R where(ScopedValue<T> key,
         T value,
         Callable<? extends R> op) throws Exception {
  return where(key, value).call(op);
 }

This method delegates the first two parameters to the Carrier.of(key, value); method

 /*  
  * Return a new collection consisting of a single binding
  */  
static <T> Carrier of(ScopedValue<T> key, T value) {  
return where(key, value, null);  
 }
/**  
  * Add a binding to this map, returning a new Carrier instance
  */
private static final <T> Carrier where(ScopedValue<T> key, T value,  
             Carrier prev) {  
return new Carrier(key, value, prev);  
 }

In the Carrier class, the where method returns a new Carrier object, which is a Chain of Responsibility design pattern.

call()

The where method mainly constructs the Carrier object, and then these are all delegated to the subsequent call method in Carrier to implement a call to the Callable. The call relationship is as follows:

graph TB
 D("ScopedValue.Carrier")
    D --> E("ScopedValue.Carrier.call(op)")
    E -->|Branch 1| F("ScopedValue.Cache.invalidate()")
    E -->|Branch 2| G("ScopedValue.Carrier.runWith(newSnapshot, op)")
    G --> H("ScopedValueContainer.call(op)")
    H --> I("ScopedValueContainer.callWithoutScope(op)")
    I --> J("Callable.call()")

There are many details in the method call chain of the call method regarding handling Snapshot and Cache. These contents may change in future Java versions, so they will not be elaborated here.

Summary

Both ThreadLocal and ScopedValue play crucial roles in Java concurrent programming. They are suitable for different scenarios, and developers need to choose based on specific requirements. ThreadLocal is mainly used for general concurrent programming. In Java, each thread has its own stack, which stores the local variables needed by this thread. ThreadLocal provides a unique mechanism so that each thread can have its own independent copy of data, which other threads cannot access. This mechanism is very suitable for scenarios in concurrent programming that require isolating thread state or achieving data isolation between threads, such as database connections, Session sessions, etc.

However, although ThreadLocal can achieve data isolation at the thread level, it cannot solve more complex concurrency problems by itself, such as concurrency control of asynchronous tasks, data sharing between asynchronous tasks, etc. This requires a new tool to solve, namely ScopedValue.

ScopedValue is a new feature introduced in Java, designed to support structured concurrent programming. Structured concurrency allows developers to manage the lifecycle of concurrent programs by defining the structure of concurrency. ScopedValue provides a method so that a value can be shared by concurrent tasks within a defined execution scope (i.e., a "scope").

In structured concurrent programming, ScopedValue is mainly used to achieve data sharing between concurrent tasks. Compared with ThreadLocal, ScopedValue can better control data sharing between concurrent tasks and also better manage the lifecycle of concurrent tasks. For example, a thread can put a value into ScopedValue, and then all child threads started by that thread can access this value. This can avoid passing a large number of parameters in asynchronous concurrent tasks, simplifying concurrent programming.