String Templates (Preview)

This document describes changes to the Java Language Specification to support String Templates, a preview feature of Java SE 21. See JEP 430 for an overview of the feature.

Changes are described with respect to existing sections of the JLS. New text is indicated like this and deleted text is indicated like this. Explanation and discussion, as needed, is set aside in grey boxes.

Chapter 2: Grammars

2.1 Context-Free Grammars

A context-free grammar consists of a number of productions. Each production has an abstract symbol called a nonterminal as its left-hand side, and a sequence of one or more nonterminal and terminal symbols as its right-hand side. For each grammar, the terminal symbols are drawn from a specified alphabet.

Starting from a sentence consisting of a single distinguished nonterminal, called the goal symbol, a given context-free grammar specifies a language, namely, the set of possible sequences of terminal symbols that can result from repeatedly replacing any nonterminal in the sequence with a right-hand side of a production for which the nonterminal is the left-hand side.

2.2 The Lexical Grammar

A lexical grammar for the Java programming language is given in 3. This grammar has as its terminal symbols the characters of the Unicode character set. It defines a set of productions, starting from the goal symbol Input (3.5), that describe how sequences of Unicode characters (3.1) are translated into a sequence of input elements (3.2).

These input elements, with white space (3.6) and comments (3.7) discarded, form the terminal symbols for the syntactic grammar for the Java programming language and are called tokens (3.5). These tokens are include the identifiers (3.8), keywords (3.9), literals (3.10), separators (3.11), and operators (3.12) of the Java programming language.

2.3 The Syntactic Grammar

The syntactic grammar for the Java programming language is given in Chapters 4, 6-10, 14, and 15. This grammar has as its terminal symbols the tokens defined by the lexical grammar. It defines a set of productions, starting from the goal symbol CompilationUnit (7.3), that describe how sequences of tokens can form syntactically correct programs.

For convenience, the syntactic grammar is presented all together in Chapter 19.

Chapter 3: Lexical Structure

This chapter specifies the lexical structure of the Java programming language.

Programs are written in Unicode (3.1), but lexical translations are provided (3.2) so that Unicode escapes (3.3) can be used to include any Unicode character using only ASCII characters. Line terminators are defined (3.4) to support the different conventions of existing host systems while maintaining consistent line numbers.

The Unicode characters resulting from the lexical translations are reduced to a sequence of input elements (3.5), which are white space (3.6), comments (3.7), and tokens. The tokens are the identifiers (3.8), keywords (3.9), literals (3.10), separators (3.11), and operators (3.12), and fragments (3.13) of the syntactic grammar.

3.1 Unicode

Programs are written using the Unicode character set (1.7). Information about this character set and its associated character encodings may be found at <https://www.unicode.org/>.

The Java SE Platform tracks the Unicode Standard as it evolves. The precise version of Unicode used by a given release is specified in the documentation of the class Character.

The Unicode standard was originally designed as a fixed-width 16-bit character encoding. It has since been changed to allow for characters whose representation requires more than 16 bits. The range of legal code points is now U+0000 to U+10FFFF, using the hexadecimal U+n notation. Characters whose code points are greater than U+FFFF are called supplementary characters. To represent the complete range of characters using only 16-bit units, the Unicode standard defines an encoding called UTF-16. In this encoding, supplementary characters are represented as pairs of 16-bit code units, the first from the high-surrogates range (U+D800 to U+DBFF), and the second from the low-surrogates range (U+DC00 to U+DFFF). For characters in the range U+0000 to U+FFFF, the values of code points and UTF-16 code units are the same.

The Java programming language represents text in sequences of 16-bit code units, using the UTF-16 encoding.

Some APIs of the Java SE Platform, primarily in the Character class, use 32-bit integers to represent code points as individual entities. The Java SE Platform provides methods to convert between 16-bit and 32-bit representations.

This specification uses the terms code point and UTF-16 code unit where the representation is relevant, and the generic term character where the representation is irrelevant to the discussion.

Except for comments (3.7), identifiers (3.8), and the contents of character literals, string literals, and text blocks, and templates (3.10.4, 3.10.5, 3.10.6, 3.13), all input elements (3.5) in a program are formed only from ASCII characters (or Unicode escapes (3.3) which result in ASCII characters).

ASCII (ANSI X3.4) is the American Standard Code for Information Interchange. The first 128 characters of the Unicode UTF-16 encoding are the ASCII characters.

3.5 Input Elements and Tokens

The input characters and line terminators that result from Unicode escape processing (3.3) and then input line recognition (3.4) are reduced to a sequence of input elements.

Input:

{InputElement} [Sub]

InputElement:

WhiteSpace

Comment

Token

Token:

Identifier

Keyword

Literal

Separator

Operator

Fragment

Sub:

the ASCII SUB character, also known as "control-Z"

Those input elements that are not white space or comments are tokens. The tokens are the terminal symbols of the syntactic grammar (2.3).

White space (3.6) and comments (3.7) can serve to separate tokens that, if adjacent, might be tokenized in another manner.

For example, the input characters - and = can form the operator token -= (3.12) only if there is no intervening white space or comment. As another example, the ten input characters staticvoid form a single identifier token while the eleven input characters static void (with an ASCII SP character between c and v) form a pair of keyword tokens, static and void, separated by white space.

As a special concession for compatibility with certain operating systems, the ASCII SUB character (\u001a, or control-Z) is ignored if it is the last character in the escaped input stream.

The Input production is ambiguous, meaning that for some sequences of input characters, there is more than one way to reduce the input characters to input elements (that is, to tokenize the input characters). Ambiguities are resolved as follows:

• A sequence of input characters that could be reduced to either an identifier token or a literal token is always reduced to a literal token.

• A sequence of input characters that could be reduced to either an identifier token or a reserved keyword token (3.9) is always reduced to a reserved keyword token.

• A sequence of input characters that could be reduced to either a contextual keyword token or to other (non-keyword) tokens is reduced according to context, as specified in 3.9.

• If the input character > appears in a type context (4.11), that is, as part of a Type or an UnannType in the syntactic grammar (4.1, 8.3), it is always reduced to the numerical comparison operator >, even when it could be combined with an adjacent > character to form a different operator.

  Without this rule for > characters, two consecutive > brackets in a type such as List<List<String>> would be tokenized as the signed right shift operator >>, while three consecutive > brackets in a type such as List<List<List<String>>> would be tokenized as the unsigned right shift operator >>>. Worse, the tokenization of four or more consecutive > brackets in a type such as List<List<List<List<String>>>> would be ambiguous, as various combinations of >, >>, and >>> tokens could represent the >>>> characters.

Consider two tokens x and y in the resulting input stream. If x precedes y, then we say that x is to the left of y and that y is to the right of x.

For example, in this simple piece of code:

class Empty {
}

we say that the } token is to the right of the { token, even though it appears, in this two-dimensional representation, downward and to the left of the { token. This convention about the use of the words left and right allows us to speak, for example, of the right-hand operand of a binary operator or of the left-hand side of an assignment.

3.10 Literals

3.10.7 Escape Sequences

In character literals, string literals, and text blocks, and fragments of a template (3.10.4, 3.10.5, 3.10.6, 3.13), the escape sequences allow for the representation of some nongraphic characters without using Unicode escapes (3.3), as well as the single quote, double quote, and backslash characters.

EscapeSequence:

\ b * (backspace BS, Unicode \u0008)*

\ s * (space SP, Unicode \u0020)*

\ t * (horizontal tab HT, Unicode \u0009)*

\ n * (linefeed LF, Unicode \u000a)*

\ f * (form feed FF, Unicode \u000c)*

\ r * (carriage return CR, Unicode \u000d)*

\ LineTerminator * (line continuation, no Unicode representation)*

\ " * (double quote ", Unicode \u0022)*

\ ' * (single quote ', Unicode \u0027)*

\ \ * (backslash \, Unicode \u005c)*

OctalEscape * (octal value, Unicode \u0000 to \u00ff)*

OctalEscape:

\ OctalDigit

\ OctalDigit OctalDigit

\ ZeroToThree OctalDigit OctalDigit #

OctalDigit:

(one of)

0 1 2 3 4 5 6 7

ZeroToThree:

(one of)

0 1 2 3

The OctalDigit production above comes from 3.10.1. Octal escapes are provided for compatibility with C, but can express only Unicode values \u0000 through \u00FF, so Unicode escapes are usually preferred.

It is a compile-time error if the character following a backslash in an escape sequence is not a LineTerminator or an ASCII b, s, t, n, f, r, ", ', \, 0, 1, 2, 3, 4, 5, 6, or 7.

An escape sequence in the content of a character literal, string literal, or text block, or fragment of a template is interpreted by replacing its \ and trailing character(s) with the single character denoted by the Unicode escape in the EscapeSequence grammar. The line continuation escape sequence has no corresponding Unicode escape, so is interpreted by replacing it with nothing.

The line continuation escape sequence can appear in a text block, but cannot appear in a character literal or a string literal because each disallows a LineTerminator.

3.13 Fragments

Chapter 7: Packages and Modules

7.3 Compilation Units

CompilationUnit is the goal symbol (2.1) for the syntactic grammar (2.3) of Java programs. It is defined by the following production:

CompilationUnit:

OrdinaryCompilationUnit

ModularCompilationUnit

OrdinaryCompilationUnit:

[PackageDeclaration] {ImportDeclaration} {TopLevelClassOrInterfaceDeclaration}

ModularCompilationUnit:

{ImportDeclaration} ModuleDeclaration

An ordinary compilation unit consists of three parts, each of which is optional:

• A package declaration (7.4), giving the fully qualified name (6.7) of the package to which the compilation unit belongs.

  A compilation unit that has no package declaration is part of an unnamed package (7.4.2).

• import declarations (7.5) that allow classes and interface from other packages, and static members of classes and interfaces, to be referred to using their simple names.

• Top level declarations of classes and interfaces (7.6).

A modular compilation unit consists of a module declaration (7.7), optionally preceded by import declarations. The import declarations allow classes and interfaces from packages in this module and other modules, as well as static members of classes and interfaces, to be referred to using their simple names within the module declaration.

Every compilation unit implicitly imports the following:

1. Every public class or interface declared in the predefined package java.lang, as if the declaration import java.lang.*; appeared at the beginning of each compilation unit immediately after any package declaration.

As a result, the names of all those implicitly imported classes and interfaces classes, interfaces and static fields are available as simple names in every compilation unit.

The host system determines which compilation units are observable, except for the compilation units in the predefined package java and its subpackages lang and io, which are all always observable.

7.5 Import Declarations

7.5.3 Single-Static-Import Declarations

A single-static-import declaration imports all accessible static members with a given simple name from a class or interface. This makes these static members available under their simple name in the module, class, and interface declarations of the compilation unit in which the single-static-import declaration appears.

SingleStaticImportDeclaration:

import static TypeName . Identifier ;

The TypeName must be the canonical name (6.7) of a class or interface.

The class or interface must be either a member of a named package, or a member of a class or interface whose outermost lexically enclosing class or interface declaration (8.1.3) is a member of a named package, or a compile-time error occurs.

It is a compile-time error if the named class or interface is not accessible (6.6).

The Identifier must name at least one static member of the named class or interface. It is a compile-time error if there is no static member of that name, or if all of the named members are not accessible.

It is permissible for one single-static-import declaration to import several fields, classes, or interfaces with the same name, or several methods with the same name and signature. This occurs when the named class or interface inherits multiple fields, member classes, member interfaces, or methods, all with the same name, from its own supertypes.

If two single-static-import declarations in the same compilation unit attempt to import classes or interface with the same simple name, then a compile-time error occurs, unless the two classes or interfaces are the same, in which case the duplicate declaration is ignored.

If a single-static-import declaration imports a class or interface whose simple name is x, and the compilation unit also declares a top level class or interface (7.6) whose simple name is x, a compile-time error occurs.

If a compilation unit contains both a single-static-import declaration that imports a class or interface whose simple name is x, and a single-type-import declaration (7.5.1) that imports a class or interface whose simple name is x, a compile-time error occurs, unless the two classes or interfaces are the same, in which case the duplicate declaration is ignored.

7.5.4 Static-Import-on-Demand Declarations

A static-import-on-demand declaration allows all accessible static members of a named class or interface to be imported as needed.

StaticImportOnDemandDeclaration:

import static TypeName . * ;

The TypeName must be the canonical name (6.7) of a class or interface.

The class or interface must be either a member of a named package, or a member of a class or interface whose outermost lexically enclosing class or interface declaration (8.1.3) is a member of a named package, or a compile-time error occurs.

It is a compile-time error if the named class or interface is not accessible (6.6).

Two or more static-import-on-demand declarations in the same compilation unit may name the same class or interface; the effect is as if there was exactly one such declaration.

Chapter 12: Execution

12.5 Creation of New Class Instances

A new class instance is explicitly created when evaluation of a class instance creation expression (15.9) causes a class to be instantiated.

A new class instance may be implicitly created in the following situations:

• Loading of a class or interface that contains a string literal (3.10.5) or a text block (3.10.6) may create a new String object to denote the string represented by the string literal or text block. (This object creation will not occur if an instance of String denoting the same sequence of Unicode code points as the string represented by the string literal or text block has previously been interned.)

• Execution of an operation that causes boxing conversion (5.1.7). Boxing conversion may create a new object of a wrapper class (Boolean, Byte, Short, Character, Integer, Long, Float, Double) associated with one of the primitive types.

• Execution of a string concatenation operator + (15.18.1) that is not part of a constant expression (15.29) always creates a new String object to represent the result. String concatenation operators may also create temporary wrapper objects for a value of a primitive type.

• Evaluation of a method reference expression (15.13.3) or a lambda expression (15.27.4) may require that a new instance be created of a class that implements a functional interface type (9.8).

Each of these situations identifies a particular constructor (8.8) to be called with specified arguments (possibly none) as part of the class instance creation process.

Whenever a new class instance is created, memory space is allocated for it with room for all the instance variables declared in the class and all the instance variables declared in each superclass of the class, including all the instance variables that may be hidden (8.3).

If there is not sufficient space available to allocate memory for the object, then creation of the class instance completes abruptly with an OutOfMemoryError. Otherwise, all the instance variables in the new object, including those declared in superclasses, are initialized to their default values (4.12.5).

Just before a reference to the newly created object is returned as the result, the indicated constructor is processed to initialize the new object using the following procedure:

1. Assign the arguments for the constructor to newly created parameter variables for this constructor invocation.

2. If this constructor begins with an explicit constructor invocation (8.8.7.1) of another constructor in the same class (using this), then evaluate the arguments and process that constructor invocation recursively using these same five steps. If that constructor invocation completes abruptly, then this procedure completes abruptly for the same reason; otherwise, continue with step 5.

3. This constructor does not begin with an explicit constructor invocation of another constructor in the same class (using this). If this constructor is for a class other than Object, then this constructor will begin with an explicit or implicit invocation of a superclass constructor (using super). Evaluate the arguments and process that superclass constructor invocation recursively using these same five steps. If that constructor invocation completes abruptly, then this procedure completes abruptly for the same reason. Otherwise, continue with step 4.

4. Execute the instance initializers and instance variable initializers for this class, assigning the values of instance variable initializers to the corresponding instance variables, in the left-to-right order in which they appear textually in the source code for the class. If execution of any of these initializers results in an exception, then no further initializers are processed and this procedure completes abruptly with that same exception. Otherwise, continue with step 5.

5. Execute the rest of the body of this constructor. If that execution completes abruptly, then this procedure completes abruptly for the same reason. Otherwise, this procedure completes normally.

Unlike C++, the Java programming language does not specify altered rules for method dispatch during the creation of a new class instance. If methods are invoked that are overridden in subclasses in the object being initialized, then these overriding methods are used, even before the new object is completely initialized.

class Point {
    int x, y;
    Point() { x = 1; y = 1; }
}
class ColoredPoint extends Point {
    int color = 0xFF00FF;
}
class Test {
    public static void main(String[] args) {
        ColoredPoint cp = new ColoredPoint();
        System.out.println(cp.color);
    }
}

ColoredPoint() { super(); }

Point() { super(); x = 1; y = 1; }

Object() { }

class Super {
    Super() { printThree(); }
    void printThree() { System.out.println("three"); }
}
class Test extends Super {
    int three = (int)Math.PI;  // That is, 3
    void printThree() { System.out.println(three); }

    public static void main(String[] args) {
        Test t = new Test();
        t.printThree();
    }
}

0
3

Chapter 15: Expressions

15.8 Primary Expressions

Primary expressions include most of the simplest kinds of expressions, from which all others are constructed: literals, object creations, field accesses, method invocations, method references, and array accesses, and template expressions. A parenthesized expression is also treated syntactically as a primary expression.

Primary:

PrimaryNoNewArray

ArrayCreationExpression

PrimaryNoNewArray:

Literal

ClassLiteral

this

TypeName . this

( Expression )

ClassInstanceCreationExpression

FieldAccess

ArrayAccess

MethodInvocation

MethodReference

TemplateExpression

This part of the grammar of the Java programming language is unusual, in two ways. First, one might expect simple names, such as names of local variables and method parameters, to be primary expressions. For technical reasons, names are grouped together with primary expressions a little later when postfix expressions are introduced (15.14).

The technical reasons have to do with allowing left-to-right parsing of Java programs with only one-token lookahead. Consider the expressions (z[3]) and (z[]). The first is a parenthesized array access (15.10.3) and the second is the start of a cast (15.16). At the point that the look-ahead symbol is [, a left-to-right parse will have reduced the z to the nonterminal Name. In the context of a cast we prefer not to have to reduce the name to a Primary, but if Name were one of the alternatives for Primary, then we could not tell whether to do the reduction (that is, we could not determine whether the current situation would turn out to be a parenthesized array access or a cast) without looking ahead two tokens, to the token following the [. The grammar presented here avoids the problem by keeping Name and Primary separate and allowing either in certain other syntax rules (those for ClassInstanceCreationExpression, MethodInvocation, ArrayAccess, and PostfixExpression, though not FieldAccess because it uses an identifier directly). This strategy effectively defers the question of whether a Name should be treated as a Primary until more context can be examined.

The second unusual feature avoids a potential grammatical ambiguity in the expression "new int[3][3]" which in Java always means a single creation of a multidimensional array, but which, without appropriate grammatical finesse, might also be interpreted as meaning the same as "(new int[3])[3]".

This ambiguity is eliminated by splitting the expected definition of Primary into Primary and PrimaryNoNewArray. (This may be compared to the splitting of Statement into Statement and StatementNoShortIf (14.5) to avoid the "dangling else" problem.)

15.8.1 Lexical Literals

A literal (3.10) denotes a fixed, unchanging value.

The following production from 3.10 is shown here for convenience:

Literal:

IntegerLiteral

FloatingPointLiteral

BooleanLiteral

CharacterLiteral

StringLiteral

TextBlock

NullLiteral

The type of a literal is determined as follows:

• The type of an integer literal (3.10.1) that ends with L or l (ell) is long (4.2.1).

  The type of any other integer literal is int (4.2.1).

• The type of a floating-point literal (3.10.2) that ends with F or f is float (4.2.3).

  The type of any other floating-point literal is double (4.2.3).

• The type of a boolean literal (3.10.3) is boolean (4.2.5).

• The type of a character literal (3.10.4) is char (4.2.1).

• The type of a string literal (3.10.5) or a text block (3.10.6) is String (4.3.3).

• The type of the null literal null (3.10.8) is the null type (4.1); its value is the null reference.

Evaluation of a lexical literal always completes normally.

15.8.6 Template Expressions

// A string template with simple embedded string variables
String firstName = "Joan";
String lastName  = "Smith";
String fullName  = STR."\{firstName} \{lastName}";

// A string template with embedded integer expressions
int x = 10, y = 20;
String s1 = STR."\{x} + \{y} = \{x + y}";

// Embedded expressions can invoke methods and access fields
String s2 = STR."You have a \{getOfferType()} waiting for you!";
String s3 = STR."Access at \{req.date} \{req.time} from \{req.ipAddress}";

// A text block template modeling an HTML element with
// embedded expressions
String title = "My Web Page";
String text  = "Hello, world";
String html = STR."""
        <html>
          <head>
            <title>\{title}</title>
          </head>
          <body>
            <p>\{text}</p>
          </body>
        </html>
        """;

// A text block template modeling a JSON value with
// embedded expressions
String name    = "Joan Smith";
String phone   = "555-123-4567";
String address = "1 Maple Drive, Anytown";
String json = STR."""
    {
        "name":    "\{name}",
        "phone":   "\{phone}",
        "address": "\{address}"
    }
    """;

StringTemplate.Processor<String, RuntimeException> UPPER = (StringTemplate st) ->
    st.interpolate().toUpperCase();

String name = "Joan";
String result = UPPER."My name is \{name}";

StringTemplate.Processor<String, RuntimeException> MY_UPPER_STRINGS = (StringTemplate st) -> {
    List<String> fragments = st.fragments()
        .stream()
        .map(String::toUpperCase)
        .toList();
    List<Object> values = st.values();
    return StringTemplate.interpolate(fragments, values);
};

String name = "Joan";
String result = MY_UPPER_STRINGS."My name is \{name}";

StringTemplate.Processor<String, RuntimeException> MY_UPPER_VALUES = (StringTemplate st) -> {
    List<String> values = st.values()
        .stream()
        .map((o) -> (o==null)?"":o.toString().toUpperCase())
        .toList();
    return StringTemplate.interpolate(st.fragments(), values);
};

String title = null;
String firstName = "Joan";
String familyName = "Smith";
String result = MY_UPPER_VALUES."Welcome \{title}\{firstName} \{familyName}";

StringTemplate.Processor<JSONObject, RuntimeException> JSON =
    (StringTemplate st) -> new JSONObject(st.interpolate());

String name    = "Joan Smith";
String phone   = "555-123-4567";
String address = "1 Maple Drive, Anytown";

JSONObject doc = JSON."""
    {
        "name":    "\{name}",
        "phone":   "\{phone}",
        "address": "\{address}"
    }
    """;

StringTemplate.Processor<JSONObject, JSONException> JSON_VALIDATE = (StringTemplate st) -> {
    String stripped = st.interpolate().strip();
    if (!stripped.startsWith("{") || !stripped.endsWith("}")) {
        throws new JSONException("Missing brace");
    }
    return new JSONObject(stripped);
};

String name    = "Joan Smith";
String phone   = "555-123-4567";
String address = "1 Maple Drive, Anytown";
try {
    JSONObject doc = JSON_VALIDATE."""
        {
            "name":    "\{name}",
            "phone":   "\{phone}",
            "address": "\{address}"
        }
        """;
} catch (JSONException ex) {
    ...
}