The line Filter

Constituent Repetition

Constituents of a line filter may optionally be followed by a quantifier which modifies the number of times the constituent must match (by default each constituent must match once). The quantifiers that may be used in line constituents are shown in the table below.

For example, to find sequences that start with a check, followed by any number of captures (including zero), followed by mate, the * quantifier may be added to the move filter constituent:

line --> check
     --> move previous capture . *
     --> mate

Constituent quantifiers are always greedy meaning they match as many times as possible, but not at the expense of the line filter failing. For example, the query:

line --> move previous capture . +

will match the longest sequence of captures starting at the current position and the query:

line --> move previous capture . + --> mate

will match a series of captures followed by mate even if the move that results in mate is a capture (the move constituent will not consume the final position, allowing the mate constituent to match instead so that the line filter can match).

If a * or + token appearing in a constituent could be interpreted as multiplication or addition, it will be. To force the token to be interpreted as a repetition quantifier it may be enclosed in braces, e.g. {?}, {*}, and {+}. In addition to being unambiguous, enclosing repetition quantifiers in braces help them stand out visually.

The m and n appearing in counted repetition quantifiers must be non-negative Numeric literals not exceeding 9999 and m, if present, must not be greater than n. Counted repetitions are expanded internally and nested counted repetitions (see Constituent Grouping below) multiply, so the combined size of the expanded pattern is additionally limited to 100000 constituents. Both limits are diagnosed at parse time.

Constituent Grouping

Multiple line constituents may be grouped by enclosing the constituents in parentheses and repetition may then be applied to the group as a whole. For example, the following query will find games that end in checkmate with at least three check-then-capture sequences immediately preceding mate:

line --> (check --> move previous capture (flipcolor A attacks k)) {3,}
     --> mate

Constituent groups may be nested and separate repetition for constituents within a group may be specified. Note that each instance of <-- or --> represents the next position in the specified direction, even within a grouping, e.g. the above query will not match sequences of less than seven positions.

An example of a game matching the above query is here. The diagrams below show the critical positions.

Auxiliary Comments

Unless the quiet parameter is specified, the selected best matching sequence (see Multiple Matching Sequences) found by a matching line filter at the current position will be annotated with auxiliary comments, as will every other sequence tied for best when keepallbest is used. The matching position at the start of this sequence will be annotated with the comment:

Start line that ends at move end-position

and the position that ends the sequence will be annotated with the comment:

End line of length sequence-length that starts at move start-position

where sequence-length is a positive non-zero numeric value that represents the length of the annotated sequence and start-position and end-position are the textual portrayals (as described in String Portrayal of Types) of the positions that start and end the sequence, respectively.

Multiple Matching Sequences

When variations are processed, it is possible for multiple sequences of the same length to be found at the current position. When the max parameter is used, multiple sequences with the same max expression value can be found even when variations are not processed, since equally-valued sequences can end at different positions of the same line. When this happens, a single sequence is selected: the one which contains auxiliary comments, comments retained by Smart Comments, the final position returned when using the lastposition parameter, and the sequence that determines the final state of modified variables. Without max, among sequences tied for best length, the selected sequence is the one that ends with the position with the lowest position ID. With max, the sequence with the greatest max expression value is selected; ties are resolved by preferring the longer sequence and then the one that ends with the position with the lowest position ID. If the keepallbest parameter or the --keepallbest option is used, comments (both user-comments and auxiliary comments) are preserved for all sequences tied for best (with max, all sequences with the greatest max expression value, even when their lengths differ, subject to the range restrictions described in The max Parameter) but the selected sequence, the position returned when lastposition is used, and the final state of modified variables are the same as when this option is not used.

When multiple constituents in a line filter contain repetition quantifiers, it is possible for a matching sequence to have several ways to match, even outside of variations. For example, consider the query:

initial
line --> comment "A" * --> comment "B" *

applied at the beginning of a game that contains two positions. Since repetition is greedy, we know that both positions will be consumed by this line filter but how many times will each constituent match? Since the * quantifier can successfully match zero times, the possible outcomes are that the first constituent matches zero times and the second one matches two times, both constituents match once, or the first constituent matches both times. In most cases it won’t make a difference but it can when the constituents have side-effects, such as modifying a variable or adding a comment. When the line filter has multiples ways to match a sequence of the same length, it is unspecified how the positions in the sequence will be correlated to the corresponding constituents. In other words, the result of the above query could be that both positions are commented with “A”, both are commented with “B”, or the first position is commented with “A” and the second with “B”.

The firstmatch Parameter

When the firstmatch parameter is specified, CQLi will find the shortest sequence that matches the constituent filters instead of the longest. Despite the name, this is not necessarily the “first” successful match that is encountered. This parameter was used in CQL 6 to partially mitigate situations in which variables modified in line constituents could have inconsistent states when backtracking while looking for a longer match although it doesn’t always achieve that goal. CQLi employs a more robust mechanism (Atomic Evaluation described below) to ensure variable consistency while evaluating constituents so specifying firstmatch is not necessary to accommodate such filters. Additionally, CQLi utilizes a different, non-backtracking, matching algorithm and the behavior that is closest to the CQL 6 behavior is simply to yield the shortest matching sequence.

The focus Parameter

The focus parameter is followed by an arbitrary filter, the focus expression, and restricts the positions considered by the line filter to those that are in focus (match the focus expression). Positions that do not match the focus expression are not evaluated against constituent filters and are not counted in the length of a matching sequence, but they do not interrupt a sequence: the sequence continues with the next position in the line that is in focus. Every candidate sequence considered by the line filter starts with the position in which the line filter is evaluated and that position must match the focus expression for the line filter to match.

For example, the query:

line focus wtm --> check {3,}

will find sequences in which White is checked three or more times in a row (positions with Black to move are not considered). In general, singlecolor is equivalent to a focus expression that requires the side to move to match the side to move of the starting position; the focus parameter generalizes this restriction to arbitrary criteria. When both singlecolor and focus are specified, only positions satisfying both restrictions are considered.

The focus expression is evaluated at each position visited by the line filter, before any constituent, in the variable state that was in effect when evaluation of the line filter began. This is guaranteed by Atomic Evaluation; if the nonatomic parameter is specified, variable modifications made by constituents are applied immediately and later focus evaluations will see the modified variable state. Comments emitted while evaluating the focus expression are not retained, even when smart comments are disabled, and neither are the values recorded by sort filters within it; a warning is issued for each comment or sort filter within the focus expression. To comment or sort on the in-focus positions of a sequence, place the filter in a constituent, which is evaluated at the same positions and considers the same moves.

Move Linearization applies within the focus expression: an ordinary move filter in the focus expression considers only the move by which the line being processed leaves the position, which is the same move considered by move filters in the constituents evaluated at that position. As a result, when variations are processed, whether a position is in focus can depend on the line being processed. A position may be in focus along one recorded move and out of focus along another, in which case it is skipped only in the lines in which it is out of focus. When the focus expression contains an ordinary move filter, it is evaluated once for each move by which the line may continue from the position, so it may be evaluated multiple times at the same position. For this reason, and because the results of modifying variables in a focus expression would depend on the number and order of these evaluations, modifying variables within a focus expression is not recommended and the resulting variable state is unspecified.

For example, the query:

initial
line focus move from N --> true {3,}

finds lines in which White’s knights move three or more times, considering only the positions from which a White knight moves and skipping every other position. Given the game:

1.Nf3 d5 2.e4 (2.Nc3) 2...dxe4 3.Ng5 e5 4.Nxe4 *

the position after 1...d5 is out of focus in the main line, where 2.e4 was played, and in focus in the variation, where 2.Nc3 was played. The sequence in the main line skips that position and consists of the positions from which 1.Nf3, 3.Ng5, and 4.Nxe4 were played, so it has length 3. The sequence through the variation consists of the positions from which 1.Nf3 and 2.Nc3 were played and has length 2. If the move filter in the focus expression were not linearized, the position after 1...d5 would be in focus in both lines because a knight move was recorded there, and it would be counted in the main line sequence even though no knight moved from it in that line. Note that an in-focus position is the position from which the qualifying move was played: the main line sequence above ends at the position before 4.Nxe4, which is the position yielded by lastposition and the position that receives the end comment.

The following details of linearization within the focus expression are also worth noting:

• The position in which the line filter is evaluated must be in focus, so only the lines that leave it by a move along which it is in focus are considered. For example, in the game 1.e4 (1.Nf3) 1...e5 2.Nc3 *, the filter initial line focus move from N --> true+ considers only the line that starts with 1.Nf3.
• A position from which the line cannot continue, such as the final position of a game or variation, has no move to linearize against. The focus expression is evaluated once at such a position and move filters within it behave as though no move was recorded there (see Move Linearization). For example, the final position of a game is never in focus for focus move from N, so a sequence of in-focus positions found by such a filter never includes it. With secondary, a position without a secondary move is also one from which the line cannot continue, and the primary move from it is not considered: in the game 1.e4 e5 *, the filter initial line secondary focus move from P --> true does not match because the primary move 1.e4 is not seen by the focus expression.
• For backwards-looking line filters, move filters in the focus expression consider the move on the path of the line and are unrestricted at the current position, exactly as described for constituents in Move Linearization. The focus expression is evaluated once at each position of a backwards-looking line.
• The primary and secondary parameters restrict the moves considered by the focus expression in the same way that they restrict the lines processed. For example, with primary, only the primary move from each position is considered.
• The nolinearize parameter disables linearization within the focus expression as well as within the constituents. When it is specified, the focus expression is evaluated once at each position and move filters within it consider every recorded move.
• Apart from the effect of secondary described above, linearization only affects games with variations, and only when variations are processed. In a game without variations, the focus expression is evaluated once at each position and, unless secondary is specified, yields the same result whether or not it is linearized. With secondary, linearization causes the primary move to be ignored even in a game without variations: in the game 1.e4 e5 *, the filter initial line secondary focus move from P --> true described above matches only when nolinearize is also specified.

When focus is used with nestban, only the in-focus positions of a matching sequence are banned from starting a later sequence. A position that a matching sequence skips is not banned, even if it is in focus along a different line, so it may start a later sequence along that line. When focus is used with firstmatch, the shortest sequence is determined by the number of in-focus positions.

The focus parameter is a CQLi extension.

The keepallbest Parameter

When multiple matching sequences are tied for best (see Multiple Matching Sequences), only one of them is normally selected: auxiliary comments are added only along the selected sequence and only its comments are retained by Smart Comments. When the keepallbest parameter is specified, every sequence tied for best is retained instead: begin and end comments are added for each retained sequence, explicit comments emitted along each retained sequence (including those emitted by the max expression at its end) are preserved, annotations requested with the verbose and piecepath parameters are applied to each retained sequence, and, when nestban is used, the positions of every retained sequence are banned from starting a later sequence. At most one sequence is retained for each ending position. The same sequence can be found more than once, for example when its last constituent contains an ordinary move filter that matches along several of the moves recorded at its ending position; only the first one found, which is also the one selected without keepallbest, is retained along with its comments. Without the max parameter, sequences are tied for best only when they have the same length, and tied sequences with different ending positions can only occur within variations, so keepallbest has no effect when variations are not processed. With max, sequences with the same max expression value can end at different positions of the same line, so keepallbest can retain multiple sequences even in games without variations. The value yielded by the line filter, the position returned when lastposition is used, and the final state of modified variables are unchanged: they are determined by the single selected sequence as described in Multiple Matching Sequences.

The keepallbest parameter is equivalent to the --keepallbest command line option limited to a single line filter; the option enables the same behavior for every line filter in the query and also affects the sort and consecutivemoves filters. The keepallbest parameter may not be combined with firstmatch.

The keepallbest parameter is a CQLi extension.

The lastposition Parameter

The lastposition parameter indicates that the last position of the best matching sequence found by line should be returned instead of the length of that sequence.

The max Parameter

The max parameter is followed by a numeric filter, the max expression, and changes how the best matching sequence is selected. Instead of selecting the longest matching sequence, CQLi evaluates the max expression at the last position of each candidate sequence and selects the sequence for which it yields the largest value. A candidate sequence is not considered if the max expression does not match at its last position; if the max expression does not match at the end of any candidate sequence, the line filter does not match. Among sequences with the same value, the longest sequence is preferred and remaining ties are broken as described in Multiple Matching Sequences. The value yielded by the line filter is unchanged: it is the length of the selected sequence or, when lastposition is also specified, its last position.

For example, the query:

line max power A - power a --> move capture .+ --> true

finds the sequence of consecutive captures that ends with the largest material advantage for White, even when a longer sequence of captures exists. The final true constituent matches the position reached after the last capture of the sequence, so that the max expression measures the material balance resulting from that capture rather than the balance before it; as a result, the length of the selected sequence includes the position after the final capture.

The max expression is evaluated at the last position of a candidate sequence in the variable state produced by the constituents along that sequence, so values accumulated by constituent filters are visible to it. Move Linearization does not apply within a max expression. Modifying variables within a max expression is not recommended and the resulting variable state is unspecified.

Comments emitted by the max expression are treated by Smart Comments like those emitted by the constituents of the candidate sequence for which it was evaluated: they are retained only for the selected sequence (or, with keepallbest, for each retained sequence) and are discarded for every other candidate and whenever the max expression does not yield a value. For example, the query:

line max {comment("Balance: " power A - power a) power A - power a}
     --> move capture .+ --> true

adds a comment with the material balance at the end of each selected sequence of captures; the balance comments emitted at the ends of the candidates that were not selected are not retained. Similarly, a sort filter within the max expression records only the values, and retains only the comments, produced for the retained sequences. When smart comments are disabled with the --nosmartcomments option, every comment emitted by the max expression is kept.

Because move linearization does not apply, an ordinary move filter in a max expression considers every move recorded at the last position of a candidate sequence, including moves that are not on the line being processed, and CQLi issues a warning when one is present. For example, given the game:

1.e4 (1.d4 d5 2.c4) 1...e5 2.Nf3 *

the query:

initial
line max move count --> move from [Pp]+

selects the sequence consisting of only the initial position: two moves are recorded there, while each longer candidate sequence ends at a position with a single recorded move. To rank sequences by a value that depends on the moves along the line, compute the value in a constituent, where move filters are linearized, and refer to it in the max expression:

initial
$m = 0
line max $m --> {$m = move count  move from [Pp]}+

Here every candidate sequence yields 1 and the longest sequence is selected. To consider every recorded move intentionally and silence the warning, place the move filter in the body of a with-position filter, e.g. max (currentposition: move count). The warning is not issued when the nolinearize parameter is specified.

The max parameter may not be combined with firstmatch. When a range is specified, the selected sequence must satisfy the range for the line filter to match. When max is used with a backwards-looking line filter, the max expression is evaluated at the final position of the match, which is the position closest to the start of the game. When max is used with nestban, the keepallbest parameter, or the --keepallbest option, the best sequences are those with the largest max expression value rather than those of the longest length. Such equally-best sequences can have different lengths; when a range is also specified, retained sequences whose lengths do not satisfy the range are discarded: they are not annotated, comments emitted along them are not retained, and their positions are not banned when nestban is used.

The max parameter is a CQLi extension.

The nestban Parameter

When nestban is used, all of the positions in the sequence of a matching line filter will be banned from starting a sequence in a later evaluation of the same line filter for the same game. The nestban parameter may not be used with backwards-looking line filters (ones where the constituent introducer is <--).

The nestban parameter is used to prevent subsequences of the longest matching sequence from being reported. For example, the following query will find sequences of 5 or more consecutive captures:

line nestban --> move capture . {5,}

Without the nestban parameter, all later subsequences of the initial sequence will also be found which is typically not desired.

The nolinearize Parameter

The nolinearize parameter disables Move Linearization within all constituents and the focus expression of the line filter. Linearization may be independently disabled for individual constituents as described below.

The nonatomic Parameter

By default, CQLi performs atomic evaluation of constituent filters that modify non-dictionary variables. The nonatomic parameter may be used to suppress this atomic evaluation. When atomic evaluation is suppressed, variable modifications made by constituents are applied immediately and are visible to later evaluations of the focus expression, if one was provided.

The piecepath Parameter

The piecepath parameter is followed by a set filter that designates one or more pieces to track. The set filter is evaluated once, at the position in which the line filter is evaluated, and each piece occupying a designated square is tracked through every position traversed by a matching sequence, including positions skipped because of singlecolor or focus. For each tracked piece, a comment listing the squares the piece visited, in game order, is added to the last position of each retained best matching sequence, e.g.:

{Path of Ng1: g1-f3-g5}

The piece is named by its piece letter (capitalized for White) and the square it occupied when it was designated. A piece that never moves during the sequence yields a single-square path. If a tracked piece is captured during the sequence, its path stops at the capture and is marked with (captured). When a title is provided, it is appended to the piece name, e.g. {Path of Ng1 (Greek gift): g1-f3-g5}.

Move Linearization does not apply within the set filter provided to piecepath. The set filter is evaluated once, at the current position before any constituent, in the variable state that was in effect when evaluation of the line filter began; variable modifications made by constituents do not affect which pieces are tracked, even when nonatomic is specified. Modifying variables within the set filter is not recommended and the resulting variable state is unspecified.

Comments emitted while evaluating the set filter are subject to Smart Comments: they are retained, at the position where they were emitted, only when the line filter matches, and appear once even when keepallbest retains several sequences. Similarly, a sort filter within the set filter records its value and retains its comments only when the line filter matches. When smart comments are disabled with the --nosmartcomments option, every comment emitted by the set filter is kept.

When multiple best matching sequences are retained (via the keepallbest parameter or the --keepallbest option), each retained sequence receives its own path comments covering the positions it traverses. The comments added by the piecepath parameter are not suppressed by the quiet parameter, but are suppressed by the --quiet command line option.

The piecepath parameter is a CQLi extension.

The primary and secondary Parameters

If secondary is specified, only positions that start a variation are considered after the current position. If primary is specified, only positions that do not start a variation are considered after the current position. With secondary, a line cannot continue from a position that has no secondary move; ordinary move filters evaluated at such a position do not consider its primary move (see Move Linearization).

The quiet Parameter

If the quiet parameter is provided, the begin and end comments normally added by the line filter will be suppressed for matching sequences. Comments explicitly requested with the verbose or piecepath parameters are still emitted, so quiet can be used to obtain only those annotations. To suppress all of the comments generated by the line filter, including these annotations, use the --quiet command line option.

The singlecolor Parameter

When singlecolor is specified, only positions with the same side to move as the position that starts the sequence are considered. For example, the query;

line singlecolor --> check {10,}

will find sequences where one side checks the other 10 or more times in a row.

The title Parameter

The title parameter is followed by a non-empty string literal that names the matching sequence in the automatic comments emitted by the line filter. The begin and end comments normally begin with “Start line that ends at ...” and “End line of length N that starts at ...”, the name specified by the title parameter replaces the word line in the comments, e.g. when using title "Greek gift" the comments would be “Start Greek gift that ends at ...” and “End Greek gift of length N that starts at ...”. The step comments added by the verbose parameter are also prefixed with the title, e.g. “Greek gift: position 2 of 5”, and the comments added by the piecepath parameter append the title to the piece name. A title is useful for distinguishing the comments of multiple line filters in the same query.

When the quiet parameter is also provided, the begin and end comments are suppressed, so a title has an effect only if verbose or piecepath is also specified; otherwise a warning is issued.

The title parameter is a CQLi extension.

The verbose Parameter

When the verbose parameter is specified, a step comment is added at each position matched by each retained best matching sequence, numbering the positions of the sequence in match order:

{Position 2 of 5}

The step total equals the reported length of the sequence: positions skipped because of singlecolor or focus receive no step comment and are not counted. For backwards-looking (<--) line filters, match order runs from the position in which the filter is evaluated toward the start of the game. When a title is provided, each step comment is prefixed with the title, e.g. “Greek gift: position 2 of 5”. When multiple best matching sequences are retained (via the keepallbest parameter or the --keepallbest option), each retained sequence is numbered independently.

The comments added by the verbose parameter are not suppressed by the quiet parameter, but are suppressed by the --quiet command line option.

The verbose parameter is a CQLi extension.