Comments — CQLi Reference Manual

Positions appearing in a recorded game in an input PGN file may contain comments and additional comments may be added to matching positions during the evaluation of a CQL query which will be included when the game is written to the output PGN file. Comments are the primary mechanism by which matching positions and information related to matching filters are communicated. CQLi provides several facilities to inspect comments appearing in PGN games, remove existing comments, add new comments, and control the various types of comments that are added during the operation of CQLi.

Comment Filters

The originalcomment filter provides access to comments appearing in the processed PGN file, the comment filter allows insertion of new comments, and the removecomment filter allows the removal of original comments appearing in the PGN file.

The `comment` Filter

The comment filter behaves like the str filter except that the string formed by the concatenation of its arguments is used to form a comment at the current position. The comment filter always yields a true value. Unless the –nosmartcomments option is used, Smart Comments ensures that a comment added with the comment filter will not be written to the output PGN file if the enclosing expression fails to match or the query ultimately does not match the position.

The `originalcomment` Filter

The originalcomment filter is used to inspect comments and annotations appearing in the original PGN game text.

When originalcomment is not followed by a string literal it yields a string containing the text of the original comment at the current position or None if there is no original comment. Note that the string returned by originalcomment reflects the comments appearing in the PGN text and are not affected by comment or removecomment filters that have been evaluated, e.g. the original comment(s) may still be retrieved even after a removecomment filter has been evaluated for the same position.

Multiple Comments in a Position

If the current position has multiple original comments, these will be combined into a single string, separated by newline characters, in the string returned by originalcomment. Because newlines in original comments are replaced with a space character when being parsed, and the regular expression anchoring characters ^ and $ will match the beginning or end of a line terminated with a newline, the ~~ filter may be used to check if one of multiple comments at a position consists entirely of some string. For example, if the a position contains the comments ABC and 123, the query:

originalcomment ~~ "^123$"

will match the position. Since, by default, newlines are not matched with the ‘.’ regular expression character, the query:

originalcomment ~~ "B.*2"

will not match. To search for a pattern that may span multiple consecutive comments (and thus multiple lines in the string returned by originalcomment), either explicitly include the newline character as in:

originalcomment ~~ "B(.|\n)*2"

or enable the ability of . to match a newline within a group using the ?s: syntax:

originalcomment ~~ "B(?s:.)*2"

Implicit Search with `originalcomment`

If originalcomment is immediately followed by a string literal that represents a Numeric Annotation Glyph (NAG), or a corresponding typographic annotation symbol recognized by CQLi, the filter yields true if there is a respective annotation at the position and false if there is not. If originalcomment is immediately followed by a string literal that does not represent such an annotation, the result has Boolean value and yields true if there is an original comment at the current position and the provided string appears in the text of any of the original comments at the current position and false otherwise.

NAGs and Symbolic Annotations with `originalcomment`

A NAG consists of a dollar sign ($) followed by one or more digits to form a decimal numeric value between 0 and 255. NAGs are the standard way to represent simple annotations in a PGN file. CQLi also recognizes several typographic annotation symbols that may follow a move in a PGN game, the list of supported symbols and their corresponding NAG values is provided in the table below.

Symbol	Meaning	NAG Value
`!`	Good move	1
`?`	Poor move	2
`!!`	Brilliant move	3
`??`	Blunder	4
`!?`	Interesting move	5
`?!`	Dubious move	6
`=`	Even position	10
`+/=`	White advantage	14
`=/+`	Black advantage	15
`+/-`	Significant white advantage	16
`-/+`	Significant black advantage	17
`+-`	Decisive white advantage	18
`-+`	Decisive black advantage	19

Examples

Given the following PGN game text:

{pre-game}
1.e4! {Best by test}
1...e5 $1 $113 {??}
2.d4 $10
*

the table below shows the results of several uses of originalcomment at each position of the above game.

Filter	Initial position	After `1.e4`	After `1...e5`	After `2.d4`
`originalcomment`	`"pre-game"`	`"Best by test"`	`"??"`	`None`
`originalcomment ""`	`true`	`true`	`true`	`false`
`originalcomment "a"`	`true`	`false`	`false`	`false`
`originalcomment "$1"`	`false`	`true`	`true`	`false`
`originalcomment "!"`	`false`	`true`	`true`	`false`
`originalcomment "$113"`	`false`	`false`	`true`	`false`
`originalcomment "="`	`false`	`false`	`false`	`true`
`originalcomment "??"`	`false`	`false`	`false`	`false`
`"??" in originalcomment`	`false`	`false`	`true`	`false`

Note in particular that originalcomment "$1" will match if the position contains either the NAG $1 or the corresponding annotation symbol !. A comment whose text resembles a NAG or symbolic annotation is still a comment and will not be matched by the NAG-querying version of originalcomment, the in filter may be used to check for the presence of a comment containing text that would be interpreted as a NAG by originalcomment. Positions may contain multiple NAGs but at most one symbolic annotation.

The `removecomment` Filter

The removecomment filter removes any original comments associated with the current position. The removecomment filter always yields true.

Adroit use of comment and removecomment may be employed to remove or replace part of an original comment. For example, to remove clock information embedded in comments that looks like [%clk 0:09:16], the following query may be used:

if originalcomment ~~ "^(.*)\[%clk \d+:\d+:\d+\](.*)$" {
    removecomment
    comment(\1 \2)
}

To replace all instances of one string in a comment with another, e.g. all instances of “XX” with “Y”, use the query:

$new_comment = replace(originalcomment "XX" "Y")
removecomment
comment $new_comment

Note that removecomment does not employ smart-comment-like semantics. The effects of a previously evaluated removecomment filter will be realized even if the same position later fails to match.

The –noremovecomment option may be used to prevent comment removal with the removecomment filter. In this case evaluation of removecomment filters still yield a true value but the original comment is not removed when the PGN game is written to the output file.

Comments Added by CQLi

There are six situations in which CQLi may add comments to a position which are described in the following sections.

User Comments

User comments are added by the comment filter. These comments always appear in the output PGN file, subject to the provisions of Smart Comments, unless the –silent option or the silent header parameter is used. The comment filter may be used to annotate a position with multiple comments.

Sort Comments

Each sort filter appearing in a query will cause a corresponding sort comment to be inserted at the beginning of each matching game with the best value encountered for the filter in that game. Sort comments are not emitted for sort filters using the quiet keyword parameter or when either the --silent option or the silent header parameter is used.

Header Comments

By default, every matching game contains a header comment which includes the game number of the game which is the index of the game in the input PGN file. The –noheader option may be used to suppress these comments.

Match Comments

By default, CQLi will annotate every matching position of a game with the comment CQL. The –matchstring option or the matchstring header parameter may be used to change the string used to annotate matching positions. Specifying an empty match string will effectively suppress these comments. Match comments are also suppressed if the --silent or --quiet options are used or if the silent or quiet parameters appear in the CQL header.

Auxiliary Comments

Auxiliary comments are those added during the operation of the consecutivemoves, echo, find, and line filters. See the descriptions of these filters for information about the comments they add. Auxiliary comments are suppressed if the --silent or --quiet options are used or if the silent or quiet parameters appear in the CQL header.

Position ID Comments

A position ID comment has the form id:position-id where position-id is the numeric position ID value of the position in which the comment appears. Position ID comments are added to variation positions to help identify them in the following situations:

when a variation position appears as the argument to a message or comment filter.
when the comment filter is evaluated in a variation position.
when the source or target position in a matching iteration of the echo filter is a variation position.
when the starting or ending position of a matching line filter is a variation position.

Position ID comments are suppressed if auxiliary comments are suppressed.

Comment Order

Multiple comments added to a single position will appear in the following order: header comment, match comment, sort comments, and user and auxiliary comments in the order in which they were added by the corresponding filters. Header and sort comments appear only at the initial position, before the first move. When multiple sort filters are present, sort comments appear in the order in which their respective filters appear in the CQL query. CQLi will never add more than one header comment to a game or more than one match comment to a position.

If the game from the input PGN originally contained comments, all comments added by CQLi in a given position will appear after any original comments at that position.

For example, given the following input:

{Pre-game comment} e4 {Best by test} {second comment} e5 *

and the query:

sort line --> comment "comment 1"
          --> comment "comment 2"
          --> comment "comment 3"

the result will be emitted as:

{Pre-game comment} {Game number 1} {CQL} {<sort-id-0>: 3}
{comment 1} {Start line that ends at move 2(wtm)} 1.e4
{Best by test} {second comment} {comment 2} 1...e5 {comment 3}
{End line of length 3 that starts at move 1(wtm)} *

Comment Coalescing

By default, multiple comments at a position will be written out as multiple distinct comments in the PGN file, each enclosed by a separate set of braces. If the –coalescecomments option is used, multiple comments at a single position will be combined into a single comment with space characters separating each coalesced comment component. For example, the query:

initial
comment("A" "B") comment "X" comment "Y"

produces three comments at the initial position: one containing AB, one containing X, and one containing Y. By default, these comments would be represented in the PGN file as three separate comments:

{AB} {X} {Y}

If comment coalescing is enabled, the combined comment will be represented as:

{AB X Y}

Separately written comments make it clear where each comment begins and ends but some chess software does not handle multiple comments well and may not recognize or preserve multiple comments.

Unique comments

By default, multiple comments with the same text at the same position are not written to the output PGN file. This deduplication occurs before comment coalescing and includes both original comments and comments added by CQLi. If there are duplicate original comments at the same position, all but one of them is removed, even if the duplicate comments were not adjacent. Duplicates between new and original comments are likewise removed. The –nouniquecomments option may be used to allow such duplicate comments.

Smart Comments

The Smart Comments mechanism ensures that only appropriate comments are added to the resulting matching games by suppressing unnecessary comments. Comments added by filters such as comment, line, and find are suppressed in the following cases:

The position does not match the provided CQL query.
A subsequent filter in the same compound expression fails to match.
A filter enclosing the filter responsible for the comment does not match, even if the full query ultimately does.
Comments added in a filter that only accumulates comments associated with the best value encountered by the filter as described below.

Position Does Not Match

Any comments added while evaluating the current position will be suppressed if the position ultimately does not match the query. For example:

comment "Terminal position"
terminal

will only add the comment “Terminal position” to positions that have no children as all other positions will fail to match the terminal filter which will suppress preceding comments for that position.

Subsequent Filter Fails to Match

If a later filter in the compound expression containing a comment does not match, previous comments are elided even if the position ultimately matches the query. For example:

if terminal {
    comment "End of mainline"
    mainline
}

will only comment the end of the main line, even though the enclosing if filter will match all non-terminal positions.

Enclosing Filter Does Not Match

If a comment is added by a filter that is enclosed by another filter that fails to match later, the comment is suppressed. For example:

(2 < { comment("Pinned pieces:" pin) pin } < 5) or true

will only comment on positions in which there are 3-4 absolute pins as the enclosing comparison will not otherwise match even though the query itself will match every position.

Best Values

The min, max, sort, line, sorted echo, and consecutivemoves filters accumulate only those comments that are associated with the best value(s) encountered by those filters. The details of effect of this behavior are described below.

`min` and `max` Filters

Comments appearing in a min or max filter will only be emitted for values that correspond to the lowest or highest argument values. For example:

min( {comment "A" 1} {comment "B" 2} {comment "X" 1} {comment "Y" 3})

will emit the comments “A” and “X”.

`sort` Filters

Comments appearing within a sort filter will only be emitted for the lowest or highest value evaluated for the sort filter within a particular game. If multiple evaluations yield the same best value, only the comments associated with the first occurrence of the best value are kept unless the –keepallbest option is used. For example:

sort {
    num_moves = move legal count
    comment("Number of moves available:" num_moves)
    num_moves
}

will only emit a comment for the first position encountered having the maximum number of moves of all evaluated positions.

`line` Filters

Comments appearing in a line filter are only emitted for the longest matching line found. For example:

checks = 0
line --> { check checks += 1 comment("Check #" checks) } +

will only comment the checks associated with the longest line of checks found at the current position. If there are multiple matching lines of the longest length in a line filter, only the comments associated with one of the matching lines are kept unless the --keepallbest option is used.

Sorted `echo` Filters

When the echo filter is used as the target of a sort filter, only the comments associated with the largest matching value of the echo filter’s target are retained. If multiple evaluations of the echo target filter yield the largest value, only the comments generated with the first evaluation of the largest value are retained unless the --keepallbest option is used.

The `consecutivemoves` Filter

Comments appearing in the arguments to the consecutivemoves filter are only retained for the evaluation of the filter that yields the longest matching sequence.