Contains one of UC_TOKEN_PREFIX_SPACE or UC_TOKEN_PREFIX_TAB as the lowest byte, and a number of
repeats (excluding the symbol itself) as the higher byte. At most 255 repeats corresponding to 256 chars.
Such padding always emitted for spaces and tabs around reserved characters, line terminators, after input
beginning, and before input end. Spaces and tabs e.g. between words may be emitted as part of string tokens.
Contains UC_TOKEN_PREFIX_INSET as the lowest byte. The tokens after this token considered an inset
input chunks to be processed by appropriate lexer. The token itself is used as inset format
identifier. The inset input ends with UC_TOKEN_INSET_END token.
URI charge token.
Can be one of:
Non-empty percent-decoded string.
Subsequent string chunks allowed.
Number corresponding to one of reserved characters represented as one of
UC_TOKEN_constants.Number corresponding to one of the line terminators:
Number containing encoded padding.
Contains one of UC_TOKEN_PREFIX_SPACE or UC_TOKEN_PREFIX_TAB as the lowest byte, and a number of repeats (excluding the symbol itself) as the higher byte. At most 255 repeats corresponding to 256 chars.
Such padding always emitted for spaces and tabs around reserved characters, line terminators, after input beginning, and before input end. Spaces and tabs e.g. between words may be emitted as part of string tokens.
Contains UC_TOKEN_PREFIX_INSET as the lowest byte. The tokens after this token considered an inset input chunks to be processed by appropriate lexer. The token itself is used as inset format identifier. The inset input ends with UC_TOKEN_INSET_END token.