Class TokenSupport

constructor

• new TokenSupport(tokens: Token[], isContinuous: boolean): TokenSupport

• Parameters

  • tokens: Token[]

    The tokens of this support.
  • isContinuous: boolean

    Is this support a continuous sequence of tokens, i.e. a token span?

  Returns TokenSupport

isContinuous

tokens

charSpan

• get charSpan(): CharSpan

• The character span between the first and last token relative to the enclosing paragraph; for discontinuous support this includes intervening gaps.

  Returns CharSpan

first

• get first(): Token

• Returns Token

firstCharParaOffset

• get firstCharParaOffset(): number

• Offset of the first character of these tokens within the enclosing paragraph.

  Returns number

ids

• get ids(): string[]

• Returns string[]

last

• get last(): Token

• Returns Token

lastCharParaOffset

• get lastCharParaOffset(): number

• Offset of the last character of these tokens within the enclosing paragraph.

  Returns number

paragraph

• get paragraph(): Paragraph

• Returns Paragraph

sentence

• get sentence(): Sentence

• Returns Sentence

size

• get size(): number

• Returns number

text

• get text(): string

• Substring of a full text as denoted by this support (before correction). For discontinuous supports, the result includes the intervening gaps.

  Returns string

spans

• spans(): TokenSupport[]

• Breaks this token support into continuous sub-sequences of tokens.

  Returns TokenSupport[]

textSpans

• textSpans(): string[]

• The coverage texts of each of the continuous spans, ordered by word-order.

  Returns string[]

toString

• toString(): string

• Returns string

Staticof

• of(tokens: Token[]): TokenSupport

• Creates a TokenSupport object from a list of tokens.

  Parameters

  • tokens: Token[]

    Non-empty list of tokens (no need for them to be sorted).

  Returns TokenSupport