Skip to main content

Token

Token (word, separator etc.)

Fields

PropertyRequiredTypeDescription
idTruestringID of the token used to refer to it from other objects
offTrueintegerThe offset in token's parent paragraph
textTruestringThe token's text
origOffFalseintegerThe token's offset in the original paragraph (omitted when identical to "off")
origTextFalsestringThe token's text in the original paragraph (omitted when identical to "text")
lemmaFalsestringLemma, the canonical or dictionary form of the word
posFalsestringUniversal POS
parIdFalsestringID of the syntactic parent token; missing for the root token
fncFalsestringGrammatical function of this token according to UD (nsubj, obj, ...)
featsFalseMap[string, string]Universal and custom features