Table: cohereai_tokenize
Get Tokens for a given input string.
Notes:
- A
textis a string to get the tokens from. - Maximum length of text is 65536 characters.
Examples
Basic example with default settings
select tokensfrom cohereai_tokenizewhere text = 'hello world';Schema for cohereai_tokenize
| Name | Type | Operators | Description |
|---|---|---|---|
| _ctx | jsonb | Steampipe context in JSON form, e.g. connection_name. | |
| text | text | = | The text to tokenize for, encoded as a string. |
| token_strings | text | Tokens in the form of input string. | |
| tokens | text | Tokens for a given text prompt. |