Python

from unstructured import UnstructuredClient

client = UnstructuredClient(
    username="My Username",
    password="My Password",
)
response = client.metadata.generate.generate_batch(
    data_connector_name="data_connector_name",
)
print(response.message)

{
  "message": "<string>"
}

Metadata

Generate Batch

Classify all files in a data connector in batches with the provided tags. To run sensitivity detection, use tags with a strategy of SENSITIVITY or a group which includes the magic word “Sensitivity”

Request Body

Field	Type	Description
`data_connector_name`	`str`	Name of the data connector (vector database profile) to use for classification.
`total_data_sets`	`Optional[int]`	Total number of files to classify.
`tag_names`	`Optional[List[str]]`	Names of tags to use for classification (if `tag_datas` not provided).
`tag_datas`	`Optional[Dict]`	Tag data to use for classification.
`overwrite`	`bool`	Whether to overwrite existing tags. Default: `false`
`job_id`	`Optional[str]`	Custom job ID for tracking the classification task.
`hierarchy_name`	`Optional[str]`	Name of the hierarchy/graph to use (if `hierarchy_data` not provided).
`hierarchy_data`	`Optional[Dict]`	Hierarchy/graph data to use for classification. Default: `{}`
`dataslice_id`	`Optional[str]`	ID of the dataslice to use for file filtering.
`conditions`	`Optional[Condition]`	Conditions to use for file filtering.

Response

200: Classification job started successfully
- Returns: { "message": str, "job_id": str }
500: Internal Server Error

Example

{
  "data_connector_name": "my-connector",
  "tag_names": ["category", "sentiment"],
  "overwrite": false,
  "dataslice_id": "abc123"
}

POST

classify_bulk

Python

from unstructured import UnstructuredClient

client = UnstructuredClient(
    username="My Username",
    password="My Password",
)
response = client.metadata.generate.generate_batch(
    data_connector_name="data_connector_name",
)
print(response.message)

{
  "message": "<string>"
}

Body

application/json

data_connector_name

string

required

total_data_sets

integer | null

tag_names

string[] | null

tag_datas

Tag Datas · object

Show child attributes

tag_datas.{key}

Tag · object

Show child attributes

tag_datas.{key}.name

string

required

tag_datas.{key}.description

string

required

tag_datas.{key}.tag_id

string | null

tag_datas.{key}.username

string | null

tag_datas.{key}.output_type

enum<string> | null

default:string

Available options:

string,

binary,

number,

date

tag_datas.{key}.available_values

(string | number | integer)[] | null

tag_datas.{key}.examples

(string | object)[] | null

tag_datas.{key}.created_at

string<date-time> | null

tag_datas.{key}.updated_at

string<date-time> | null

tag_datas.{key}.maxValues

tag_datas.{key}.neg_examples

string[] | null

tag_datas.{key}.retry_feedback

Retry Feedback · object

tag_datas.{key}.date_format

string | null

tag_datas.{key}.strategy

string | null

default:LLM

tag_datas.{key}.scope

string | null

default:ALL_PAGES

tag_datas.{key}.truncated_available_values

boolean | null

default:false

tag_datas.{key}.enhance_file_metadata

boolean | null

default:true

tag_datas.{key}.regex

string | null

tag_datas.{key}.patterns

RegexPattern · object[] | null

Show child attributes

tag_datas.{key}.patterns.pattern

string

required

tag_datas.{key}.patterns.description

string | null

tag_datas.{key}.patterns.output_value

string | null

tag_datas.{key}.patterns.confidence

number | null

tag_datas.{key}.patterns.validator

string | null

tag_datas.{key}.patterns.active

boolean | null

default:true

tag_datas.{key}.visual_rules

RuleBasedRule · object[] | null

Show child attributes

tag_datas.{key}.visual_rules.tag_value

tag_datas.{key}.visual_rules.condition

Condition · object

Show child attributes

tag_datas.{key}.visual_rules.condition.condition

enum<string> | null

Available options:

AND,

OR

tag_datas.{key}.visual_rules.condition.children

any[] | null

tag_datas.{key}.visual_rules.condition.tag

BaseTag · object

Show child attributes

tag_datas.{key}.visual_rules.condition.tag.name

string

required

tag_datas.{key}.visual_rules.condition.tag.values

(string | number | integer)[]

required

tag_datas.{key}.visual_rules.condition.tag.operator

string | null

default:any

tag_datas.{key}.visual_rules.description

string | null

tag_datas.{key}.visual_rules.confidence

number | null

tag_datas.{key}.group

string | null

tag_datas.{key}.min_confidence

number | null

overwrite

boolean

default:false

job_id

string | null

hierarchy_name

string | null

hierarchy_data

Hierarchy Data · object

dataslice_id

string | null

conditions

Condition · object

Show child attributes

conditions.condition

enum<string> | null

Available options:

AND,

OR

conditions.children

any[] | null

conditions.tag

BaseTag · object

Show child attributes

conditions.tag.name

string

required

conditions.tag.values

(string | number | integer)[]

required

conditions.tag.operator

string | null

default:any

Response

Successful Response

message

string

required

Generate Upsert

⌘I

Data Connectors

Tags

Taxonomies

Metadata

Task Tracking

Data Slices

Destinations

Generate Batch

Request Body

Response

Example

Body

Response