Data Connectors

Data Connectors establish secure connections between the platform and your document storage. Once connected, the platform can discover documents, read content for metadata extraction, and write enriched metadata back to the source.

Supported Data Connectors

Amazon S3

Connect to AWS S3 buckets for scalable cloud storage.

Azure Blob Storage

Connect to Azure Blob containers.

Google Cloud Storage

Connect to GCS buckets.

SharePoint

Integrate with Microsoft 365 document libraries.

OneDrive

Connect to Microsoft OneDrive sites and folders.

PostgreSQL

Connect to PostgreSQL databases with pgvector support.

Qdrant

Vector database integration for semantic search.

Configuration Details

Source Type	Description	Key Configuration	Ideal Use Case
Amazon S3	AWS cloud object storage	Bucket name, Access Key, Secret Key	Large-scale document archives, cloud-native workflows
Azure Blob Storage	Azure cloud object storage	Account, Container, Credentials	Azure-native document archives
Google Cloud Storage	GCP cloud object storage	Bucket, Service Account	GCP-native document archives
SharePoint	Microsoft 365 document management	Client ID, Client Secret, Tenant ID, Site Name	Enterprise document libraries, Office 365 environments
OneDrive	Microsoft OneDrive cloud storage	Client ID, Client Secret, Tenant ID, Site	Personal and team drives in Microsoft 365
PostgreSQL	Relational database with pgvector extension	Host URL, Database name, User credentials, Port	Structured + unstructured hybrid data, existing database workflows
Qdrant	Purpose-built vector database for AI	API Key, Collection name, URL	Semantic search applications, RAG pipelines

See the Integrations Overview for the complete matrix of sources, destinations, and supported file types.

Key Features

Multiple Profiles

Create and manage multiple Data Connector connections

Connection Testing

Validate credentials before saving

Active Profile Selection

Switch between Data Connectors with one click

Schema Configuration

Customize field mappings (filename key, text key, tags key)

How Data Connectors Work

Create a Connector

Select your storage type and provide the required credentials.

Test the Connection

Validate that the platform can access your documents before saving.

Configure Schema Mapping

Map your data fields (filename, text content, tags) to the platform’s expected format.

Start Processing

Your documents are now available for metadata extraction.

Python SDK

Create Connector
List Connectors
Ingest Data
Delete Connector

from unstructured import UnstructuredClient

client = UnstructuredClient(
    base_url="https://unstructured.your-company.com/rest/unstructured",
    username="your-username",
    password="your-password",
)

# Create an S3 connector
connector = client.data_source.create(
    connector_name="my-s3-bucket",
    connector_body={
        "type": "S3DataSourceManager",
        "name": "my-s3-bucket",
        "bucket_name": "my-documents",
        "aws_access_key_id": "YOUR_ACCESS_KEY",
        "aws_secret_access_key": "YOUR_SECRET_KEY",
        "region": "us-east-1",
    },
)
print(f"Created connector: {connector.profile_id}")

# List all connectors (returned as a dict keyed by connector name)
connectors = client.data_source.list()
for name, config in connectors.connectors.items():
    print(f"{name}: {config.type}")

import uuid

# Ingest documents from a connector (runs as a background job)
job_id = str(uuid.uuid4())
client.data_source.ingest(
    data_connector_name="my-s3-bucket",
    file_names=["contracts/msa.pdf"],  # Optional: omit to ingest everything
    job_id=job_id,
)

# Track progress
progress = client.task_status.get_status(job_id=job_id)
print(f"Ingestion {progress.percent_complete:.0f}% complete ({progress.status})")

# Delete a connector
client.data_source.delete(connector_name="my-s3-bucket")
print("Connector deleted")

API Reference

Create Connector

Create a new data connector

Update Connector

Update an existing connector configuration

List Connectors

List all your data connectors

Delete Connector

Remove a data connector

Ingest Data

Ingest documents from a connector

List Ingested Data

View ingested document metadata

Getting Started

Core Concepts

Cookbooks

Integrations

Supported Data Connectors

Amazon S3

Azure Blob Storage

Google Cloud Storage

SharePoint

OneDrive

PostgreSQL

Qdrant

Configuration Details

Key Features

Multiple Profiles

Connection Testing

Active Profile Selection

Schema Configuration

How Data Connectors Work

Python SDK

API Reference

Create Connector

Update Connector

List Connectors

Delete Connector

Ingest Data

List Ingested Data

​Supported Data Connectors

Amazon S3

Azure Blob Storage

Google Cloud Storage

SharePoint

OneDrive

PostgreSQL

Qdrant

​Configuration Details

​Key Features

Multiple Profiles

Connection Testing

Active Profile Selection

Schema Configuration

​How Data Connectors Work

​Python SDK

​API Reference

Create Connector

Update Connector

List Connectors

Delete Connector

Ingest Data

List Ingested Data

Supported Data Connectors

Configuration Details

Key Features

How Data Connectors Work

Python SDK

API Reference