Connect Multi FileFeeds to Azure Blob Storage

Availability: Multi FileFeeds with Azure Blob Storage connections enabled.

This guide walks through three things:

1. Creating an Azure Blob connection in OneSchema, including running the setup script that provisions a federated identity credential on a Microsoft Entra ID app registration in your tenant.
2. Using that connection as a source on a Multi FileFeed, so OneSchema polls a container on a schedule and ingests new blobs.
3. Using that connection as a destination on a Multi FileFeed, so OneSchema writes processed output files into a container.

A single Azure Blob connection can back both directions — you do not need separate connections for source and destination usage. RBAC role assignments on the storage account decide what the connection's principal can do.

---

Terminology

---

How it works

OneSchema does not store long-lived storage account keys, client secrets, or SAS tokens. Instead, OneSchema's workloads present a short-lived OIDC token issued by their own runtime to the Microsoft identity platform, which validates it against a federated identity credential you configure on a Microsoft Entra ID app registration in your tenant.

sequenceDiagram
    participant OS as OneSchema runtime
    participant AAD as Microsoft identity platform
    participant SP as Your Entra ID app registration
    participant BLOB as Your Azure Storage account

    OS->>OS: Read runtime OIDC token<br/>(projected by the runtime, never persisted)
    OS->>AAD: Token request — present OIDC token<br/>as client_assertion for {tenant_id, client_id}
    AAD->>SP: Validate token against federated identity credential
    SP-->>AAD: Federated credential trusts this issuer/subject
    AAD-->>OS: Microsoft Entra ID access token<br/>(scope: storage.azure.com/.default)
    OS->>BLOB: List / Get / Put blob
    BLOB-->>OS: Response (gated by RBAC on the storage account)

A few things follow from this design:

• No keys to rotate. OneSchema never sees a client secret, SAS token, or storage account key. Credentials live for the duration of one operation (Microsoft Entra ID access tokens are valid for ~1 hour).
• Read vs. write is RBAC. Microsoft Entra ID issues a single resource scope (https://storage.azure.com/.default); whether the token can read or write a given blob is decided by RBAC role assignments on the storage account. Grant Storage Blob Data Reader for source-only access and Storage Blob Data Contributor for destination access.
• Setup is one-time per app registration. The setup script provisions a Microsoft Entra ID app registration, configures the federated identity credential, and grants the role assignments. Once it's in place, you can create as many OneSchema connections as you like against the same tenant — though each OneSchema connection is scoped to a single storage account.

---

Prerequisites

You will need:

You will also need the storage account name and the resource group that contains it.

---

Federation values

These values identify the OIDC token OneSchema's runtime presents to Microsoft Entra ID. They are stable per OneSchema deployment, so the same values apply to every Azure Blob connection you create. The generated setup script contains FEDERATION_ISSUER='{ONESCHEMA_OIDC_ISSUER}' and FEDERATION_SUBJECT='{ONESCHEMA_OIDC_SUBJECT}' placeholders — replace them with the Issuer and Subject you receive from OneSchema before running the script. The Audience is already hardcoded in the script and doesn't need editing. If you set up the federated credential through the Azure Portal instead, paste the Issuer and Subject into the corresponding fields there.

The audience matches Microsoft's default for the "Kubernetes accessing Azure resources" federated-credential scenario, so the Azure Portal will pre-fill it for you if you set up the credential through the UI.

---

Step 1 — Create an Azure Blob connection in OneSchema

1. Open Connections → Azure Blob Storage and click Create Azure Blob Storage connection.
2. Give the connection a name your team will recognize (e.g. acme-prod-blob).
3. Fill in your Microsoft Entra ID tenant ID and your target storage account name. The form generates a setup script tailored to those values.
4. Copy the generated script into an editor and replace the two {ONESCHEMA_OIDC_ISSUER} and {ONESCHEMA_OIDC_SUBJECT} placeholders with the values from the Federation values section above. The other variables are already filled in for you.
5. Run the script in a terminal where az is authenticated against the right subscription. The script:
   • Creates a Microsoft Entra ID app registration named oneschema-blob (rename via the advanced section).
   • Creates a service principal for the app.
   • Configures a federated identity credential trusting OneSchema's OIDC issuer.
   • Grants the service principal Storage Blob Data Reader and Storage Blob Data Contributor on the storage account.
   • Prints the Application (client) ID at the top of its output.
6. Paste the printed Client ID into the App registration (client) ID field on the form.
7. Click Test connection. OneSchema's backend will mint a federated access token against your tenant and verify that it can call the Blob endpoint. The Save button unlocks once the test succeeds.

Prefer the Azure Portal? You can do the same setup through Microsoft Entra ID → App registrations → New registration, then add a federated credential using the "Kubernetes accessing Azure resources" scenario with the Issuer and Subject from the Federation values section. The Audience field defaults to api://AzureADTokenExchange for that scenario. Finally, grant the app Storage Blob Data Reader and/or Storage Blob Data Contributor on the storage account.

---

Step 2 — Use the connection as a source

1. Open a Multi FileFeed and switch to its settings.
2. Under Source, choose Azure Blob.
3. Pick the connection you just created.
4. Enter the container name (no az:// scheme, no leading slash) and an optional prefix for the folder OneSchema should poll.
5. Optionally add a file filter (a case-insensitive regular expression matched against each blob's key after the prefix is stripped).
6. Pick a pull frequency — hourly or daily at a chosen UTC time.

Each poll processes only matching blobs that have not been ingested before (deduplicated by (container, key, ETag)). Source connections use only the Storage Blob Data Reader role.

---

Step 3 — Use the connection as a destination

1. Open a Multi FileFeed and switch to its settings.
2. Under Destination, choose Azure Blob.
3. Pick the connection you just created.
4. Enter the container name and an optional prefix.

Each output file the workflow produces will be uploaded into <container>/<prefix>/<file-name>. Existing blobs are overwritten in place; Azure assigns a new ETag, and any retention policy on the storage account decides whether the old version is kept. Destination connections use the Storage Blob Data Contributor role.

---

Troubleshooting