> ## Documentation Index
> Fetch the complete documentation index at: https://docs.starfort.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Batch overview

> The Starfort S3 de-identification batch: asynchronous processing driven by src and tgt bucket references — submit, poll, collect (Starfort v1.4 docs)

The **S3 de-identification batch** is an **asynchronous** processing path that de-identifies files already sitting in a bucket, addressed by location alone. Pass just the source address (`src_s3_url`) and the target address (`tgt_s3_url`), and Starfort reads the original straight from the bucket, detects personal data (PII) and sensitive topics (Topic), and writes the de-identified (masked) file to the target location.

**Starfort does the data movement.** Starfort reads `src` and writes `tgt` using standard S3 operations (GetObject / PutObject). The caller has exactly three jobs — put the file in the bucket, grant access, and collect the result from `tgt`. There's no transfer logic to build.

## How it relates to the inline Guard API

The batch is a path **added alongside** the existing [Guard API](/en/v1.4/api/quickstart). The synchronous API stays as it is; existing callers are unaffected.

| Axis             | Inline Guard API                                       | S3 de-identification batch                                                                        |
| ---------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------- |
| File delivery    | Base64-embedded in the request body                    | **Bucket reference** — just the `src` address                                                     |
| Processing model | Synchronous, one turn (the response is the completion) | **Asynchronous jobs** — intake and completion are separate                                        |
| Result delivery  | Returned in the response body                          | **Written directly to `tgt`** — per-verdict [output rules](/en/v1.4/api/batch/outputs-and-limits) |
| Request size     | Bounded by the intake ceiling and base64 inflation     | No file in the body, so **no request-size constraint**                                            |

The inspection itself doesn't change — the batch runs the same Guardian engine with the same policies and verdict rules as the inline path. Inspection policy is decided by **the server (project settings)**, not the request; there is no batch-specific policy system.

## Prerequisites

<Steps>
  <Step title="Prepare a project and Guardian">In the console, prepare a project and a Guardian that defines which policies to inspect with.</Step>
  <Step title="Register a Storage Connection">Register the integration settings that let Starfort reach your bucket — endpoint, access credentials, and allowed targets (read/write scopes). See [Storage Connection](/en/v1.4/api/batch/storage-connection).</Step>
  <Step title="Issue an API key">Issue one from the project's API Keys menu. See [Manage API keys](/en/v1.4/admin/api-keys).</Step>
</Steps>

We recommend keeping the source input folder (e.g. `inbox/`) and the output folder (e.g. `deidentified/`) separate.

## Authentication

The **same API key system** as the Guard API: send an `sf_`-prefixed key in the `X-Starfort-Guard-Api-Key` header, and jobs are scoped to the Project Guardian that key points to. See [Authentication](/en/v1.4/api/authentication).

## Job states

A batch job has four states:

| State        | Meaning                                                                                       |
| ------------ | --------------------------------------------------------------------------------------------- |
| `PENDING`    | Accepted and waiting.                                                                         |
| `PROCESSING` | In progress. Internal retries aren't exposed — the state stays "processing".                  |
| `COMPLETED`  | Inspection ran to completion — finalized together with a verdict (`PASS` / `MASK` / `BLOCK`). |
| `FAILED`     | Inspection couldn't be completed — finalized together with a failure reason (`error`).        |

After `PENDING → PROCESSING`, a job finalizes as `COMPLETED` or `FAILED`, and a finalized state is terminal.

* **Jobs are processed in intake order (FIFO).** There is no priority tier.
* **Completion is confirmed by polling.** Check state with the [job status API](/en/v1.4/api/batch/jobs).

## BLOCK is not a failure

**BLOCK** means the inspection completed normally and returned a blocking verdict, so it's expressed as `COMPLETED` + `action: "BLOCK"`. `FAILED` means the inspection couldn't be completed: no output file is written, and a failure is never treated as "nothing detected" — the same lineage as the inline API's [fail-closed principle](/en/v1.4/api/errors).

<Note>
  While a [Kill Switch](/en/v1.4/admin/kill-switch) is active, both job creation and status queries are rejected, and waiting or in-progress jobs are finalized as failed with no output. Outputs already written to `tgt` are not retroactively recalled.
</Note>

## Next steps

* [Create & poll jobs](/en/v1.4/api/batch/jobs) — endpoints, request/response fields, and the polling pattern
* [Storage Connection](/en/v1.4/api/batch/storage-connection) — S3 integration settings and allowed scopes
* [Output rules & limits](/en/v1.4/api/batch/outputs-and-limits) — per-verdict `tgt` outputs, supported formats, limits
* [Batch errors](/en/v1.4/api/batch/errors) — intake-time HTTP errors and processing-time failure codes