# data-inventory attestor

Source: https://www.testifysec.com/docs/cilock/attestors/data-inventory/

The cilock data-inventory attestor signs the ent schema tables a repository declares with the fields it can read, each at its exact file and line, and lists per Go package what it saw and did not read.

Reads the repository at one commit and signs the data stores its **ent** schemas declare: each table with the fields the reader followed, the file and line each is declared at, and ent's own `StorageKey` and `Sensitive()` marks. It reads source only. It never connects to a database and never samples rows.

 

It signs **observations only**. There is no `complete`, `ok` or status field: the predicate lists, per Go package, what the reader saw and did not read (`unread[]`), and each store lists the parts it could not follow (`unresolved[]`). A field list is never "the columns of the table"; whether an inventory is enough is for policy, which knows the repository layout. The design and the full rule table are in `docs/design/planned/data-inventory-fail-closed.md`.

 

It assigns no sensitivity classes. It is the deterministic, CI-signed list that an agent's data-discovery findings are checked against for recall: every ent-backed store the agent reports should match a store id and line here, and every store here should be reported or explained.

 

| Name | `data-inventory` |
| --- | --- |
| Predicate type | `https://aflock.ai/attestations/data-inventory/v0.2` |
| Lifecycle | `prematerial` |
| Default binary? | No |
| Category | asset-inventory (primary) |
| Recommended trace | off — no syscall tracing needed |
| Auto-attaches when | - prefile\_glob: ent/schema/\*.go \| \*\*/ent/schema/\*.go |

The facts in this box are generated from the CI/lock binary's own catalog (`cilock tools list`). Do not hand-edit — run `npm run gen:catalog`.

 

## Validated invocation

 

```bash
cilock run --step data-inventory \
  --signer-file-key-path key.pem --outfile data-inventory.att.json \
  -a data-inventory \
  --attestor-data-inventory-search-root services/api/ent/schema \
  -- true
```

 

`--attestor-data-inventory-scope-base` (default `auto`) says what the `ent/schema` directory test is relative to: `auto` is the nearest ancestor of the working directory that holds `.git`, else the working directory. Use `working-directory` when the source is an archive unpacked inside an unrelated checkout; the predicate's `scopeBase` records the choice.

 

`--attestor-data-inventory-search-root` (default `.`) narrows the walk. It does not change store ids: every path in the predicate is relative to the scope base (the git root, else the working directory), so two services of one repository never share an id.

 

## What gets captured

 

- `root`: the search root as a slash-separated path relative to the scope base (`.` for the base itself), never a host path.
 - `scopeBase`: `git` when the `ent/schema` directory test was made relative to the nearest ancestor of the working directory that holds `.git`, else `working-directory`. The predicate never holds a host path, so one commit signs the same bytes wherever it is checked out.
 - `sources[]`: every non-test `.go` file of every package that imports an ent package or is an `ent/schema` directory, with `framework` and the sha256 of the exact bytes parsed (each file is read once, hashed and parsed from the same buffer). A file that did not parse is still a source.
 - `packages[]`, sorted by dir: `dir` and `unread[]` (fixed text, sorted): what the reader saw in that directory and did not read: a build-constrained file, a symlink, a file that does not parse, an import outside its allowlist, a type or method it does not follow, a `_test.go` file. Every `ent/schema` directory (by repository-relative path) is listed even with no ent import, and one whose only Go files are tests is listed with them; one with no readable store says so. An empty `unread` is a fact about the reader, not a verdict.
 - `stores[]`, sorted by id. Each has `id` (`ent:<path>#<Type>`), `framework`, `kind` (`table`), `name` (the Go type name), `source` (`path`, `line`), `fields[]` and `unresolved[]`.
 - Each field: `name`, `column` (ent `StorageKey` when set), `type` (the ent kind word), `sensitive` (ent `Sensitive()`), `source` (`path`, `line`).
 - `counts`: `stores`, `fields`, `packages`, `packagesWithUnread`.

 

A store is read only when it is a struct embedding `ent.Schema` directly, with `Fields()` in the same file, whose body is a single `return` of a `[]ent.Field` literal made of `field.<Kind>("name")` chains. When a `StorageKey` is set more than once, the last call wins, as in ent. A field the reader cannot follow (a helper call, a non-literal name) is listed in the store's `unresolved` with the line and reason, never dropped.

 

Not read in this version, and so reported rather than guessed: mixins, edges (foreign-key columns and join tables), `entsql` table annotations, project and third-party imports, aliases and defined types, other schema methods, build variants. Later versions read them one at a time.

 

Subjects: one per store, `data-store:<id>`, digest `sha256(<id>)`. It is a lookup key: it joins a store across evidence and vouches for no artifact bytes. In the signed statement cilock prefixes the name with the predicate type.

 

## Why this shape

 

- **Facts, not a verdict.** Everything in the predicate is something the schema states. `sensitive` is the schema author's own marking, not a judgement about the data.
 - **Exact positions.** Every store and field carries the file and line it is declared at, so another tool's claim about a column can be checked against the source line.
 - **No caps.** The walk visits every file under the root except directories named `.git`, `.svn`, `.hg`, `node_modules`, `vendor`, `__pycache__`, `venv` and `.venv`, and does not follow symlinks. Other dot-directories are walked. An I/O error fails the attestor; a partial inventory is never signed.
 - **Stable ids.** A store id depends only on the file path relative to the scope base and the type name, so it joins across runs and attestors by `sha256(id)`.

 

## Validate it locally

 

```bash
cilock run --step data-inventory --signer-file-key-path key.pem \
  -a data-inventory -o att.json -- true
jq -r .payload att.json | base64 -d \
  | jq '.predicate.attestations[] | select(.type|test("data-inventory")) | .attestation.counts'
```

 

The testkit fixture `plugins/attestors/data-inventory/testdata/fixtures/ent-schema` replays five real ent schema files in workdir mode. Every one declares `Mixin()` or `Edges()`, so the fixture's package lists them as unread and its stores list them as unresolved: the fixture proves the reader reports what it does not read on real schemas.

 

## How a verifier consumes this

 

The fields are explicit field declarations only. Ent also adds an implicit `id` column and, through edges and mixins, more columns; the list of fields is never an exhaustive column inventory of the database.

 

The completeness verdict is policy's. Rego over the predicate: refuse an inventory with an unread package, and read only stores with nothing unresolved as "the data this product stores":

 

```rego
package datainventory

deny contains msg if {
  p := input.packages[_]
  count(p.unread) > 0
  msg := sprintf("%s was not read completely: %v", [p.dir, p.unread])
}

read_stores := {s.id | count(deny) == 0; s := input.stores[_]; count(s.unresolved) == 0}
```

 

A policy that knows the repository can also require that every directory it expects (for example `ent/schema`) has an entry in `packages[]`.

 

## Notes

 

- v0.2 is structure only. v0.1 carried rule-assigned classes and was never released. This is the first slice: it signs only what it can prove.
 - Only ent is read. Other ORMs, SQL migrations and infrastructure-as-code stores are left to the agentic data-discovery workflow.

 

## Gotchas

 

- `sensitive` is ent `Sensitive()`, which some schemas use only to keep a field off an API (a public key, say). It says the author marked the field, nothing more.
 - The table name is not in the predicate yet (it needs `Annotations()`, which this version does not read). Ent default table names are never computed.
 - A schema with `Edges()` or `Mixin()` lists them as unresolved and its package as unread: those add columns and join tables this version does not read. Most real ent schemas are in this state until the later versions land.
 - Directories named `node_modules`, `vendor`, `.git`, `.svn`, `.hg`, `__pycache__`, `venv` and `.venv` are not walked, and a schema under one is absent without an entry. A schema under `build`, `dist` or `target` is read.
 - The pre-gate fires on `ent/schema/*.go`. A repository that keeps its schema elsewhere needs `-a data-inventory`.

 

## FAQ

 

**Does it read the database?** No. Source only.

 

**Why no PII or secret classes?** Classes come from the agentic data-discovery workflow, which reads how the code uses each field. This attestor gives that workflow a signed list to be checked against.

 

## See also

 

- [`lockfiles`](https://www.testifysec.com/docs/cilock/attestors/lockfiles), the same committed-source shape.
 - [`k8smanifest`](https://www.testifysec.com/docs/cilock/attestors/k8smanifest), for joining stores to the workloads that reach them.

 

---

**This page is generated from the CI/lock tool catalog.** Don't edit it here — the source is [`attestation/detection/docs/data-inventory.doc.md`](https://github.com/aflock-ai/rookery/edit/main/attestation/detection/docs/data-inventory.doc.md) in [aflock-ai/rookery](https://github.com/aflock-ai/rookery). The same catalog powers `cilock tools show data-inventory` in the CLI, so a single edit keeps the terminal and the website in sync.

[Edit this page →](https://github.com/aflock-ai/rookery/edit/main/attestation/detection/docs/data-inventory.doc.md) ·  [Add a new tool →](https://github.com/aflock-ai/rookery/new/main/attestation/detection/docs)

When you open the pull request: if you are an **AI agent**, add the `ai-agent` label. If you are a **human**, add the `human` label.

Reference generated from the product documentation. Match commands and support details to your installed release.
