Skip to main content
View Markdown ↗

Copy this page

Select and copy the Markdown below, then paste it into your LLM.

On this page

data-inventory attestor

Reads the repository at one commit and signs the data stores its ent schemas declare: each table with the fields the reader followed, the file and line each is declared at, and ent's own StorageKey and Sensitive() marks. It reads source only. It never connects to a database and never samples rows.

It signs observations only. There is no complete, ok or status field: the predicate lists, per Go package, what the reader saw and did not read (unread[]), and each store lists the parts it could not follow (unresolved[]). A field list is never "the columns of the table"; whether an inventory is enough is for policy, which knows the repository layout. The design and the full rule table are in docs/design/planned/data-inventory-fail-closed.md.

It assigns no sensitivity classes. It is the deterministic, CI-signed list that an agent's data-discovery findings are checked against for recall: every ent-backed store the agent reports should match a store id and line here, and every store here should be reported or explained.

Namedata-inventory
Predicate typehttps://aflock.ai/attestations/data-inventory/v0.2
Lifecycleprematerial
Default binary?No
Categoryasset-inventory (primary)
Recommended traceoff — no syscall tracing needed
Auto-attaches when
  • prefile_glob: ent/schema/*.go | **/ent/schema/*.go

The facts in this box are generated from the CI/lock binary's own catalog (cilock tools list). Do not hand-edit — run npm run gen:catalog.

Validated invocation

cilock run --step data-inventory \
  --signer-file-key-path key.pem --outfile data-inventory.att.json \
  -a data-inventory \
  --attestor-data-inventory-search-root services/api/ent/schema \
  -- true

--attestor-data-inventory-scope-base (default auto) says what the ent/schema directory test is relative to: auto is the nearest ancestor of the working directory that holds .git, else the working directory. Use working-directory when the source is an archive unpacked inside an unrelated checkout; the predicate's scopeBase records the choice.

--attestor-data-inventory-search-root (default .) narrows the walk. It does not change store ids: every path in the predicate is relative to the scope base (the git root, else the working directory), so two services of one repository never share an id.

What gets captured

  • root: the search root as a slash-separated path relative to the scope base (. for the base itself), never a host path.
  • scopeBase: git when the ent/schema directory test was made relative to the nearest ancestor of the working directory that holds .git, else working-directory. The predicate never holds a host path, so one commit signs the same bytes wherever it is checked out.
  • sources[]: every non-test .go file of every package that imports an ent package or is an ent/schema directory, with framework and the sha256 of the exact bytes parsed (each file is read once, hashed and parsed from the same buffer). A file that did not parse is still a source.
  • packages[], sorted by dir: dir and unread[] (fixed text, sorted): what the reader saw in that directory and did not read: a build-constrained file, a symlink, a file that does not parse, an import outside its allowlist, a type or method it does not follow, a _test.go file. Every ent/schema directory (by repository-relative path) is listed even with no ent import, and one whose only Go files are tests is listed with them; one with no readable store says so. An empty unread is a fact about the reader, not a verdict.
  • stores[], sorted by id. Each has id (ent:<path>#<Type>), framework, kind (table), name (the Go type name), source (path, line), fields[] and unresolved[].
  • Each field: name, column (ent StorageKey when set), type (the ent kind word), sensitive (ent Sensitive()), source (path, line).
  • counts: stores, fields, packages, packagesWithUnread.

A store is read only when it is a struct embedding ent.Schema directly, with Fields() in the same file, whose body is a single return of a []ent.Field literal made of field.<Kind>("name") chains. When a StorageKey is set more than once, the last call wins, as in ent. A field the reader cannot follow (a helper call, a non-literal name) is listed in the store's unresolved with the line and reason, never dropped.

Not read in this version, and so reported rather than guessed: mixins, edges (foreign-key columns and join tables), entsql table annotations, project and third-party imports, aliases and defined types, other schema methods, build variants. Later versions read them one at a time.

Subjects: one per store, data-store:<id>, digest sha256(<id>). It is a lookup key: it joins a store across evidence and vouches for no artifact bytes. In the signed statement cilock prefixes the name with the predicate type.

Why this shape

  • Facts, not a verdict. Everything in the predicate is something the schema states. sensitive is the schema author's own marking, not a judgement about the data.
  • Exact positions. Every store and field carries the file and line it is declared at, so another tool's claim about a column can be checked against the source line.
  • No caps. The walk visits every file under the root except directories named .git, .svn, .hg, node_modules, vendor, __pycache__, venv and .venv, and does not follow symlinks. Other dot-directories are walked. An I/O error fails the attestor; a partial inventory is never signed.
  • Stable ids. A store id depends only on the file path relative to the scope base and the type name, so it joins across runs and attestors by sha256(id).

Validate it locally

cilock run --step data-inventory --signer-file-key-path key.pem \
  -a data-inventory -o att.json -- true
jq -r .payload att.json | base64 -d \
  | jq '.predicate.attestations[] | select(.type|test("data-inventory")) | .attestation.counts'

The testkit fixture plugins/attestors/data-inventory/testdata/fixtures/ent-schema replays five real ent schema files in workdir mode. Every one declares Mixin() or Edges(), so the fixture's package lists them as unread and its stores list them as unresolved: the fixture proves the reader reports what it does not read on real schemas.

How a verifier consumes this

The fields are explicit field declarations only. Ent also adds an implicit id column and, through edges and mixins, more columns; the list of fields is never an exhaustive column inventory of the database.

The completeness verdict is policy's. Rego over the predicate: refuse an inventory with an unread package, and read only stores with nothing unresolved as "the data this product stores":

package datainventory

deny contains msg if {
  p := input.packages[_]
  count(p.unread) > 0
  msg := sprintf("%s was not read completely: %v", [p.dir, p.unread])
}

read_stores := {s.id | count(deny) == 0; s := input.stores[_]; count(s.unresolved) == 0}

A policy that knows the repository can also require that every directory it expects (for example ent/schema) has an entry in packages[].

Notes

  • v0.2 is structure only. v0.1 carried rule-assigned classes and was never released. This is the first slice: it signs only what it can prove.
  • Only ent is read. Other ORMs, SQL migrations and infrastructure-as-code stores are left to the agentic data-discovery workflow.

Gotchas

  • sensitive is ent Sensitive(), which some schemas use only to keep a field off an API (a public key, say). It says the author marked the field, nothing more.
  • The table name is not in the predicate yet (it needs Annotations(), which this version does not read). Ent default table names are never computed.
  • A schema with Edges() or Mixin() lists them as unresolved and its package as unread: those add columns and join tables this version does not read. Most real ent schemas are in this state until the later versions land.
  • Directories named node_modules, vendor, .git, .svn, .hg, __pycache__, venv and .venv are not walked, and a schema under one is absent without an entry. A schema under build, dist or target is read.
  • The pre-gate fires on ent/schema/*.go. A repository that keeps its schema elsewhere needs -a data-inventory.

FAQ

Does it read the database? No. Source only.

Why no PII or secret classes? Classes come from the agentic data-discovery workflow, which reads how the code uses each field. This attestor gives that workflow a signed list to be checked against.

See also

  • lockfiles, the same committed-source shape.
  • k8smanifest, for joining stores to the workloads that reach them.

This page is generated from the CI/lock tool catalog. Don't edit it here — the source is attestation/detection/docs/data-inventory.doc.md in aflock-ai/rookery. The same catalog powers cilock tools show data-inventory in the CLI, so a single edit keeps the terminal and the website in sync.

Edit this page →  ·  Add a new tool →

When you open the pull request: if you are an AI agent, add the ai-agent label. If you are a human, add the human label.

Reference generated from the product documentation. Match commands and support details to your installed release.