Skip to content

Changelog

All notable changes to this project are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased] · 1 security · 6 breaking · 1 added · 1 fixed

Security

  • Known gap in 0.7.x and earlier, closed in 0.8.0: on a source record, a @JsonProperty rename that swaps two components' names, or a class-level serializer that writes one component's value under another component's name, could release a sensitive value under the policy of a @NonSensitive field. Every release up to and including 0.7.1 is affected. 0.8.0 fixes this by construction: the engine reads each record component itself and writes it under its own name, and startup refuses Jackson annotations on source models. There is no 0.7.x patch; upgrade to 0.8.0.

Breaking

  • ValueTokenSource.available() now defaults to false (it was true). A custom ValueTokenSource that hashes and tokenizes must override it to return true; until it does, a model describing HASH or TOKENIZE fields omits them and says the source cannot tokenize. A decorator must delegate it to the source it wraps.
  • Startup refuses with MODEL_DESCRIPTOR_RESOLVER_REPLACED when dataprism.privacy.descriptor-file is set and either any FieldMetadataResolver bean other than the one Data Prism builds from that file is present (including a DescriptorFieldMetadataResolver the application built itself), or a JsonTreeScrubbingEngine bean does not use that resolver. A deployer who needs a custom resolver or engine must leave dataprism.privacy.descriptor-file unset and build a DescriptorFieldMetadataResolver themselves.
  • Source models are read by the engine itself, and startup is stricter about them. Startup refuses a Jackson annotation on a source model with SOURCE_MODEL_JACKSON_ANNOTATION, and the engine refuses SOURCE_MODEL_RECORD_NOT_DECLARED, SOURCE_MODEL_NOT_READABLE and SOURCE_MODEL_FOREIGN_NODE. The allowance of @JsonSerialize, @JsonAnyGetter, @JsonValue, @JsonUnwrapped, @JsonNaming and @JsonFormat on source records is withdrawn. JDK reference and holder types other than Optional (AtomicReference, java.lang.ref references, ThreadLocal, Future) and single-use types (Stream, Iterator, Spliterator, Enumeration) are refused in source models. A pre-built JsonNode holding a POJONode or a non-Jackson node class is refused. An enum is written by name() as a value and as a map key, whatever its annotations. A record at array level two or deeper, or anywhere inside a map value, is now also refused at scrub time with its startup code. A record at an Object, interface or other undeclared position is now refused whether or not it is classified. @JsonAnyGetter is refused: a record source can no longer carry undeclared keys, and extra keys go in a named Map<String, Object> or JsonNode component, which is never descended into, so the unclassified-structure handling applies to it whole (or in a configured JSON source, for per-key handling). Component values are read from fields, not accessors. When reading a source value throws (an exception from the adapter's own collection, map or other object), that source alone is refused with SOURCE_VALUE_READ_FAILED and the other sources still answer: where a caller saw REQUEST_FAILED (a Java caller) or "the request could not be completed" (an MCP client, ToolCalls.java:164-166 on wave4), a caller now gets a response in which that source is marked REFUSED, and sees SOURCE_VALUE_READ_FAILED as the call's refusal (a Java caller) or refused: SOURCE_VALUE_READ_FAILED at <path> (an MCP client) only when no other source answered (trees.isEmpty()) (DefaultContextOrchestrator.java:209-215 on wave4). Errors and checked exceptions from such a read, which escaped unaudited, are now refused and audited. To fix a model refused with SOURCE_MODEL_JACKSON_ANNOTATION, remove the annotation the message names. From 0.8, Data Prism reads a source record itself: each component is written under its own name with its own value, and Jackson annotations on the record, its components, its methods and the enums it holds change nothing. If the annotation renamed a component in the output, rename the component. If @JsonProperty mapped an upstream response's name onto the component, use @JsonAlias for that name instead. If it changed how a value is written (@JsonValue, @JsonSerialize, @JsonFormat), give the component the type you want written: for a single sensitive value, a String with @SensitiveData instead of a record that serialises as one; for a formatted date, a String holding the text. If it hid a component (@JsonIgnore), remove the component or classify it. If it added keys to the object (@JsonAnyGetter), the keys can no longer sit beside the components: hold them in a named Map<String, Object> or JsonNode component. That component is never descended into. Classify it as a whole: unannotated or marked @NonSensitive, the profile's setting for unclassified data decides the whole value, and the bundled DEFAULT profile refuses it. If the keys must be handled one by one, serve that payload as a configured JSON source instead. @JsonAlias, @JsonCreator, @JsonDeserialize, @JsonSetter and @JsonIgnoreProperties(ignoreUnknown = true) stay allowed, because they affect only how an adapter reads the upstream response. To fix SOURCE_MODEL_RECORD_NOT_DECLARED, declare the position as the record type itself, not Object, an interface, Record, a type variable, an unbounded wildcard or a raw collection. To fix SOURCE_MODEL_NOT_READABLE, which can happen only on the module path, add an unqualified opens for the model's package to its module-info.java. To fix SOURCE_MODEL_NOT_A_RECORD on an AtomicReference or another reference type, hold the value itself, or an Optional of it. On a Stream, an Iterator or another single-use type, collect it into a List first. To fix SOURCE_MODEL_FOREIGN_NODE, build the tree from Jackson's own node types and do not put a Java object into it (putPOJO).
  • A source whose value holds more than 1,000,000 nodes (records, elements, map entries, JsonNodes and leaves, each counted once, whatever their size) is refused with SOURCE_TREE_TOO_LARGE; that source alone is refused and the others still answer.
  • A configured JSON source whose response carries a structure (an object or an array) at a root leaf field, one with no nested: of its own, is now refused with NESTED_LEAF_NOT_SCALAR. Before, that case was refused as UNCLASSIFIED_STRUCTURE. A root leaf the profile removes (action REMOVE) is exempt and is dropped as before. A single dataprism.json-sources.config-location value is no longer split at commas; several files are listed as indexed entries.
  • A tool call that breaks the tool's input schema now returns TOOL_INPUT_INVALID, or, on a list_sources call that carries a page key, the link refusal (LINK_INVALID, "The link is not valid."), instead of Tool (<name>) input validation failed: .... The check moved from the MCP SDK into the tool handlers, after authentication, so every such call is now audited once and counted as denied, and a call with a page key counts toward the link limiter. A client that matched the old text must change. A blank entityType or subjectId no longer answers "entityType and subjectId are both required"; it is TOOL_INPUT_INVALID. The served schemas do not change.

Added

  • AuditChainVerifierCli accept-crash-line <audit-directory> --checkpoints <anchor-store-file> --segment <name> --offset <n> --sha256 <hex> (exit code 6 on refusal, see --help). Use it after a purge refuses with AUDIT_RETENTION_CHAIN_UNVERIFIED because a crash left a line that parses but was never hashed, and an investigation finds a crash line, not tampering. It appends one CRASH_LINE_ACCEPTED checkpoint to the anchor store, and purge then tolerates that exact line (segment, offset and the SHA-256 of its bytes) and nothing else. No audit file is changed. The checks do not prove a crash; your investigation decides, so accept a line only after confirming a known crash or power loss of that instance at that time. Someone who can write the audit directory can plant a line that passes every check in order to get it accepted. Stop the server first, run the command as the account that owns the anchor store, and re-run purge until it refuses only at the crash line's day (finish a partly deleted day first). The command refuses, writing nothing, when a BOOT, PERIODIC or SHUTDOWN checkpoint shows the writer got past the line (a power loss on a disk that does not honour fsync can trigger this; extend retention by hand), and when the store holds no BOOT for the line's instance, which in a deployment with a checkpoint sink is a strong sign of tampering. Only deployments with no checkpoint sink, and instances that ran before the store existed, should pass --no-boot-checkpoint, after investigating; it waives that one check, the waiver is recorded in the acceptance (bootWaived), and every later CLI verify run given --checkpoints prints a NOTICE for it (a program calling verify() directly does not see it). These checks refuse a line whose own evidence contradicts the store or the keys; they do not defend against someone who can write the audit directory. In a keyed deployment pass --hmac-key-file. The command reports whether the line's own hash recomputes: KEYED_MATCH <i> (the 1-based position of the matching key), UNKEYED_MATCH, NO_MATCH_LOOKS_CUT (looks like a cut or an edit), NO_UNKEYED_MATCH (no key given; the line looks cut if the deployment is unkeyed) and UNKEYED_ONLY_FORGER_SHAPE (the shape a forger without the key produces; refused). Apart from that refusal the report does not decide acceptance. An anchor store that contains an acceptance cannot be read by 0.7.x or earlier: a 0.7.x verifier stops with an unreadable checkpoint file, and a server rolled back to 0.7.x fails closed, whenever it has an expired day to purge, with AUDIT_RETENTION_CHAIN_UNVERIFIED: could not read the earlier retention anchors.

Fixed

  • Retention purge could erase audit records while verification still passed. Affected: 0.4.0 through 0.7.1. Anyone with write access to the audit directory (but without the chain key or the checkpoint store) could, by appending or editing a line that the verifier skips without hashing, make a purge record a forged retention anchor, or delete a day's records, including records still inside the retention period, with no trace. In 0.8.0 a purge builds anchors and day boundaries only from records the verifier actually hashed. A day is deleted only when every record in it that parses was verified, each writer's next record after the day links to the day's last verified record, and no record is dated after the day in its file's name. Otherwise the purge keeps the day and every later expired day and fails with AUDIT_RETENTION_CHAIN_UNVERIFIED. A day is also kept when a writer's SHUTDOWN checkpoint contradicts its records; the message says to compare the checkpoint store (dataprism.audit.checkpoint.file-path) with the audit directory before purging. The clean-shutdown cover needs a checkpoint sink that implements the SHUTDOWN read-back (AuditCheckpointSink.shutdownCheckpoints()); FileAuditCheckpointSink does, and a custom sink without it falls back to the residual below. A crash can leave a line that looks like a complete record but was never verified. The purge of that day, and of every later expired day, then stops with AUDIT_RETENTION_CHAIN_UNVERIFIED, and the dataprism.audit.retention.unverified metric fires until the operator acts. A crash fragment that does not parse as a record is tolerated and does not stop the purge. Recovery from a complete-looking line is not part of this release. Residual, not closed: if a writer stopped without a clean shutdown (for example after a crash) and never writes again, deleting its last records before a purge is not detected by the purge. Periodic checkpoints (dataprism.audit.checkpoint.interval) are the defence; a writer that shut down cleanly is covered by its SHUTDOWN checkpoint. Also not closed: the purge still deletes an expiring day from which every line of a writer was removed. The full verifier reports that afterwards (MISSING_WRITER, or a non-GENESIS start when the writer survives elsewhere), except when the writer's highest checkpoint is BOOT, when a retention anchor covers it, or when the writer has no checkpoint at all (no checkpoint sink, or a verify run without a checkpoint file); then nothing detects it. This behaviour has been present since 0.4.0, as has the defect above. The defence is an audit directory writable only by the server. Workaround for anyone staying on an affected version: set dataprism.audit.retention to a period longer than the oldest segment, for example P100Y, and keep the audit directory and dataprism.audit.checkpoint.file-path writable only by the server. Plain-Java embedders who never call purge() are unaffected. (No affected version has an on/off switch for the purge: it runs whenever dataprism.audit.directory is set.)
0.7.1 — 2026-10-21 · 3 security · 3 changed · 2 fixed · 5 build

Security

  • A field whose declared type was a user generic type (for example Envelope<Leaf>) was scrubbed, and its prohibited values were collected, with the type argument's rules instead of the field's own type. When the argument type declared the same field names as @NonSensitive, values from the outer type could be released under the DEFAULT profile, and neither output validator caught them. A Map whose values are records was scrubbed as if the map were a record. The defect is present in every release up to and including 0.7.0. It affects deployments whose source models use user generic types or maps of records; models built only from records, scalars, JDK collections, Optional and arrays were not affected by this entry (nested collections are the next entry). Fixed in 0.7.1: upgrade, and change any model that is now refused at startup (see Changed).
  • When a field held an array of arrays (for example List<List<Leaf>> or Leaf[][]), the field's own policy was applied to each inner array whole. A field marked @NonSensitive therefore released every record inside the inner arrays unscrubbed under the DEFAULT profile, and neither output validator caught it. The same happened at request time for an Object or List<Object> field holding nested lists of records. The defect is present in every release up to and including 0.7.0. It affects deployments whose source models hold records inside a collection inside a collection, or whose Object-typed fields can carry nested lists of records; a single level of List, Set, array or Optional around a record was not affected. Fixed in 0.7.1: upgrade, and change any model that is now refused at startup (see Changed).
  • Embedded Tomcat is managed at 11.0.27. Spring Boot 4.1.1 manages Tomcat 11.0.24, which is affected by GHSA-9xv2-5v5q-p794 (CVE-2026-65905), GHSA-gcx9-497g-6cp6 (CVE-2026-65182) and GHSA-h3x4-894j-xpx5 (CVE-2026-68525), all fixed in 11.0.25. Data Prism manages 11.0.27 for the server, the starter and the quickstart images. Data Prism does not configure Tomcat DIGEST or FORM authentication anywhere: requests are authenticated by Spring Security JWT validation, and the quickstart issuer signs the tokens. An application that embeds data-prism-spring-boot-starter and imports its own Spring Boot BOM must manage org.apache.tomcat.embed at 11.0.25 or later itself, ahead of that BOM (docs/extending.md, "The pom shape").

Changed

  • Two new startup refusal codes for source models. SOURCE_MODEL_GENERIC is raised for a generic record, or for a component whose declared type is a parameterized user type. SOURCE_MODEL_MAP_OF_RECORD is raised for a JDK map whose value type contains a record. A JDK wrapper outside collections, Optional and maps (for example AtomicReference<X>) is no longer scrubbed with X's rules; it takes the unclassified-structure path, which refuses under DEFAULT. These deliberately refuse models that 0.7.0 accepted, in a patch release, because those models were not protected.
  • A new startup refusal code, SOURCE_MODEL_NESTED_COLLECTION_OF_RECORD, is raised for a component that puts a record inside a collection or array inside another collection or array (Optional does not count as a level). At request time, a record inside a nested array is refused as an unclassified structure (UNCLASSIFIED_STRUCTURE under DEFAULT). On a field holding nested arrays of scalars, an action other than pass-through now applies to each scalar rather than to each inner array whole; pass-through output is unchanged. Like the entry above, this deliberately refuses models that 0.7.0 accepted, in a patch release, because those models were not protected.
  • Clients that got an MCP protocol error, or "the request could not be completed", when a denial's audit write failed now get an error tool result refused: AUDIT_UNAVAILABLE. AuditUnavailableException is deprecated for removal.

Fixed

  • On both tools (get_entity_context and compare_entity_sources), every denial whose audit write fails now answers refused: AUDIT_UNAVAILABLE and increments the new metric dataprism.audit.denial_unavailable. Before, an orchestrator-side DENY that could not be written surfaced as a generic failure.
  • A re-identification refusal or operator denial whose audit write fails now answers AUDIT_UNAVAILABLE (HTTP 503 on the operator API) and increments dataprism.audit.denial_unavailable, instead of returning the original code with no audit record. A resume that fails after its ALLOW:RESUMED is written now leaves a linked DENY:<code> under the same correlation id, and state stays unchanged.

Build

  • Every GitHub Actions reference is pinned to a full version, and a check on pull requests that touch .github/ confirms each reference resolves.
  • A weekly advisory scan of the resolved Maven dependency trees (all modules, including data-prism-server). High, critical and unrated advisories fail the scan; every ignore must carry a reason and an expiry date. Dependabot now also covers the pip files and load-test.
  • One SBOM build is shared by the GitHub Release and Central: the release attaches the same SBOM files that go in the Central bundle, and a step asserts their SHA-256 values are equal.
  • Each per-architecture image digest reachable from a manifest list is signed and attested, and the image publish fails if any list or platform digest is unsigned. The Central publish is refused unless the GitHub release exists. docs/verifying-releases.md shows the per-architecture and release-SBOM checks.
  • The Jackson 3 override stays at 3.1.8: Spring Boot 4.1.1 still manages 3.1.5, which is below the fixed 3.1.7.
0.7.0 — 2026-10-20 · 11 breaking · 9 added · 9 changed · 7 fixed · 4 security · 6 build

Breaking

The full list, grouped by who has to act, is in docs/migration-0.7.md.

  • get_entity_context response shape. Each sources entry is now { "status": "<CODE>", "entity": { ... } }, with entity present only for ANSWERED, and the top-level entity field is removed. Sources are no longer merged, so two sources that use the same field name for different values are both returned. ContextResponse.sources() is Map<String, SourceResult>; ContextResponse.entity() and the constructors that took it are removed.
  • Per-source refusal and a new status. A source whose scrub or validation refuses is withheld and marked REFUSED, and the other sources are returned. The call is refused only when every source that answered is refused, or the final check over the assembled response fails. The seven status codes are a documented contract, and a client must treat an unknown code as "not answered".
  • Audit logs. A refused source is recorded as REFUSED:<code> under <source>:<refused> (it was REFUSED) and is listed as <source>:REFUSED among the source systems. A log written by 0.7 cannot be read by a reader or verifier older than 0.7. Logs written before 0.7 still verify with the new one.
  • LlmResponseValidator. The final pass now receives a JSON array of the returned sources' trees, not an object. A custom validator that assumes an object root throws, and the call fails closed (DENY:REQUEST_FAILED). Per-source calls still pass one object.
  • SensitivePatternValidator is always on in Spring. The starter, the packaged server and the configured-JSON-sources orchestrator now register it as the PRIVACY_CRITICAL bean dataPrismSensitivePatternValidator. Before, only RawValueLeakValidator ran there. A sensitive-shaped value, for example an email-shaped one, in a @NonSensitive field is now refused with VALIDATION_FAILED / SENSITIVE_PATTERN. Classify such fields correctly; there is no opt-out property.
  • Record and response-size caps are enforced. A response over dataprism.limits.max-records or dataprism.limits.max-response-size that 0.6.x returned is now refused (see Changed).
  • Hazelcast approvals cannot be upgraded in place. Approval entries, id claims and pending entries carry a creation token, and approval entries written by 0.6.x are unreadable and refused. Stop every member and start 0.7; do not run old and new members together, and re-request approvals that were pending.
  • Hazelcast tcp-ip approval quorum. With approvals in use, the approval minimum defaults to a majority of the listed members, so a two-member cluster stops approval-gated tools while either member is down. Set dataprism.hazelcast.approval-minimum-members: 1 to opt out.
  • Hand-wired PrivacyCluster.embedded(ClusterMembership, boolean). The 0.6 overload now applies the default approval quorum whether or not approvals are in use. A hand-wired tcp-ip membership of hostnames that started on 0.6 now throws PrivacyClusterRefusal MISSING_APPROVAL_QUORUM, a wildcard entry gets INVALID_APPROVAL_QUORUM, and an IP-only list now gets a majority write quorum on the approval map. Pass an explicit minimum through the new overload.
  • Cluster refusals now reach callers as their own codes (see Changed): the operator API's 503 OVERSIGHT_UNAVAILABLE and the re-identification REIDENTIFICATION_UNAVAILABLE become CLUSTER_LOCK_TIMEOUT or CLUSTER_OPERATION_REFUSED when the cluster was the cause.
  • RequestLimits bean. An application that supplied its own RequestLimits bean is refused at startup with FORBIDDEN_PRIVACY_OVERRIDE; use dataprism.limits.*.

Added

  • The operator approval view (GET /operator/approvals and GET /operator/approvals/{id}) now carries status: PENDING, APPROVED, REJECTED, CONSUMED or EXPIRED. A stored PENDING past its expiry is shown as EXPIRED.
  • dataprism.limits.max-sources, max-records, max-response-size, per-source-timeout, max-concurrency and scope-read-budget set the request limits that were fixed at their defaults, which are unchanged. The per-source timeout is measured from the start of the fan-out, not per call. Startup is refused with INVALID_REQUEST_LIMITS for a zero or negative limit or a response size that does not fit in an int.
  • dataprism.hazelcast.lock-timeout (default PT2S, PT100MS to PT30S; startup code INVALID_LOCK_TIMEOUT, unconditional).
  • dataprism.hazelcast.approval-minimum-members and split-brain protection on the approval map. Startup codes: MISSING_APPROVAL_QUORUM (applied only when approvals are in use) and INVALID_APPROVAL_QUORUM (an explicit minimum below 1, set with join.mode: none, or above the counted members is refused unconditionally; an uncountable tcp-ip member only when approvals are in use). lock-timeout and approval-minimum-members are refused with CLUSTER_SETTINGS_IGNORED under single-node, and approval-minimum-members also beside an application-supplied PrivacyCluster.
  • dataprism.hazelcast.approval-sweep-interval, a Duration (default PT5M, PT1S to P1D) for how often each member sweeps pending approvals and approval-id claims. A value outside the range is refused with INVALID_APPROVAL_SWEEP_INTERVAL, and the key is refused with CLUSTER_SETTINGS_IGNORED under single-node and beside an application-supplied PrivacyCluster.
  • Refusal codes TOO_MANY_RECORDS, RESPONSE_TOO_LARGE, CLUSTER_LOCK_TIMEOUT and CLUSTER_OPERATION_REFUSED.
  • Metrics dataprism.privacy.source.withheld and dataprism.approval.release.skipped.
  • docs/verifying-releases.md: how to verify the GPG signatures, SBOMs, image signatures and build provenance of a release.
  • docs/migration-0.7.md.

Changed

  • Enforce the configured record and response-size caps before returning a context. Exceeding either refuses the whole call with TOO_MANY_RECORDS or RESPONSE_TOO_LARGE and one correlated DENY:<code> audit event. Equality is allowed. The size cap bounds the compact body returned to the client, including statuses and findings, and for compare_entity_sources its projection; it is not a per-source limit and does not count the MCP envelope.
  • Audit: a refused source is recorded as REFUSED:<code>; see Breaking.
  • compare_entity_sources: identity takes each field from the first answered source, in ascending key order, that holds it.
  • dataprism.privacy.failclosed counts refused calls only; each source withheld from a returned response is counted in dataprism.privacy.source.withheld.
  • A call that waits for a cluster lock now refuses after two seconds by default with CLUSTER_LOCK_TIMEOUT instead of waiting, including a read blocked on the shared read budget. The failed lock wait changes nothing, but steps that ran earlier in the same call are not undone: an approval consumed by admission is not restored, so a retry of an approval-gated tool after a refused read needs a fresh approval. See "What a cluster refusal leaves behind" in docs/multiple-instances.md.
  • A tool call that the cluster refuses an operation for returns CLUSTER_OPERATION_REFUSED (audited DENY:CLUSTER_OPERATION_REFUSED) instead of OVERSIGHT_UNAVAILABLE, and a read whose cluster budget lock the cluster rejects returns refused: CLUSTER_OPERATION_REFUSED at $ instead of SCOPE_READ_BUDGET. Any other budget failure still refuses the read as SCOPE_READ_BUDGET.
  • Re-identification (request, approve and collect) answers CLUSTER_OPERATION_REFUSED or CLUSTER_LOCK_TIMEOUT instead of REIDENTIFICATION_UNAVAILABLE, and the operator API answers 503 with {"code":"<code>"} instead of 503 OVERSIGHT_UNAVAILABLE (approve and reject; pause, resume and the state and approval reads can return them only with a custom OversightState or ApprovalStore) and instead of REIDENTIFICATION_UNAVAILABLE (the re-identification endpoints). Any other failure keeps the old code. The cause is logged once at WARN with the cause's class name only.
  • A Hazelcast approval create that the cluster rejects (for example a configured hazelcast.lock.max.lease.time.seconds below the lease) is refused with CLUSTER_OPERATION_REFUSED instead of an IllegalArgumentException that looked like a duplicate id. Two maps are added, dataprism.approval.id and dataprism.approval.pending.
  • ClusterOperationRefusedException is in io.github.aindriub.dataprism.core.refusal. It was never released, so there is no alias.

Fixed

  • An approval can no longer be consumed twice across members: consumption is a compare-and-set, not a lock followed by a check.
  • After a split cluster heals, an approval either side consumed, rejected or expired stays so (a merge policy on the approval map). Previously the larger side's APPROVED could win.
  • A cluster lock held by another member or by something on the member port no longer hangs request threads. Every wait is bounded and refuses with CLUSTER_LOCK_TIMEOUT.
  • Hazelcast approval creation no longer relies on locks that never expire. The pending cap per requester and the uniqueness of an approval id across scopes hold through atomic map operations, so a creating member that stalls past its lock lease can no longer let a requester exceed the cap or one id exist in two scopes. Every cluster lock is leased for five times dataprism.hazelcast.lock-timeout. Pending entries carry no TTL; a periodic sweep bounds their retention.
  • Every Hazelcast approval creation carries a random token that every release, removal, withdrawal, undo, takeover, sweep and merge compares, so a clean-up that belongs to one creation of an id can no longer strip the claim or pending slot of a later creation of the same id in the same scope. An approval is usable only while the claim for its id names its entry and token.
  • Hazelcast approvals: approving or rejecting no longer waits for ever, or reports failure after it has committed, when the requester's pending entry is locked; the release is bounded and an over-count is left for the sweep. A decision commits its new value and its expiry in one step. Ending a scope or forgetting its approvals is bounded and refuses with CLUSTER_LOCK_TIMEOUT, leaving the approvals it could not lock in place. The pending sweep skips keys that are locked.
  • The key scan in the leak validators honours the match cap and the character budget; 1,000 sensitive keys no longer give 1,000 matches.

Security

  • Jackson 3 (tools.jackson) is managed at 3.1.8, ahead of the Spring Boot 4.1.1 BOM, fixing GHSA-wv8q-qhhj-9h54, GHSA-cxp5-3px4-pw24, GHSA-q4xh-88c3-wmh7, GHSA-wjgm-6hv5-3cvf and GHSA-gx83-3vf8-gh7j in the published artifacts and images. An application that uses the Data Prism libraries with its own Spring Boot 4.1.1 BOM must import tools.jackson:jackson-bom 3.1.8 or later itself, ahead of the Boot BOM, because the override does not reach a consumer's build. The consumer snippet in docs/extending.md now does.
  • The response leak validators scan property names as well as values, so a map keyed by an email address or other identifier is refused even under a Java-built PASS_THROUGH_UNSAFE profile. A flagged name is reported as <undeclared>. A key that matches no known identifier shape still passes.
  • For all five images (the server and the four quickstart images), each image's manifest list and per-architecture build index are signed keylessly with cosign, and images, jars and release assets carry GitHub build-provenance attestations. See docs/verifying-releases.md.
  • A failed audit write on an approve could leave a usable but unaudited approval, because the undo could itself fail. Re-identification and tool-call approvals now write ALLOW:APPROVED first and commit second, so a failed write leaves the approval PENDING, and a decision that then does not apply is followed by a DENY with the same correlationId. New metrics dataprism.approval.decision.audit_unavailable and dataprism.approval.decision.unconfirmed.

Build

  • Release builds attach a CycloneDX SBOM to every module published to Maven Central and to the GitHub Release. Images are pushed with BuildKit SBOM and provenance attestations.
  • Added a report-only mutation Maven profile (PIT) and a weekly and manual mutation workflow over the core, pseudonymisation, validation and orchestration modules; it sets no threshold and never fails a build.
  • Added load-test/, a standalone load-test harness (outside the root reactor) with scenarios for 8, 32, 100 and 300 sources and a load-test workflow.
  • Pin the build to Maven 3.9.16 through the Maven Wrapper; CI and release workflows now call ./mvnw. Maven 3.10.0 on the runner image broke the Central bundle.
  • The Central publish workflow now fails unless Maven is 3.9.x, in both the stage and publish jobs.
  • Dependabot no longer proposes moving the Maven Wrapper to Maven 3.10 or later.
0.6.0 — 2026-10-09 · 14 breaking · 6 added · 4 changed · 2 fixed · 4 build

Breaking

0.6.0 is a clean break. There are no deprecated overloads and no forwarding types at the old names, because there are no external users to carry through a deprecation cycle. Every consumer recompiles and changes imports. The full old-to-new tables, grouped by who has to act, are in docs/migration-0.6.md.

  • The core package split. The root package io.github.aindriub.dataprism.core is now empty and its 34 types moved to core.spi, core.model, core.engine, core.refusal, core.limits and core.metrics. core.spi is the extension-facing package: it holds the interfaces an adapter or integration implements (DataSourceAdapter, IdentityResolver, FieldMetadataResolver, ScrubbingEngine, SecretKeyProvider, SyntheticValueSource, ValueTokenSource, EntityCorrelationService) and the request types they exchange (DataRequest, SourceCallContext), plus PassThroughIdentityResolver.
  • The audit package split. The contract stays in io.github.aindriub.dataprism.audit, and 15 types moved to audit.format, audit.sink, audit.checkpoint, audit.retention and audit.verify.
  • Operator-visible: the verifier command changed. The main class is now io.github.aindriub.dataprism.audit.verify.AuditChainVerifierCli. The old name io.github.aindriub.dataprism.audit.AuditChainVerifierCli no longer exists, so any script, cron job or runbook that runs java -cp <classpath> io.github.aindriub.dataprism.audit.AuditChainVerifierCli fails until the name is changed. Arguments and exit codes are unchanged.
  • The MCP tool constructors, the DataPrismMcpServer factories, ContextRequest and SourceFanOut lose their overloads. GetEntityContextTool and CompareEntitySourcesTool have one constructor each, DataPrismMcpServer.stdio and streamableHttp have one method each, and they take a ToolOptions record (admission, fingerprinter, correlation requirement, MDC, audited entity types). There is no default admission: name noAdmission() or a real policy. SourceFanOut takes a SourceFanOutOptions, and ContextRequest keeps its canonical constructor, of and comparison. A request built through the removed shorter constructors used to derive its audited entity type from the shape fallback; of and comparison audit <unregistered>.
  • The nested DataPrismProperties types are now 13 top-level classes in io.github.aindriub.dataprism.spring.boot (TransportProperties, SecurityProperties, AuditProperties and so on). DataPrismContractValidator is now public in spring.boot.validation, beside the new DataPrismPropertiesValidator.
  • JwtDecoderSupport and JwtCallerContextExtractor moved to io.github.aindriub.dataprism.spring.boot.jwt.
  • DataPrismAutoConfiguration keeps its FQCN but no longer declares beans. It imports 14 package-private configuration classes. Operators who set log levels on ...DataPrismAutoConfiguration$AuditSinkSelection or ...DataPrismAutoConfiguration$JsonProjection must use io.github.aindriub.dataprism.spring.boot.AuditSinkSelection and io.github.aindriub.dataprism.spring.boot.JsonProjection, and the logger of JwtCallerContextExtractor moved with its package. The bean names of the configuration classes themselves changed with their class names.
  • Jackson 3. The classpath is Jackson 3 (tools.jackson) and the Jackson 2 artifacts are banned in this build; jackson-annotations stays. The MCP SDK binding is mcp-json-jackson3 (was mcp-json-jackson2) and the Spring side is spring-boot-starter-jackson (was spring-boot-jackson2). Public signatures that carried Jackson 2 types now carry Jackson 3 ones: SourceTree (and SourceTree.text now returns StringNode), ScrubResult, Generalizer, ContextResponse, ComparisonResponse.identity, LlmResponseValidator.validate, RawValueLeakValidator.validate, SensitivePatternValidator.validate and SensitiveDataScanner.scan. DataPrismObjectMapper and its create() are no longer public, and the two tool constructors no longer take an ObjectMapper.
  • Source models must be records. A DataSourceAdapter response type that is not a record, or that holds a bean, a user collection or map subclass, or a class-level serializer on a non-record at any depth, is refused with SOURCE_MODEL_NOT_A_RECORD (at startup for declared types, at request time for the actual object graph). Records are read by their components only. java.time values and Optional are now accepted: java.time is written as ISO-8601 text and an Optional is unwrapped. Jackson 2 refused both.
  • The YAML configuration readers (SecurityPolicy, PrivacyProfiles, ModelDescriptors, RestSources, VocabularyRegistry and ConfiguredJsonSources) refuse at startup what 0.5.x accepted, so a configuration that loaded on 0.5.x may now refuse to start. The nine codes are DUPLICATE_CONFIG_KEY, UNKNOWN_CONFIG_KEY, NON_STRING_CONFIG_SCALAR, INVALID_CONFIG_BOOLEAN, LEADING_ZERO_CONFIG_NUMBER, TRAILING_CONFIG_CONTENT, UNSUPPORTED_CONFIG_YAML (aliases, anchors and tags), INVALID_CONFIG_SHAPE and NULL_LIKE_CONFIG_SCALAR. A present but null value is refused, and an empty tls: is refused where it used to disable requireHttps. Quoting rule: a value in a string-typed field that YAML reads as a number or boolean, such as 1.0, 1e3, 42 or true, must be quoted. Messages name the key and its path (a key is cut to 64 characters) and never the value.
  • YAML is now parsed as YAML 1.2 (Jackson 3). yes, no, on, off, True, FALSE and any other spelling than lower-case true/false are text, not booleans: in a boolean-typed field they are refused as INVALID_CONFIG_BOOLEAN, and in a string-typed field they now load as the text you wrote, where 0.5.x loaded true or false. Leading-zero numbers are no longer octal: they are refused in numeric fields (band bounds, vocabulary versions) as LEADING_ZERO_CONFIG_NUMBER, and in a string-typed field 010 now loads as the text 010, where 0.5.x loaded 8. Neither string-field change is refused, so check string fields that held such a value.
  • Error messages no longer echo configured URLs or credentials (ConfiguredJsonSources, RestSources, RestSource and the json-sources initializer). A config location in a message is shown as a placeholder, a local path cut at ? or #, or scheme://host[:port].
  • A JSON audit projection (dataprism.audit.output.json-directory) now needs the built-in hash-chained sink. An application AuditSink bean combined with dataprism.audit.sink=hash-chained and a json-directory refuses to start with AUDIT_JSON_PROJECTION_WITHOUT_BUILT_IN_SINK, and a json-directory with no registered projection refuses with AUDIT_JSON_PROJECTION_MISSING.
  • A tool call whose response cannot be converted for the MCP result now returns "the response could not be serialised"; on 0.5.x a failed conversion fell through to "the request could not be completed".

Not changed: the dataprism.* property names, the io.github.aindriub.dataprism.spring.boot.DataPrismAutoConfiguration FQCN and its entry in AutoConfiguration.imports, the existing refusal codes, the audit record format (recordVersion 3) and the dataprism.audit logger name.

Added

  • Added AuditEventListener, a read-only SPI called after the configured audit sink accepted each event, on one bounded asynchronous dispatcher (dataprism.audit.listeners.queue-capacity, default 1024). It is best effort: events are dropped for listeners only when the queue is full, and the audit log is unaffected. Failures are logged by class name only as AUDIT_LISTENER_FAILED, and drops as AUDIT_LISTENER_DROPPED.
  • Shutdown of the audit listener dispatcher is bounded. On close it drains queued events for up to 5 seconds, then counts and logs the rest as dropped and interrupts the dispatcher. The final drop report is written on a short-lived thread joined for at most 1 second, so a blocked listener or appender cannot hang shutdown, and closing from an already interrupted thread still drains, joins and restores the interrupt flag.
  • The data-prism-spring-boot-autoconfigure jar now ships Spring configuration metadata (META-INF/spring-configuration-metadata.json), so IDEs complete and describe dataprism.* keys in YAML. The processor runs only at build time and is not a dependency, and a build test keeps the metadata and docs/configuration.md in step.
  • New startup refusal codes, each failing closed: INVALID_AUDIT_LISTENER_QUEUE_CAPACITY (dataprism.audit.listeners.queue-capacity below 1), AUDIT_EVENT_LISTENERS_NOT_REPLACEABLE (a second dispatcher bean), SOURCE_MODEL_NOT_A_RECORD, AUDIT_JSON_PROJECTION_WITHOUT_BUILT_IN_SINK, AUDIT_JSON_PROJECTION_MISSING and the nine YAML codes listed under Breaking.
  • The offline verifier reports a damaged checkpoint line as the anomaly TORN_CHECKPOINT_LINE (exit code 4), never as a break.
  • TeeAuditSink is Closeable. It closes the JSON projection and then the primary sink, and closes the primary even if the projection close throws.

Changed

  • get_entity_context's sources entries are now in ascending key order (the alias, or the real name for a caller holding EXPOSE_SOURCE_NAMES), where before the order varied between JVM runs.
  • ToolAdmission.none() returns one shared instance, and ToolOptions.defaults().admission(ToolAdmission.none(), null) is accepted without a fingerprinter. Any other admission still needs one.
  • The JSON audit projection is its own bean, dataPrismJsonAuditProjection, classified PRIVACY_CRITICAL with COMPETING_BEAN_REFUSAL. It is deliberately not an AuditSink, because that made injecting an AuditRecorder ambiguous. The hash-chained sink bean resolves the projection before it opens the authoritative file, and closes the authoritative file again if the tee cannot be built.
  • INVALID_MODEL_DESCRIPTOR_FILE now chains a value-free cause when the reader's message carries its own code (for example UNKNOWN_CONFIG_KEY), so the operator learns which rule the file broke. Uncoded, enum and I/O messages are not chained.

Fixed

  • TeeAuditSink now poisons itself on any Throwable thrown by the JSON projection after the primary write, including an Error and a sneakily thrown checked exception, and rethrows the original unchanged. Before, only a RuntimeException from the projection did.
  • A resumed checkpoint writer now terminates a torn tail (a checkpoint file not ending in a newline) with \r\n and fsyncs before its first append, and fails the open if it cannot. The verifier reads such a line, or a final unterminated chunk that is not a checkpoint, as TORN_CHECKPOINT_LINE rather than failing, and still uses the intact checkpoints. Any other unparseable checkpoint line is still unreadable input (exit code 1).

Build

  • ci-gate is a single summary check in build.yml that is green only when the build matrix and container-smoke all succeeded. The Pages build job is renamed docs-site.
  • build.yml now builds the Javadoc with the release profile (unsigned, no secrets) on every pull request, on the JDK 21 leg. 0.5.0 would otherwise have found a Javadoc error only at publish time, because release.yml runs a plain mvn verify.
  • spring-boot-configuration-processor runs on the autoconfigure module's annotation-processor path only. maven-dependency-plugin is managed at 3.11.0.
  • ArchUnit now locks the new layout: an empty core root, core.spi not depending on core.engine, acyclic core and audit subpackages, audit and oversight under the core outer-layer rule, validation and jwt independent of each other, and rules for obtained mappers and streaming JSON factories.
0.5.0 — 2026-10-08 · 1 breaking · 8 added · 4 changed · 2 fixed · 1 upgrade notes

Breaking

  • The starter and auto-configuration now require Spring Boot 4.1 (4.1.1, Spring Framework 7.0.9). Spring Boot 3 users stay on 0.4.x. The library jars still run on Java 21 or newer.

Added

  • Inbound correlation id. A validated external correlation id can arrive on MCP tool calls through a configured HTTP header (dataprism.correlation.inbound.*). The strict default pattern accepts only a UUID, 16 to 128 hex characters containing at least one letter a to f, or a W3C traceparent. Digit-only values, such as card numbers, are refused. A broader pattern is available only by explicit configuration. It is off unless a header is set.
  • Outbound correlation header. REST sources send the id as a header. It is off unless configured, and a call without an id sends nothing.
  • dataprism.correlation.outbound.header, an optional global outbound header. A per-source correlation-header overrides it.
  • dataprism.correlation.mdc-key, unset by default. When set, it puts only the validated id into the SLF4J MDC on the tool handler thread and on each fan-out task, and clears it afterwards. Warning: whatever the inbound pattern admits then appears in every log line written on those threads, so keep the pattern to generated ids.
  • A JSON/ECS audit projection, written through a tee, with dataprism.audit.output.* configuration (field preset, field names, routing constants and an optional NDJSON json-directory). The hash-chained .log stays authoritative and the JSON is a separate projection. ECS event.outcome is derived from policyDecision, and the raw decision is kept beside it.
  • dataprism.audit.entity-types, an optional list of the entity types that may appear verbatim in an audit record.
  • New startup refusal codes, each failing closed: INVALID_CORRELATION_FORMAT, INVALID_CORRELATION_PATTERN, CORRELATION_PATTERN_NOT_APPLICABLE, INVALID_CORRELATION_HEADER, CORRELATION_REQUIRED_WITHOUT_HEADER, CORRELATION_REQUIRES_HTTP_TRANSPORT, INVALID_CORRELATION_MDC_KEY, CORRELATION_MDC_KEY_RESERVED, CORRELATION_MDC_KEY_WITHOUT_HEADER, INVALID_AUDIT_FIELD_PRESET, INVALID_AUDIT_FIELD_PATH, UNKNOWN_AUDIT_FIELD, AUDIT_FIELD_MAPPING_CONFLICT, INVALID_AUDIT_ROUTING_VALUE, AUDIT_JSON_REQUIRES_SEGMENTED_SINK, AUDIT_JSON_DIRECTORY_SAME_AS_AUDIT, AUDIT_JSON_DIRECTORY_UNUSABLE and INVALID_AUDIT_ENTITY_TYPE.
  • New runtime codes, which are not startup refusals: a call refused and audited before any source is called with EXTERNAL_CORRELATION_ID_REQUIRED or EXTERNAL_CORRELATION_ID_INVALID when inbound.required is true; EXTERNAL_CORRELATION_ID_DROPPED logged at WARN when an invalid inbound id is dropped and the call proceeds without one; and AUDIT_PROJECTION_FAILED, which fails later audited calls closed after a JSON projection write fails, until the process restarts. The verifier reports FIELD_COUNT_MISMATCH and INTERRUPTED_WRITE_FRAGMENT as anomalies, not as startup refusals.

Changed

  • The published images run Java 25. Library bytecode is still Java 21 (--release 21, class major 65).
  • Audit record version 3. Records are written as recordVersion 3 with 25 fields, and externalCorrelationId is inside the hash. Version 1 and version 2 records still verify. The offline verifier reports a line with too many fields, or a full-length line that declares the wrong version, as FIELD_COUNT_MISMATCH, a break (exit code 2). A shorter line that could be a torn write is reported as an interrupted write (exit code 4), not a break.
  • The audited entityType. entityType is audited verbatim only when it is a registered entity type or an upper-case-identifier-shaped token; otherwise it is recorded as <unregistered>. With no list configured, an upper-case token still passes, so set dataprism.audit.entity-types.
  • Build and tests. Test ports are claimed by an OS file lock and probes run sequentially, to stop port-race flakes. Dependabot version updates are grouped into one pull request per ecosystem per week, and Java image majors and non-LTS Java 26 and later are ignored.

Fixed

  • An interrupted audit write is no longer reported as tampering after a restart. A writer that opened a file or segment ending in an unterminated fragment appended its first record onto that same line, and the verifier reported the over-long line as FIELD_COUNT_MISMATCH (exit code 2). A resumed writer now writes \r\n and fsyncs before its first record when the last byte is not a newline. It never truncates or rewrites existing bytes, and it fails the open (AUDIT_SINK_OPEN_FAILED) if it cannot do so. The verifier reads a line ending in a raw carriage return, which a serialized record never contains, as an INTERRUPTED_WRITE_FRAGMENT (exit code 4) and does not parse it. Directory mode uses the same terminator. The rule that an over-count line with a parseable version is a break is unchanged. One live writer per file or segment directory is assumed.
  • The configured outbound correlation header (including traceparent) is now removed from every outbound REST request, so a client-level default or preset header can no longer send an unvalidated value when no valid id is present.

Upgrade notes

  • Logs written before 0.5.0, including every 0.4.x release, can already hold a torn fragment fused with a restarted writer's first record on one line. The 0.4.x verifier mostly reported these as INTERRUPTED_WRITE_FRAGMENT (exit code 4). The 0.5.0 verifier reports them as FIELD_COUNT_MISMATCH (exit code 2), and retention stops there. To check, see whether the trailing 20, 24 or 25 fields of the line parse as a GENESIS record of a new instanceId, and whether the record after the line continues that writer. If so it is a legacy interrupted write, not an edit.
0.4.1 — 2026-10-07 · 2 breaking · 3 security · 6 added · 1 fixed · 2 known limitations · 3 upgrading starter consumers to hazelcast 5.7.0

Real, explicit clustering. In 0.4.0, topology: embedded started a bare Hazelcast member that was never configured to join anything, so separate instances never shared pause, approvals, the read budget or rate limits, even though the docs said they did. This release lets members join one cluster by explicit configuration, refuses to start without it, and says plainly what is and is not shared and protected. Member traffic is not encrypted and members do not authenticate each other: open-source Hazelcast has neither.

Breaking

  • topology: embedded now refuses startup unless dataprism.hazelcast.cluster-name and dataprism.hazelcast.join.mode are set (MISSING_CLUSTER_NAME, MISSING_CLUSTER_JOIN), and the cluster name must not be dev (RESERVED_CLUSTER_NAME). The migration for a single member is one line: dataprism.hazelcast.join.mode: none (plus a cluster-name), an explicit single member bound to loopback that neither discovers nor accepts other members. A multi-instance deployment chooses tcp-ip or kubernetes instead. 0.4.0 embedded members never clustered, so no working multi-instance deployment breaks; only the configuration needs editing.
  • dataprism.hazelcast.tls-key-reference and tls-trust-reference now refuse startup with HAZELCAST_TLS_UNSUPPORTED. They were read by nothing in 0.4.0.

Security

  • Hazelcast auto-detection, multicast and phone-home are always disabled, so a member can no longer join an unrelated cluster named dev by discovery. PrivacyCluster also refuses an enabled advanced network config and an application-supplied instance that has auto-detection or multicast on (UNSAFE_HAZELCAST_DISCOVERY). Application PrivacyCluster beans are validated, and cluster properties set beside one refuse with CLUSTER_SETTINGS_IGNORED.
  • Member TLS is unsupported, because open-source Hazelcast 5.7.0 has no member TLS engine and no member authentication. Member traffic is not encrypted. Network isolation (a private network, a NetworkPolicy or an mTLS mesh) is the deployer's responsibility, and member.interface should pin the member to that network. Without it, a tcp-ip or kubernetes member binds every interface.
  • The server ships, and the starter declares, Hazelcast 5.7.0. 0.4.0 shipped 5.5.0, because Spring Boot's dependency management overrode the declared version; the root pom now pins 5.7.0 ahead of the Boot BOM. A starter consumer must pin it too (see "Upgrading starter consumers to Hazelcast 5.7.0" below). 0.4.0's multi-member behaviour was only ever exercised on 5.5.0.

Added

  • dataprism.hazelcast.cluster-name, join.mode (tcp-ip, kubernetes or none), join.members, join.kubernetes.namespace, join.kubernetes.service-name or service-dns, member.port (default 5701) and member.interface, with startup refusals for missing or invalid values and for a member port shared with another listener (CLUSTER_PORT_SHARED).
  • docker/multi-instance/: a two-member Docker Compose example and a Kubernetes manifest (DNS mode first; API mode, with its own ServiceAccount and Role, commented).
  • A new "multiple instances" page, docs/multiple-instances.md, covering what members share, what they do not, and the residual risk that losing an entry's owner and its backup together loses the entry (backup count is 1).
  • EXPOSE 5701 in the server image. It is for private networks only; do not publish it, for example with docker run -P.
  • server.json gains the cluster variables (DATAPRISM_HAZELCAST_CLUSTERNAME, join and member settings) and the operator variables (DATAPRISM_OPERATOR_*).
  • A multi-member test that runs real server members and checks pause, approvals, the read budget, rate limits and re-identification are shared, and that refusals survive losing the member that owns the pause key.

Fixed

  • The docs claimed cluster-wide sharing, member TLS and member validation that did not exist. The configuration, EU AI Act support, architecture and re-identification pages now say what clustering shares and what it does not.

Publication

0.4.1 publishes the library modules to Maven Central (not data-prism-server), the quickstart images and the server image (ghcr.io/aindriub/data-prism-server) to GHCR, and the server entry to the MCP Registry.

Known limitations

  • Anyone who can reach the member port can read and write cluster state. They can read subject ids, forge an APPROVED approval and bypass four-eyes, lift a pause by deleting its flag, and reset read budgets and rate limits. Member traffic is neither authenticated nor encrypted on open-source Hazelcast, so isolating the member port is the deployer's job. This includes a plain Hazelcast client configured with the cluster name: it can connect to a tcp-ip or kubernetes member's port and read or write maps. A none member refuses both members and clients.
  • The Docker Compose example's cluster network is not a security boundary. The members bind and advertise only their cluster address, but OrbStack was observed to route across Docker networks, so a container on another network reached port 5701 on that address. Docker Desktop may behave the same (not tested). Linux Docker's network isolation is expected to block it (not tested here). Isolate 5701 with a host firewall, a NetworkPolicy or a private network in production.

Upgrading starter consumers to Hazelcast 5.7.0

The starter declares Hazelcast 5.7.0, but Spring Boot's dependency management in your build overrides that declaration, so a starter consumer gets 5.7.0 only by saying so:

  • With spring-boot-starter-parent: set <hazelcast.version>5.7.0</hazelcast.version> in your <properties>.
  • When importing spring-boot-dependencies: add your own <dependencyManagement> entry for com.hazelcast:hazelcast:5.7.0. The property override does nothing for an imported BOM.
  • With Gradle and the Spring dependency-management plugin: ext['hazelcast.version'] = '5.7.0'.

See "Using the starter" in docs/multiple-instances.md.

0.4.0 — 2026-10-06 · 10 breaking and behaviour changes · 7 added · 3 changed · 2 security

Human oversight, a separate operator surface, audited re-identification, and an audit trail with daily segments, external checkpoints and retention. These features support an operator's work toward the human-oversight, record-keeping and special-category obligations in the EU AI Act and GDPR Art. 9; they do not by themselves establish that a deployment meets those obligations. Several defaults and refusals change (see "Breaking and behaviour changes").

Breaking and behaviour changes

  • Release note (task 94): a PHI rule weaker than REDACT now refuses to start with SPECIAL_CATEGORY_EXPOSED, and a Java-built profile with no PHI rule resolves PHI to REMOVE. Seven GDPR Art. 9 special-category classifications fail closed. Check any custom profile before upgrading.
  • All denials are recorded in the audit trail as DENY:<code>, from both the orchestrator and the MCP tools. A malformed code is recorded as DENY:INVALID_REFUSAL_CODE. Anything that filters audit records on the old denial value must change.
  • Audit record version 2: the hash is computed over an unambiguous, length-prefixed encoding and the record carries per-field dispositions and approval identity. Version 1 records still verify.
  • A configured audit retention below 6 months refuses startup unless dataprism.audit.retention-override is set.
  • Audited calls are refused with AUDIT_CHECKPOINT_UNAVAILABLE while an audit checkpoint cannot be written.
  • ContextOrchestrator.buildContext now throws AuditedRefusalException (a PrivacyRefusedException) with code REQUEST_FAILED for audited internal failures. Starter users who map PrivacyRefusedException to 403 should check code().
  • The MCP server always enforces admission (pause, rate limit, approval). Four-eyes defaults to on for re-identification (dataprism.reidentification.four-eyes=true). Tool calls need an approval only when the tool is listed in dataprism.oversight.approval-required-tools, which is empty by default. There are at most 5 live pending approvals per requester and kind, by default (TOO_MANY_PENDING); the oversight and re-identification caps are configured separately.
  • data-prism-server now excludes Spring Boot's HazelcastAutoConfiguration. This is visible only with a hazelcast.xml or hazelcast.yaml on its classpath.
  • Refusal codes returned by application scrubbers, validators and resolvers are validated before they reach the client; a malformed one becomes INVALID_REFUSAL_CODE.
  • The MCP serverInfo version is read from the build rather than hardcoded.

Added

  • Human oversight: pause (cluster-wide with Hazelcast), per-caller rate limits, approvals bound to the call's purpose, profile, client and capabilities, and four-eyes approval, with in-memory and Hazelcast implementations.
  • The operator surface, served on dataprism.operator.port in the same process with its own filter chain; the MCP and operator endpoints never share a port. It carries audited pause, approvals and re-identification.
  • The data-prism-reidentification module and index: audited, purpose-bound re-identification with optional four-eyes approval, wired over the application's synthetic value source.
  • External audit checkpoints (BOOT, PERIODIC and SHUTDOWN) and audit retention that purges expired segments behind RETENTION_ANCHOR checkpoints; the verifier gains directory mode and --checkpoints, which exits 5 on tail truncation or a missing boot.
  • Opt-in daily audit segments named audit-YYYY-MM-DD.log, written when dataprism.audit.directory is set. dataprism.audit.file-path still writes a single file.
  • Per-field dispositions in the audit record, and a correlationId returned in the MCP tool result _meta.
  • The EU AI Act and GDPR Art. 9 support page, mapping what the project supports to each provision, and a table of every policyDecision form in docs/audit.md.

Changed

  • Undeclared payload keys render as <undeclared> and bracketed indices as [*] on every refusal and warning sink; undeclared property names never reach the model (<undeclared-N> under REDACT_AND_WARN, dropped under DROP_AND_WARN) and are not scanned by validators.
  • InMemoryApprovalStore now refuses non-pending, duplicate and null-approver approvals exactly as the Hazelcast store does.
  • Integration tests derive the artifact version from the build.

Security

  • Special-category (Art. 9) data fails closed: see the release note above.
  • Refusal paths no longer leak undeclared payload keys, and refusal codes from application code cannot carry free text to the client.

Publication

0.4.0 is published to Maven Central (the library modules; data-prism-server stays off Central and comes from source, a GitHub Release or GHCR), to GHCR, and to the MCP Registry.

0.3.1 — 2026-09-24 · 3 added · 3 changed · 1 fixed

No runtime behaviour changed on the server. This release refreshes the docs site and the MCP Registry listing, and corrects several documentation and CHANGELOG claims found while writing them.

Added

  • A MkDocs Material docs site published from the existing user docs, with an identity (mark, favicon, palette, landing page), six concept diagrams embedded in context, and dedicated FAQ, comparison and three intent use-case pages.
  • A developer guide overview with tutorials for writing an adapter and writing a custom IdentityResolver, both checked against the code they document, plus an example IdentityResolver and its unit tests in data-prism-quickstart-extension.
  • websiteUrl and the current canonical tagline in the MCP Registry server.json entry, so the next registry publish carries them instead of the stale v0.3.0 description and missing websiteUrl.

Changed

  • The changelog page on the docs site now collapses every release but the latest, without editing CHANGELOG.md itself.
  • CHANGELOG: corrected the 0.3.0 AuditChainVerifier entry, which attached "caught even on the last record" to deletions as well as edits; only edits are caught at a chain's tail, a deletion is caught only when a later record follows it (see docs/audit.md).
  • Adopted a new canonical project description in README.md and the root pom.xml, replacing v0.3.0's "A privacy layer between MCP clients and enterprise APIs.": it now says Data Prism pseudonymises personal data per privacy scope and refuses anything unclassified. The same text is the abstract of the new CITATION.cff.

Fixed

  • docs/tools.md's claim that an unclassified field is "dropped"; the shipped DEFAULT/STRICT profiles both refuse the whole request instead.

Not published to Maven Central; the Maven artifacts remain at 0.3.0.

0.3.0 — 2026-09-23 · 6 added · 3 changed · 3 fixed · 4 behavioural changes · 1 not changed

Nested JSON catalogues, one level deep, and a durable, hash-chained audit trail — protect a real API without writing Java, and prove what happened.

Added

  • The configuration-driven JSON REST connector gains one level of named nested catalogues: a field can declare nested: <name>, pointing at an entry in a top-level nested-catalogues map whose own leaves may be nonSensitive or classified only — a nested leaf carries no identifier of its own (it inherits its subject from the enclosing record), so identifier: true and a further nested: are both refused there. No dotted paths, no JSONPath, no wildcard descent, no inferring structure from the wire — subject-json-path is untouched and a nested object never carries its own subject. A response nesting deeper than declared — a leaf the catalogue says is a scalar turning up as a structure — refuses with the new, distinct NESTED_LEAF_NOT_SCALAR code rather than falling through to core's generic UNCLASSIFIED_STRUCTURE; the mirror case, a declared structure that turns up as a scalar, refuses with NESTED_FIELD_NOT_STRUCTURED.
  • A durable, append-only, hash-chained audit sink. Selecting dataprism.audit.sink: hash-chained, alongside the now-required dataprism.audit.file-path, produces a FileAuditSink bean: one file, fsync per record, no rotation. Omitting the file path refuses at startup with MISSING_AUDIT_FILE_PATH; a path this process cannot open refuses with AUDIT_SINK_FILE_UNUSABLE, instead of degrading silently to no auditing. The canonical audit-record hash covers nineteen fields, including timestamp and sourceSystems.
  • An offline AuditChainVerifier CLI replays a hash-chained audit file's writers from a copy and reports one of five outcomes: exit 0 intact; 1 unreadable input; 2 a detected break — an edit inside one writer's chain, caught anywhere including that chain's own last record, or a deletion caught only when a later record follows it in the same chain; 3 the final record has no terminating newline, reported as possibly in flight, which is proof of neither health nor tampering; and 4 a structural anomaly (an interrupted-write fragment, a duplicate sequence, or a chain not starting at GENESIS immediately after another anomaly), never returned together with a break. Exit codes 3 and 4 name shapes ordinary operation can also produce; neither rules out tampering. The verifier cannot detect truncation of a writer's most recent records at all — a file with its tail removed verifies intact at exit 0, because nothing remains in the file to disagree with — and the same blind spot extends to a whole process boot: deleting every record of one boot leaves the surviving writers reporting intact and never mentions the deleted one, indistinguishable from that boot never having run.
  • An opt-in dataprism.identity.resolver: pass-through property selects PassThroughIdentityResolver, refusing an unrecognised value with UNSUPPORTED_IDENTITY_RESOLVER; an application-supplied IdentityResolver bean still wins. Combined with the configuration-driven JSON REST connector, this makes protecting a flat JSON API genuinely possible with no Java class. A Spring FailureAnalyzer now renders every DataPrismConfigurationException as an operator-facing block naming the refusal code, what to supply, and the two docs pages that explain it, docs/configuration.md and docs/quickstart.md, with no stack frame. A new walkthrough, docs/protect-your-own-api.md, takes a reader with a flat JSON REST API from nothing to a pseudonymised MCP response using only YAML.
  • A one-command demo, examples/quickstart-demo/run.sh, drives the running Compose quickstart end to end over its real MCP transport (obtaining a token, handshaking over SSE-framed tools/call bodies, and printing a pseudonymised response), extracted out of the CI smoke test so a reader can run the same thing locally.
  • The four Compose quickstart images (server-with-extension, fixtures, issuer, certs-init) are now published as multi-architecture GHCR manifests; docker compose up pulls them by default, with the from-source build path moved to docker compose -f compose.yaml -f compose.build.yaml up --build.

Changed

  • data-prism-example is renamed to data-prism-integration-tests: it hosts 11 integration test classes with no duplicate elsewhere, including PiiLogScanTest, the sole enforcement of privacy rule 7, and was never a demo. Package io.github.aindriub.dataprism.example and ExampleApplication are unchanged. Six data-prism-example strings survive deliberately inside data-prism-integration-tests — its own JWT issuer and audit writer-id config values, and the tests asserting on them — because they are observable audit output, not a module identifier.
  • A configured JSON source's DataSourceAdapter bean no longer needs a matching dataprism.sources entry: DataPrismContractValidator's cross-check between dataprism.sources and the supplied adapters now requires only that dataprism.sources be a subset of the supplied adapters, not an exact match, so a source whose transport lives entirely in its own JSON catalogue can supply an adapter with no corresponding dataprism.sources entry. Every DataSourceAdapter bean still has to earn its way onto the review allow-list, though: it must be named by dataprism.sources or supplied by the JSON-catalogue mechanism (the catalogue's own source names), or startup refuses with the stable code UNREVIEWED_SOURCE_ADAPTER — an adapter bean present on the classpath for neither reason is not implicitly approved.
  • Documentation reconciled against the shipped code rather than the plan that preceded it: two consumer guides, docs/extending.md and docs/tools.md, are now linked from README.md, docs/quickstart.md and docs/agents/README.md; every stale "one tool" claim across those files and docs/architecture.md is corrected to name both shipped tools, get_entity_context and compare_entity_sources; docs/architecture.md now attributes ArchitectureTest to data-prism-architecture, the module that hosts it, instead of the renamed module; and README.md's "Until Task 20 delivers…" claim is replaced — the configuration-driven JSON REST mode shipped as the published data-prism-connectors-rest artefact, self- registering via Spring's AutoConfiguration.imports and requiring no Java.

Fixed

  • AuditRecorder no longer advances its in-memory previousHash until the sink's record call actually succeeds. Previously, a throwing sink still left the chain head pointing past an event that was never durably written, so the next successful write chained against a hash for a record that does not exist; a throwing sink now rolls the recorder back to byte-identical prior state instead.
  • AutoConfiguredBeanClassificationTest's sweep now walks @Imported and nested configuration classes recursively, closing a gap that let a bean be placed specifically to dodge classification.
  • A sink failure raised while recording an audit event no longer reaches the MCP client carrying its own exception text. GetEntityContextTool and CompareEntitySourcesTool now catch only the audit-record failure and rethrow it as AuditUnavailableException, exposed to the client as the stable code AUDIT_UNAVAILABLE with no text derived from the caught exception — closing a path by which a hash-chained sink's failure could disclose the server-side audit file's path to the MCP client. The response is still refused, exactly as before; only what the client is told changed.

Behavioural change for API consumers

  • HmacSyntheticGenerator's discriminator widens from 20 bits (masked out of a single 4-byte digest word, four Crockford base32 characters) to 40 bits (eight distinct digest bytes, eight characters), and ADDRESS — which previously rendered no discriminator at all — now carries one like every other namespace. Every pseudonym this generator produces changes as a result; a pseudonym stored or compared under 0.2.0 will not match the one produced under 0.3.0 for the same input. PseudonymisationVersion now rejects a MAC algorithm whose digest is too short for the generator's own reads at construction time, with the stable code pseudonymisation.algorithm-digest-too-short, rather than surfacing an ArrayIndexOutOfBoundsException later; HmacMD5 and HmacSHA1 are both now rejected, so an operator configured with either must move to an algorithm whose MAC output is at least 24 bytes. This reduces collision probability substantially; it does not make collisions impossible, and no such claim is made.
  • AuditRecorder now derives instanceId as <writer-id>/<per-boot random UUID> instead of the writer-id alone, so a restart under the same writer-id is reported as a new writer starting at GENESIS rather than a false chain break. instanceId values recorded before this change are not comparable to ones recorded after it. A writer-id containing / is now refused at startup with the new code INVALID_AUDIT_WRITER; rename any writer-id that contains a / before upgrading.
  • Configuring dataprism.audit.sink: approved-sink with no matching AuditSink bean now refuses with the new code AUDIT_SINK_BEAN_REQUIRED instead of the generic MISSING_AUDIT_SINK, which is retained unchanged for the separate case of an absent or blank dataprism.audit.sink property. Anything keyed on the old code for the bean-absent case must switch to the new one.
  • In 0.2.0, DataPrismContractValidator compared dataprism.sources and the supplied DataSourceAdapter beans for exact equality, so an adapter bean present on the classpath with no matching dataprism.sources entry refused with UNRESOLVED_SOURCE_ADAPTER. In 0.3.0 that same case — an adapter named by neither dataprism.sources nor the JSON-catalogue mechanism — refuses instead with the new code UNREVIEWED_SOURCE_ADAPTER (see the Changed entry above); UNRESOLVED_SOURCE_ADAPTER is retained, unchanged, for the other direction: a dataprism.sources entry with no adapter bean supplied for it. Anything keyed on the old code for the adapter-present case must switch to the new one.

Not changed

  • What the hash-chained audit trail's tamper-evidence covers, stated precisely: an edit of a record inside one writer's chain, caught anywhere including that writer's own last record, and a deletion of a record inside one writer's chain, caught only when a later record follows it. It does not cover truncation of a writer's most recent records, or deletion of an entire process boot's records — both are undetectable from inside the file alone (see the verifier entry above). AuditEventHash is also unkeyed SHA-256, so anyone with write access to the audit file can recompute the whole chain after tampering with it; the trail does not resist an operator, or anyone else who already has that access. Nothing here should be read as, or later restated as, a claim that the audit log is tamper-proof, immutable, or independently complete.
0.2.0 — 2026-09-17 · 1 added · 1 changed · 1 behavioural change · 1 not changed

A second MCP tool, compare_entity_sources, and no other new feature.

Added

  • compare_entity_sources, the second MCP tool: per-field identity plus findings over the same correlated, scrubbed ContextResponse get_entity_context already builds. Findings report agreement, disagreement (INCONSISTENT/FORMATTING_ONLY/ABBREVIATION) and MISSING_IN_SOME_SOURCES, each distinguishable by an explicit discriminator rather than by absence, so a caller can tell "compared and consistent" from "never compared".

Changed

  • The MCP registry namespace is corrected to io.github.AindriuB/data-prism, matching the casing the registry actually grants for the GitHub login. Maven Central's io.github.aindriub and the GHCR path ghcr.io/aindriub/data-prism-server are different identifiers, each correct in its own system and untouched by this correction — do not "fix" the casing inconsistency between them; doing so would break two already-published artifacts. No MCP registry listing for this server exists yet; this corrected namespace, and the label it is checked against, is what the first successful publish will use. server.json's OCI package reference is also corrected to the canonical form the registry requires: no registryBaseUrl, and identifier now carries the image tag directly (ghcr.io/aindriub/data-prism-server:0.2.0) rather than a bare path paired with a separate version field.

Behavioural change for API consumers

  • ContextResponse gained a new record component (fieldsByNamespace), so its equals, hashCode and toString now include it. This is not a linkage break — binary compatibility was verified with javap against the published 0.1.1 jar, and the old constructor signature still works — but two ContextResponse values that compared equal under 0.1.1 may no longer compare equal under 0.2.0.

Not changed

  • The privacy engine, pseudonymisation and security modules: no behaviour change in this release.
0.1.1 — 2026-09-17 · 3 changed

A release-plumbing patch, no features. 0.1.1 supersedes 0.1.0 for the published image: the v0.1.0 tag predates the multi-architecture publish pipeline, so a dispatch against that tag would silently re-run the old single-architecture workflow rather than publish a multi-arch image. main has also diverged from the tag since (CI action bumps, documentation, a quickstart dependency pin), so an image built from main and labelled 0.1.0 would not correspond to the tagged tree. Cutting a patch version is cheaper and more honest than force-moving a tag a published GitHub Release already points at. No production source differs from 0.1.0, so the library artifacts on Maven Central are functionally identical and consumers of those jars have no functional reason to upgrade; 0.1.1 exists for the image and the publish pipeline, where multi-architecture support is a real improvement for anyone running the image.

Changed

  • The server image is now published as a multi-architecture manifest list (linux/amd64 + linux/arm64) instead of linux/amd64 only.
  • Three GitHub Actions dependency bumps in CI workflows.
  • nimbus-jose-jwt bumped to 10.9.1 in data-prism-quickstart-issuer. This affects the quickstart issuer only, not the published server or library artifacts, which never depend on it directly.
0.1.0 — 2026-09-16 · 9 added · 4 not included

First release: the walking skeleton and every slice through S9a.

Added

  • Privacy engine that assigns one deterministic synthetic identity per (scope, subject, namespace, algorithm version, key), never random and never stored in plaintext.
  • Correlation and consistency findings that surface source-data inconsistencies instead of hiding them.
  • Parallel mTLS connectors to enterprise source APIs.
  • Embedded Hazelcast identity cache and per-scope read budget.
  • OAuth2 resource server with session-derived PrivacyContext.
  • Audit logging and metrics.
  • One MCP tool, get_entity_context.
  • Standalone server as the primary deployment surface, and a Spring Boot starter for embedding the privacy layer directly.
  • A one-command local Compose quickstart.

Not included in this release

  • The re-identification operator surface, deferred past V1 by decision (see docs/architecture.md#decisions-worth-knowing).
  • The Elasticsearch connector and its search tools.
  • The three additional MCP tools named in the design review — only get_entity_context exists today.
  • An append-only audit sink with hash-chain verifier. The only audit sink in this release writes to a file and to SLF4J.