6 Data Quality as an Execution Gate

Contracts describe the interface. Quality rules decide whether the current delivery is acceptable.

6.1 Start from one decision

library(dataraft)

policy_delivery <- read.csv("data/policies.csv")

policies <- dr_product("policies") |>
  dr_add_contract(dr_contract(
    "policies",
    columns = c(
      policy_id = "integer",
      broker_id = "integer",
      premium = "numeric",
      status = "character"
    ),
    key = "policy_id"
  )) |>
  dr_add_quality(dataraft.core::dr_quality_rule(
    "premium_non_negative",
    ~ premium >= 0,
    action = "block",
    threshold = 0,
    dimension = "accuracy"
  ))

A rule should exist because its result changes what happens next.

6.2 Block, warn, or quarantine

The current API supports three actions:

block: reject the delivery under the current rule;
warn: permit the delivery but retain a warning;
quarantine: remove failing rows before writing and retain the rejected rows locally on the result.

Quarantine is intentionally narrow. It is available when DataRaft can identify the failing rows of a native row predicate. It is not a generic mechanism for ignoring errors.

6.3 Inspect a blocked result

bad_delivery <- policy_delivery
bad_delivery$premium[3] <- -5

blocked <- dr_run(
  policies,
  data = bad_delivery,
  write = FALSE,
  stop_on_failure = FALSE
)

blocked$status

[1] "blocked"

dr_quality_report(blocked)

# A tibble: 9 × 7
  stage     rule                 status n_failed n_total failure_rate message
  <chr>     <chr>                <chr>     <dbl>   <dbl>        <dbl> <chr>  
1 candidate schema               passed        0       1         0    ""     
2 candidate types                passed        0       4         0    ""     
3 candidate nonempty             passed        0       1         0    ""     
4 candidate not_null:policy_id   passed        0       4         0    ""     
5 candidate not_null:broker_id   passed        0       4         0    ""     
6 candidate not_null:premium     passed        0       4         0    ""     
7 candidate not_null:status      passed        0       4         0    ""     
8 candidate unique_key           passed        0       4         0    ""     
9 candidate premium_non_negative failed        1       4         0.25 ""

dataraft.core::dr_quality_rows(
  blocked,
  rule = "premium_non_negative"
)

# A tibble: 1 × 4
  policy_id broker_id premium status
      <int>     <int>   <dbl> <chr> 
1         3        10      -5 lapsed

The report answers which checks passed or failed. The row view answers which records were involved when row evidence exists.

6.4 Thresholds change policy

A threshold is the permitted fraction of failed test units. A zero threshold requires every evaluated unit to pass.

warning_rule <- dataraft.core::dr_quality_rule(
  "small_negative_share",
  ~ premium >= 0,
  action = "warn",
  threshold = 0.10
)

warning_product <- dr_product("policies_warning") |>
  dr_add_quality(warning_rule)

warning_result <- dr_run(
  warning_product,
  data = bad_delivery,
  write = FALSE,
  stop_on_failure = FALSE
)

dr_quality_report(warning_result)

# A tibble: 4 × 7
  stage     rule                 status    n_failed n_total failure_rate message
  <chr>     <chr>                <chr>        <dbl>   <dbl>        <dbl> <chr>  
1 candidate schema               unvalida…        0       1         0    "No de…
2 candidate types                unvalida…        0       4         0    "No de…
3 candidate nonempty             passed           0       1         0    ""     
4 candidate small_negative_share warning          1       4         0.25 ""

Changing a threshold can change which deliveries reach consumers. Treat it as an acceptance-policy change, not merely tuning.

6.5 Structural checks and business checks

A missing required column means the interface is broken. A negative premium means values violate a business expectation. Both may block publication, but they point to different owners and different fixes.

This distinction improves diagnosis:

Question	Best home
Is the required field present?	contract
Is the key unique and non-null?	contract
Is premium non-negative?	quality rule or stable constraint
Is a small amount of drift tolerable?	quality policy
Who owns the meaning?	metadata

6.6 Optional Pointblank execution

Pointblank is a specialized validation engine. DataRaft can use it for checks while retaining its own product-level gate and evidence model. That keeps the core workflow usable without Pointblank and lets richer validation remain an optional responsibility.

Failure evidence is part of the product story

A useful quality system explains a rejected delivery. The failed result, rule identity, affected rows, and retained evidence are as important as the passing case.

6.7 Exercises

Run the same defect with block and warn. Compare the result status and quality report.
Choose a real rule where quarantine would have a clear business meaning. Explain why dropping the row would be safer than blocking the whole delivery.
Change a threshold from zero to a positive value and describe the governance decision you have made.