V04 Use a custom transformation
Use your current course session. Preparation and course runner · R script ↓
Optional deep dive. Enable only the options named in each block, after preparing its packages or services.
V04a Sort rows
Goal: Apply one additional recipe verb.
Prerequisites: H01 and H08; extras.
These variants use separate objects. There is no explicit output contract for these isolated verb demonstrations, so validation is unvalidated.
training_step(
"V04a",
{
variant_recipe <- dr_step_arrange(dr_recipe(), dplyr::desc(premium))
variant_product <- dr_product("policies.variant", policies) |>
dr_add_recipe(variant_recipe)
print(dr_collect(dr_run(variant_product, write = FALSE)))
},
needs = c("H08"),
enabled = training_options$extras,
optional = TRUE
)Expected result: Policy 4 appears first.
Task: Does sorting change the business key?
Show solution
No, only the order.
V04b Remove duplicates
Goal: Apply a distinct step.
Prerequisites: H01 and H08; extras.
Use a separate object. With no explicit output contract, validation is unvalidated.
training_step(
"V04b",
{
variant_recipe <- dr_step_distinct(dr_recipe(), policy_id, .keep_all = TRUE)
variant_product <- dr_product("policies.variant", policies) |>
dr_add_recipe(variant_recipe)
print(dr_collect(dr_run(variant_product, write = FALSE)))
},
needs = c("H08"),
enabled = training_options$extras,
optional = TRUE
)Expected result: Each ID appears once.
Task: Should bad duplicates always be silently removed?
Show solution
No. First decide which row is valid for the business.
V04c Rename columns
Goal: Make an output name explicit.
Prerequisites: H01 and H08; extras.
Use a separate object. This verb demonstration has no explicit output contract, so validation is unvalidated.
training_step(
"V04c",
{
variant_recipe <- dr_step_rename(dr_recipe(), monthly_premium = premium)
variant_product <- dr_product("policies.variant", policies) |>
dr_add_recipe(variant_recipe)
print(dr_collect(dr_run(variant_product, write = FALSE)))
},
needs = c("H08"),
enabled = training_options$extras,
optional = TRUE
)Expected result: monthly_premium replaces premium.
Task: What must happen to an existing contract?
Show solution
Explicitly revise its columns and rules to match.
V04d Aggregate data
Goal: Change the output grain.
Prerequisites: H01 and H08; extras.
Use a separate object. This verb demonstration has no explicit output contract, so validation is unvalidated.
training_step(
"V04d",
{
variant_recipe <- dr_step_summarise(dr_recipe(), total = sum(premium))
variant_product <- dr_product("policies.variant", policies) |>
dr_add_recipe(variant_recipe)
print(dr_collect(dr_run(variant_product, write = FALSE)))
},
needs = c("H08"),
enabled = training_options$extras,
optional = TRUE
)Expected result: One row with total=740.
Task: Is policy_id still the correct key?
Show solution
No. The grain is now the whole portfolio.
V04 Use a custom transformation
Goal: Apply an ordinary R function.
Prerequisites: H01 and H08; extras.
This separate demonstration has no explicit output contract, so validation is unvalidated.
training_step(
"V04",
{
variant_recipe <- dr_step_transform(dr_recipe(), function(data) {
dplyr::mutate(data, premium = premium * 1.01)
})
variant_product <- dr_product("policies.variant", policies) |>
dr_add_recipe(variant_recipe)
print(dr_collect(dr_run(variant_product, write = FALSE)))
},
needs = c("H08"),
enabled = training_options$extras,
optional = TRUE
)Expected result: Every premium increases by one percent.
Task: Can a custom function have side effects?
Show solution
Yes. write=FALSE does not prevent side effects inside callbacks.