H33 Build a model from lake releases

Use your current course session. Preparation and course runner · R script ↓

H33a Publish three member tables

Goal: Store the model building blocks.

Prerequisites: H26 and H18.

Use separate asset IDs for the relational member tables.

training_step(
  "H33a",
  {
    model_policy_release <- dr_publish(
      dr_product("model.policies", linked_policies),
      to = training_lake
    )
    model_customer_release <- dr_publish(
      dr_product("model.customers", customers),
      to = training_lake
    )
    model_broker_release <- dr_publish(
      dr_product("model.brokers", brokers),
      to = training_lake
    )
    print(c(
      policies = model_policy_release$release_id,
      customers = model_customer_release$release_id,
      brokers = model_broker_release$release_id
    ))
  },
  needs = c("H26", "H18"),
  enabled = training_options$lake,
  optional = TRUE
)

Expected result: Three fixed release IDs. Without explicit table contracts only the structure is captured automatically.

Task: Are these three publications one transaction?

H33 Build a model from lake releases

Goal: Pin every member table.

Prerequisites: H33a.

dr_model builds a dm model from stored tables, with each member release ID specified explicitly.

training_step(
  "H33",
  {
    lake_model <- dr_model(
      training_lake,
      tables = c(
        policies = "model.policies",
        customers = "model.customers",
        brokers = "model.brokers"
      ),
      releases = c(
        policies = model_policy_release$release_id,
        customers = model_customer_release$release_id,
        brokers = model_broker_release$release_id
      ),
      primary_keys = list(
        policies = "policy_id",
        customers = "customer_id",
        brokers = "broker_id"
      ),
      foreign_keys = list(
        list(
          table = "policies",
          columns = "customer_id",
          ref_table = "customers",
          ref_columns = "customer_id"
        ),
        list(
          table = "policies",
          columns = "broker_id",
          ref_table = "brokers",
          ref_columns = "broker_id"
        )
      )
    )
    print(lake_model)
  },
  needs = c("H33a"),
  enabled = training_options$lake,
  optional = TRUE
)

Expected result: Three related tables on fixed releases.

Task: Is the model automatically flattened?