Libraries used for the DAG analysis

library(tidyverse)
library(dagitty)
library(ggdag)
knitr::opts_chunk$set(cache=TRUE) #cache results

Creating a DAG

Here we model all causal relations

climate_dag <- dagify(
  vehicle_variant ~ city,
  climate_sw ~ vehicle_variant + rnd,
  energy ~ city + vehicle_variant + climate_sw + temperature_setting,
  temperature_setting ~ climate_performance + climate_preferences,
  climate_performance ~ climate_sw + climate_preferences,
  climate_preferences ~ city)

Double check that the DAG is acyclic

dagitty::isAcyclic(climate_dag)
## [1] TRUE

Analysis of the DAG

What variables should we restrict to run an A/B test?

To see what adjustments are necessary to compute an unbiased total causal effect (i.e. run an A/B test)

adjustmentSets(climate_dag, 
               effect = 'total',   
               exposure = 'climate_sw', 
               outcome = 'energy')
## { vehicle_variant }

Checking the conditional independences

Here we check what relations should be independent. Note that we can only check with data the conditional independence of the variables that are observed.

dagitty::impliedConditionalIndependencies(climate_dag, max.results = 200)
## city _||_ climate_performance | climate_preferences, climate_sw
## city _||_ climate_performance | climate_preferences, vehicle_variant
## city _||_ climate_sw | vehicle_variant
## city _||_ rnd
## city _||_ temperature_setting | climate_performance, climate_preferences
## city _||_ temperature_setting | climate_preferences, climate_sw
## city _||_ temperature_setting | climate_preferences, vehicle_variant
## climate_performance _||_ energy | city, climate_sw, temperature_setting
## climate_performance _||_ energy | climate_preferences, climate_sw, temperature_setting
## climate_performance _||_ rnd | climate_sw, vehicle_variant
## climate_performance _||_ rnd | city, climate_sw
## climate_performance _||_ rnd | climate_preferences, climate_sw
## climate_performance _||_ vehicle_variant | city, climate_sw
## climate_performance _||_ vehicle_variant | climate_preferences, climate_sw
## climate_preferences _||_ climate_sw | vehicle_variant
## climate_preferences _||_ climate_sw | city
## climate_preferences _||_ energy | city, climate_sw, temperature_setting
## climate_preferences _||_ rnd
## climate_preferences _||_ vehicle_variant | city
## climate_sw _||_ temperature_setting | climate_performance, climate_preferences
## energy _||_ rnd | climate_sw, vehicle_variant
## rnd _||_ temperature_setting | climate_performance, climate_preferences
## rnd _||_ temperature_setting | climate_preferences, climate_sw
## rnd _||_ temperature_setting | city, climate_sw
## rnd _||_ temperature_setting | climate_sw, vehicle_variant
## rnd _||_ vehicle_variant
## temperature_setting _||_ vehicle_variant | city, climate_sw
## temperature_setting _||_ vehicle_variant | climate_preferences, climate_sw
## temperature_setting _||_ vehicle_variant | climate_performance, climate_preferences

We can see some common conditional independence relations:

  • (energy || rnd | climate_sw, vehicle_variant) is equivalent to an A/A test on the main metric
  • (temperature_setting || vehicle_variant | city, climate_sw) is equivalent to the A/A test on the guardrail metric
  • (rnd || vehicle_variant) is equivalent to SRM in a the restricted groups
  • (city || rnd) is equivalent to SRM in one of the strata variables

The actual SRM criteria on the treatment assignment is for checking that the arrow between randomization and the treatment is actually as designed (since this is an artificially introduced variable on the DAG).

Direct and Indirect effects

Let’s compute the direct effect

adjustmentSets(climate_dag, 
               effect = 'direct',   
               exposure = 'climate_sw', 
               outcome = 'energy')
## { city, temperature_setting, vehicle_variant }
## { climate_preferences, temperature_setting, vehicle_variant }
## { climate_performance, climate_preferences, vehicle_variant }

We can use only the first adjustment set, since the others contain non-observable or latent variable.

Assuming linearity in the causal effect, the indirect effect is:

\[\text{Total} = \text{Direct} + \text{Indirect}\]

Session information

sessionInfo()
## R version 4.1.1 (2021-08-10)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 18363)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=Swedish_Sweden.1252  LC_CTYPE=Swedish_Sweden.1252   
## [3] LC_MONETARY=Swedish_Sweden.1252 LC_NUMERIC=C                   
## [5] LC_TIME=Swedish_Sweden.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] ggdag_0.2.4     dagitty_0.3-1   forcats_0.5.1   stringr_1.4.0  
##  [5] dplyr_1.0.7     purrr_0.3.4     readr_2.0.2     tidyr_1.1.4    
##  [9] tibble_3.1.5    ggplot2_3.3.5   tidyverse_1.3.1
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.7       lubridate_1.7.10 assertthat_0.2.1 digest_0.6.28   
##  [5] utf8_1.2.2       V8_3.4.2         R6_2.5.1         cellranger_1.1.0
##  [9] backports_1.2.1  reprex_2.0.1     evaluate_0.14    httr_1.4.2      
## [13] pillar_1.6.3     rlang_0.4.11     curl_4.3.2       readxl_1.3.1    
## [17] rstudioapi_0.13  jquerylib_0.1.4  rmarkdown_2.11   igraph_1.2.6    
## [21] munsell_0.5.0    broom_0.7.9      compiler_4.1.1   modelr_0.1.8    
## [25] xfun_0.26        pkgconfig_2.0.3  htmltools_0.5.2  tidyselect_1.1.1
## [29] fansi_0.5.0      crayon_1.4.1     tzdb_0.1.2       dbplyr_2.1.1    
## [33] withr_2.4.2      MASS_7.3-54      grid_4.1.1       jsonlite_1.7.2  
## [37] gtable_0.3.0     lifecycle_1.0.1  DBI_1.1.1        magrittr_2.0.1  
## [41] scales_1.1.1     cli_3.1.0        stringi_1.7.5    fs_1.5.0        
## [45] xml2_1.3.2       bslib_0.3.1      ellipsis_0.3.2   generics_0.1.0  
## [49] vctrs_0.3.8      boot_1.3-28      tools_4.1.1      glue_1.4.2      
## [53] hms_1.1.1        fastmap_1.1.0    yaml_2.2.1       colorspace_2.0-2
## [57] tidygraph_1.2.0  rvest_1.0.1      knitr_1.36       haven_2.4.3     
## [61] sass_0.4.0