Last updated on 2025-09-11 15:49:26 CEST.
| Flavor | Version | Tinstall | Tcheck | Ttotal | Status | Flags |
|---|---|---|---|---|---|---|
| r-devel-linux-x86_64-debian-clang | 1.2.1 | 7.88 | 251.08 | 258.96 | NOTE | |
| r-devel-linux-x86_64-debian-gcc | 1.2.1 | 5.15 | 170.57 | 175.72 | NOTE | |
| r-devel-linux-x86_64-fedora-clang | 1.2.1 | 315.90 | ERROR | |||
| r-devel-linux-x86_64-fedora-gcc | 1.2.1 | 316.63 | ERROR | |||
| r-devel-windows-x86_64 | 1.2.1 | 11.00 | 219.00 | 230.00 | NOTE | |
| r-patched-linux-x86_64 | 1.2.1 | 8.42 | 237.61 | 246.03 | NOTE | |
| r-release-linux-x86_64 | 1.2.1 | 7.65 | 229.63 | 237.28 | NOTE | |
| r-release-macos-arm64 | 1.2.1 | 111.00 | NOTE | |||
| r-release-macos-x86_64 | 1.2.1 | 213.00 | NOTE | |||
| r-release-windows-x86_64 | 1.2.1 | 11.00 | 223.00 | 234.00 | NOTE | |
| r-oldrel-macos-arm64 | 1.2.1 | 99.00 | NOTE | |||
| r-oldrel-macos-x86_64 | 1.2.1 | 192.00 | NOTE | |||
| r-oldrel-windows-x86_64 | 1.2.1 | 14.00 | 288.00 | 302.00 | NOTE |
Version: 1.2.1
Check: CRAN incoming feasibility
Result: NOTE
Maintainer: ‘Jakub Wiśniewski <jakwisn@gmail.com>’
The Description field contains
<arXiv:2104.00507>.
Please refer to arXiv e-prints via their arXiv DOI <doi:10.48550/arXiv.YYMM.NNNNN>.
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: Rd files
Result: NOTE
checkRd: (-1) choose_metric.Rd:35: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) choose_metric.Rd:36: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) choose_metric.Rd:37: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:23: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) disparate_impact_remover.Rd:28: Lost braces
28 | pigeonholing. The number of pigeonholes is fixed and equal to min{101, unique(a)}, where a is vector with values for subgroup. So if some subgroup is not numerous and
| ^
checkRd: (-1) fairness_check.Rd:47: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:48: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:49: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:50: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:51: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:52: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:53: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:54: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:55: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:56: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:57: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:58: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:61: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:62: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:63: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:64: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:65: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:66: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:12: Lost braces
12 | \item{scale}{logical, if code{TRUE} metrics will be scaled to mean 0 and sd 1. Default \code{FALSE}}
| ^
checkRd: (-1) fairness_heatmap.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_radar.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_radar.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:25: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:26: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:27: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:28: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:30: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:31: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:32: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:33: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) metric_scores.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) metric_scores.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:23: Lost braces in \itemize; \value handles \item{}{} directly
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc, r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc, r-devel-windows-x86_64, r-patched-linux-x86_64, r-release-linux-x86_64, r-release-macos-arm64, r-release-macos-x86_64, r-release-windows-x86_64, r-oldrel-macos-arm64, r-oldrel-macos-x86_64, r-oldrel-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 1.280546e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.06718651 , mean = 0.6975836 , max = 0.9967857
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7225752 , mean = 0.002416363 , max = 0.634748
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-linux-x86_64-fedora-clang
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [58s/62s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1612009 , mean = 0.5448795 , max = 0.86324
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8492543 , mean = 6.504016e-07 , max = 0.7821585
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -173.8227 , mean = 745.5907 , max = 1761.633
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -339.9737 , mean = 1.368747e-12 , max = 326.8232
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -173.8227 , mean = 745.5907 , max = 1761.633
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -339.9737 , mean = 1.368747e-12 , max = 326.8232
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6449584
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 327.8173 , mean = 756.3617 , max = 1159.246
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -635.2394 , mean = 0.1289064 , max = 608.1891
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-fedora-clang
Version: 1.2.1
Check: re-building of vignette outputs
Result: ERROR
Error(s) in re-building vignettes:
--- re-building ‘Advanced_tutorial.Rmd’ using rmarkdown
[WARNING] Deprecated: --no-highlight. Use --syntax-highlighting=none instead.
--- finished re-building ‘Advanced_tutorial.Rmd’
--- re-building ‘Basic_tutorial.Rmd’ using rmarkdown
Quitting from Basic_tutorial.Rmd:254-257 [unnamed-chunk-19]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `rep()`:
! attempt to replicate an object of type 'object'
---
Backtrace:
▆
1. ├─base::plot(fheatmap, text_size = 3)
2. └─fairmodels:::plot.fairness_heatmap(fheatmap, text_size = 3)
3. └─base::ifelse(...)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: processing vignette 'Basic_tutorial.Rmd' failed with diagnostics:
attempt to replicate an object of type 'object'
--- failed re-building ‘Basic_tutorial.Rmd’
SUMMARY: processing the following file failed:
‘Basic_tutorial.Rmd’
Error: Vignette re-building failed.
Execution halted
Flavors: r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 1.280546e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [56s/71s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1611047 , mean = 0.5454043 , max = 0.8667267
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.849369 , mean = -0.0005242329 , max = 0.7752409
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -164.9767 , mean = 749.8455 , max = 1520.65
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -344.4796 , mean = 1.359789e-12 , max = 291.3585
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -164.9767 , mean = 749.8455 , max = 1520.65
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -344.4796 , mean = 1.359789e-12 , max = 291.3585
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6343798
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc