Check if group variable has exactly 2 levels

Description

Check if group variable has exactly 2 levels

Usage

check_binary_group(group_levels, gcol_name)

Map spark to base R types

Description

Map spark to base R types

Usage

spark_to_r_type(spark_type)

Compute Standardized Differences for Binary Variables (Spark)

Description

Calculates standardized differences for binary variables using a Spark DataFrame. Equivalent to stddiff::stddiff.binary but operates on Spark data.

Usage

stddiff.binary(data, gcol, vcol, verbose = FALSE)

Arguments

Details

Variables are encoded using lexicographic ordering since Spark does not have factor types. The first level alphabetically becomes 0, the second becomes 1.

The standardized difference is computed as:

d = \frac{|p_t - p_c|}{\sqrt{(p_t(1-p_t) + p_c(1-p_c))/2}}

Value

A numeric matrix with one row per variable and columns:

• p.c: Proportion in control group (first level alphabetically)

• p.t: Proportion in treatment group (second level alphabetically)

• missing.c: Number of missing values in control group

• missing.t: Number of missing values in treatment group

• stddiff: Standardized difference

• stddiff.l: Lower bound of 95% confidence interval

• stddiff.u: Upper bound of 95% confidence interval

See Also

stddiff.category, stddiff.numeric

Examples

sc <- sparklyr::spark_connect(master = "local")

spark_df <- sparklyr::copy_to(sc, mtcars)

result <- stddiff.binary(
  data = spark_df,
  gcol = 9,   # column index of grouping variable
  vcol = c(8) # columns of binary variables
)

sparklyr::spark_disconnect(sc)

Compute Standardized Differences for Categorical Variables (Spark)

Description

Calculates standardized differences for categorical variables using a Spark DataFrame. Equivalent to stddiff::stddiff.category but operates on Spark data.

Usage

stddiff.category(data, gcol, vcol, verbose = FALSE)

Arguments

Details

For categorical variables with K levels, the standardized difference is computed using a multivariate approach that accounts for all K-1 levels simultaneously (excluding the reference level). Category levels are sorted lexicographically; the first level alphabetically serves as the reference.

Value

A numeric matrix with one row per category level and columns:

• p.c: Proportion in control group

• p.t: Proportion in treatment group

• missing.c: Number of missing values in control group (first row only)

• missing.t: Number of missing values in treatment group (first row only)

• stddiff: Standardized difference (first row only)

• stddiff.l: Lower CI bound (first row only)

• stddiff.u: Upper CI bound (first row only)

Row names are formatted as "variable_name level_name".

See Also

stddiff.binary, stddiff.numeric

Examples

sc <- sparklyr::spark_connect(master = "local")

spark_df <- sparklyr::copy_to(sc, as.data.frame(Titanic))

result <- stddiff.category(
  data = spark_df,
  gcol = 4,   # column index of grouping variable
  vcol = c(1) # columns of categorical variables
)

sparklyr::spark_disconnect(sc)

Compute Standardized Differences for Numeric Variables (Spark)

Description

Calculates standardized differences for continuous numeric variables using a Spark DataFrame. Equivalent to stddiff::stddiff.numeric but operates on Spark data.

Usage

stddiff.numeric(data, gcol, vcol, verbose = FALSE)

Arguments

Details

The standardized difference for continuous variables is computed as:

d = \frac{|\bar{x}_t - \bar{x}_c|}{\sqrt{(s_t^2 + s_c^2)/2}}

where \bar{x} represents means and s^2 represents variances.

This is equivalent to Cohen's d with pooled standard deviation.

Value

A numeric matrix with one row per variable and columns:

• mean.c: Mean in control group

• sd.c: Standard deviation in control group

• mean.t: Mean in treatment group

• sd.t: Standard deviation in treatment group

• missing.c: Number of missing values in control group

• missing.t: Number of missing values in treatment group

• stddiff: Standardized difference

• stddiff.l: Lower bound of 95% confidence interval

• stddiff.u: Upper bound of 95% confidence interval

See Also

stddiff.binary, stddiff.category

Examples

sc <- sparklyr::spark_connect(master = "local")

spark_df <- sparklyr::copy_to(sc, mtcars)

result <- stddiff.numeric(
  data = spark_df,
  gcol = 8,          # column index of grouping variable
  vcol = c(1, 2, 5)  # columns of numeric variables
)

sparklyr::spark_disconnect(sc)

Validate inputs for stddiff functions

Description

Validate inputs for stddiff functions

Usage

validate_stddiff_inputs(data, gcol, vcol, type)