Package {densemlp}

densemlp: Dense Neural Networks for Tabular Regression, Classification and Survival

Description

Dense feed-forward neural networks (multilayer perceptrons) for tabular regression, classification and survival analysis, with a formula or x/y interface. Supports residual and gated hidden blocks, batch normalization, per-layer dropout, learned cross-feature interactions, exponential moving-average weights, learning-rate schedules, internal bootstrap ensembles, case weights (for example importance weights under covariate shift) and Adam optimization. Survival outcomes are trained with either a batch-wise Breslow-tie Cox partial likelihood or a discrete-time inverse-probability-of-censoring-weighted integrated Brier score. The numerical kernels are implemented natively in C++ via 'RcppArmadillo', with no external deep learning framework dependency (no 'torch' / 'libtorch'). Companion helpers provide k-fold cross-validation, hyperparameter search and task-aware evaluation metrics.

Author(s)

Maintainer: Imad El Badisy elbadisyimad@gmail.com

See Also

Useful links:

• https://CRAN.R-project.org/package=densemlp

• Report bugs at https://github.com/ielbadisy/densemlp/issues

Cross-validate a densemlp model

Description

Fits densemlp() on each of folds training splits and evaluates it on the held-out fold via densemlp_metrics().

Usage

cv_densemlp(
  x,
  y,
  task = c("auto", "regression", "binary", "multiclass", "survival", "count"),
  folds = 5L,
  seed = 1L,
  ncores = 1L,
  verbose = FALSE,
  weights = NULL,
  ...
)

Arguments

Value

A list of class densemlp_cv with fold_metrics (one row per fold), summary (mean and SD per metric across folds), and task.

Examples

set.seed(1)
x <- data.frame(a = rnorm(60), b = rnorm(60))
y <- x$a - 0.5 * x$b + rnorm(60, sd = 0.1)
cv <- cv_densemlp(x, y, folds = 3, epochs = 20, hidden_units = c(8))
cv$summary

Fit a fast dense multilayer perceptron

Description

A compact feedforward network for regression and classification on tabular data. Forward propagation, backpropagation, and Adam optimization are all implemented natively in C++ via RcppArmadillo – no torch / libtorch dependency.

Usage

densemlp(
  x = NULL,
  y = NULL,
  task = c("auto", "regression", "binary", "multiclass", "survival", "count"),
  loss = c("cox", "brier"),
  n_bins = 10L,
  hidden_units = c(32, 16),
  epochs = 100L,
  batch_size = 32L,
  lr = 0.001,
  validation = 0.2,
  early_stopping = TRUE,
  patience = 10L,
  min_delta = 0,
  min_epochs = max(10L, floor(epochs * 0.2)),
  residual = FALSE,
  gated = FALSE,
  dropout = 0,
  batch_norm = TRUE,
  input_projection = NULL,
  interaction = FALSE,
  ema_decay = 0,
  ensemble = 1L,
  ensemble_bootstrap = TRUE,
  lr_schedule = c("none", "cosine", "step"),
  seed = 1L,
  verbose = FALSE,
  ncores = 1L,
  formula = NULL,
  data = NULL,
  offset = NULL,
  groups = NULL,
  group_penalty = 0,
  weights = NULL
)

## S3 method for class 'densemlp'
print(x, ...)

Arguments

Details

Hidden layers are ⁠Linear -> BatchNorm -> ReLU -> optional gate -> optional dropout -> optional residual⁠ (batch statistics during training, running mean/var at prediction time, exponential decay 0.9), with He initialization for the linear weights. Set batch_norm = FALSE to drop the normalization step, leaving Linear -> ReLU hidden blocks. Residual blocks use a learned linear projection when dimensions differ. With input_projection = k, a bare linear layer maps the encoded inputs to k dimensions before the first hidden block (mutually exclusive with interaction). The optional interaction layer is a one-layer cross network initialized as the identity. With ema_decay > 0, validation and final predictions use moving-average parameters. With ensemble > 1, fully fitted internal members are trained with successive seeds and optionally bootstrapped rows, and their response probabilities/predictions are averaged transparently. The output layer is plain ⁠Linear -> task activation⁠ (no BN): linear (regression, MSE loss), sigmoid (binary, binary cross-entropy), softmax (multiclass, categorical cross-entropy), exponential (count, Poisson log-likelihood), or a linear risk score (survival, Cox partial likelihood); the first four share the output-gradient simplification dZ = (yhat - y) / n (times the case weight when weights is given), while survival uses its own closed-form Cox gradient (see the "Survival" section below). With validation > 0 and early_stopping = TRUE (both on by default), the parameters (weights, biases, and BN affine/running-stat parameters) from the best validation epoch are restored at the end.

offset and groups both enter the linear predictor additively, after the output layer and before its activation, so the fitted model is ⁠g(E[y]) = f(x) + offset + sum_k alpha_k[group_k]⁠, where f is the network and the alpha_k are the per-level intercepts (see group_effects()).

Value

A densemlp object.

Survival

task = "survival" supports two losses, both trained batch-wise against a ⁠(time, event)⁠ outcome:

• loss = "cox" (the default) trains a single linear risk score using a batch-wise Breslow-tie Cox partial log-likelihood – the same "risk set = current mini-batch" approximation used by DeepSurv/pycox, which turns Cox regression into an ordinary per-batch loss compatible with masked-free Adam training, BN, and early stopping unchanged. For a batch sorted by descending time, with each observation's risk set approximated by the sorted prefix of observations with a time at least as large, the loss is the negative mean, over events, of the risk score minus the log of the cumulative sum of exponentiated risk scores over its risk set – computed via a single cumulative sum with a closed-form O(n) gradient (no autodiff).

• loss = "brier" trains a discrete-time hazard head with n_bins outputs (a quantile grid of the observed follow-up times) directly against the IPCW (Graf et al.) integrated Brier score. Bin k's sigmoid output is a conditional hazard, and the survival probability at that bin is the running product of one minus every hazard up to and including it, which is monotonically non-increasing by construction. Each bin's Brier term is inverse-probability-of-censoring weighted using a Kaplan-Meier estimate of the censoring distribution fit once on the training data; subjects censored before a bin are excluded from that bin's term, per Graf's correction. The gradient is closed-form (no autodiff): a bin's survival probability depends on every hazard logit at or before it, so each bin's Brier-score gradient is distributed back across all of them.

predict(fit, newdata, type = "response") returns a linear risk score for either loss (for "brier", the negative log of the predicted survival probability at the final time bin, so higher is still riskier); with loss = "brier", type = "survival" additionally returns the full n_bins-column survival-probability matrix. densemlp_metrics() reports Harrell's concordance index for task = "survival" regardless of loss.

Examples

set.seed(1)
x <- data.frame(a = rnorm(100), b = rnorm(100))
y <- x$a - 0.5 * x$b + rnorm(100, sd = 0.1)
fit <- densemlp(x, y, epochs = 50, hidden_units = c(16))
predict(fit, x[1:5, ])

Integrated Brier score for a fitted Brier-loss survival densemlp model

Description

The IPCW (Graf et al.) integrated Brier score, evaluated at the model's own time grid (object$survival_breaks), using a Kaplan-Meier estimate of the censoring distribution fit on y (i.e. on whatever data is passed in – pass the held-out set's own outcome for an honest out-of-sample estimate). Lower is better; this is exactly the objective densemlp() minimizes when fit with ⁠task = "survival", loss = "brier"⁠.

Usage

densemlp_integrated_brier_score(object, newdata, y)

Arguments

Value

A single numeric integrated Brier score.

Prediction metrics for a fitted densemlp model

Description

Prediction metrics for a fitted densemlp model

Usage

densemlp_metrics(truth, estimate, task, prob = NULL, weights = NULL)

Arguments

Value

A named list of metrics: rmse, nrmse, rsq for regression, accuracy, balanced_accuracy, macro_auc, log_loss for classification, or concordance (Harrell's C-index) for survival.

Extract the learned group intercepts

Description

Returns the per-level intercepts that a model fitted with groups adds to its linear predictor, on the scale of the outcome for regression and of the link otherwise.

Usage

group_effects(object, ...)

## S3 method for class 'densemlp'
group_effects(object, ...)

Arguments

Value

A named list with one entry per grouping factor: a named numeric vector of intercepts (one per level), or for multiclass a matrix with one row per level and one column per class. For an ensemble, the members' intercepts are averaged.

Permutation variable importance for a fitted densemlp model

Description

Model-agnostic permutation importance: for each predictor, the column is randomly shuffled and the drop in predictive performance (relative to the unpermuted baseline) is recorded. Larger values mean the model relied more on that predictor.

Usage

perm_importance(object, new_data, truth, metric = NULL, seed = object$seed)

## S3 method for class 'densemlp_importance'
print(x, ...)

Arguments

Value

A densemlp_importance object: a list with data (a data frame of feature / importance, ordered by decreasing importance), metric, baseline and task.

Examples

set.seed(1)
x <- data.frame(a = rnorm(120), b = rnorm(120), c = rnorm(120))
y <- x$a - 0.5 * x$b + rnorm(120, sd = 0.1)
fit <- densemlp(x, y, epochs = 40, hidden_units = c(16))
perm_importance(fit, x, y)

Plot training history

Description

Plot training history

Usage

## S3 method for class 'densemlp'
plot(x, ...)

Arguments

Value

x, invisibly.

Plot permutation importance

Description

Plot permutation importance

Usage

## S3 method for class 'densemlp_importance'
plot(x, top = 20L, ...)

Arguments

Value

x, invisibly.

Plot the training history of a fitted densemlp model

Description

A named wrapper around the plot.densemlp() method: draws the per-epoch training and validation loss curves.

Usage

plot_history(object, ...)

Arguments

Value

object, invisibly.

Examples

set.seed(1)
x <- data.frame(a = rnorm(80), b = rnorm(80))
y <- x$a - 0.5 * x$b + rnorm(80, sd = 0.1)
fit <- densemlp(x, y, epochs = 40, hidden_units = c(16))
plot_history(fit)

Predict from a fitted densemlp model

Description

Predict from a fitted densemlp model

Usage

## S3 method for class 'densemlp'
predict(
  object,
  newdata,
  type = c("response", "class", "prob", "survival"),
  offset = NULL,
  groups = NULL,
  ...
)

Arguments

Value

A numeric vector ("response"), a factor ("class"), or a matrix ("prob" or "survival").

Tune a densemlp model over a hyperparameter grid

Description

Fits densemlp() for every combination in grid (each repeated repeats times with successive seeds), ranks candidates by their best internal validation loss, and optionally refits the best configuration on the full data.

Usage

tune_densemlp(
  x,
  y,
  task = c("auto", "regression", "binary", "multiclass", "survival", "count"),
  grid = NULL,
  validation = 0.2,
  seed = 1L,
  repeats = 3L,
  ncores = 1L,
  verbose = FALSE,
  refit = TRUE,
  weights = NULL
)

Arguments

Value

A list of class densemlp_tuned with results (one row per candidate, ranked best first by mean validation loss), best_config, and, when refit = TRUE, best_fit.

Examples

set.seed(1)
x <- data.frame(a = rnorm(80), b = rnorm(80))
y <- x$a - 0.5 * x$b + rnorm(80, sd = 0.1)
tuned <- tune_densemlp(
  x, y, repeats = 1,
  grid = list(hidden_units = list(c(8), c(16, 8)), epochs = c(20))
)
tuned$best_config