
<!-- README.md is generated from README.Rmd. Please edit that file -->

# DataAudit

<!-- badges: start -->

[![R-CMD-check](https://github.com/vinodhpmd/DataAudit/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/vinodhpmd/DataAudit/actions/workflows/R-CMD-check.yaml)
[![Codecov test
coverage](https://codecov.io/gh/vinodhpmd/DataAudit/graph/badge.svg)](https://app.codecov.io/gh/vinodhpmd/DataAudit)

<!-- badges: end -->

## Overview

`DataAudit` is an R package for systematic data-quality auditing and
validation.

It provides tools for detecting common data-quality problems, defining
reusable validation rules, generating structured audit reports, and
calculating an overall data-quality score.

DataAudit supports checks for missing values, blank values, duplicates,
infinite values, constant and near-zero variance variables, outliers,
ranges, categories, data types, patterns, dates, identifiers, lengths,
character case, whitespace, sequences, dependencies, uniqueness, and
cross-variable consistency.

## Installation

You can install DataAudit from GitHub with:

``` r
# install.packages("pak")
pak::pak("vinodhpmd/DataAudit")
```

Then load the package:

``` r
library(DataAudit)
```

## Quick example

Create a dataset containing several data-quality problems:

``` r
dat <- data.frame(
  ID = c(
    "A001",
    "A002",
    "A002",
    "BAD"
  ),
  Species = c(
    "Dog",
    "Horse",
    "",
    "Cat"
  ),
  Age = c(
    5,
    40,
    -2,
    NA
  ),
  stringsAsFactors = FALSE
)

dat
```

    ##     ID Species Age
    ## 1 A001     Dog   5
    ## 2 A002   Horse  40
    ## 3 A002          -2
    ## 4  BAD     Cat  NA

## General data audit

Use `audit_data()` for a general first-pass audit:

``` r
report <- audit_data(
  dat,
  include_nzv = FALSE
)

report
```

    ## DataAudit Report
    ## ================
    ## 
    ## Rows:                  4
    ## Variables:             3
    ## Variables with issues: 2
    ## Duplicated rows:       0
    ## Total issues:          2
    ## Rule violations:       0
    ## Overall issues:        2
    ## 
    ## Variables with issues
    ## ---------------------
    ##  Variable     Class Missing MissingPercent Blank Infinite Unique Constant
    ##   Species character       0              0     1        0      4    FALSE
    ##       Age   numeric       1             25     0        0      3    FALSE
    ##  NearZeroVariance Issue
    ##             FALSE  TRUE
    ##             FALSE  TRUE
    ## 
    ## No duplicated rows detected.

Detailed results are available through:

``` r
report$overview
```

    ##                         Metric Value
    ## 1                         Rows     4
    ## 2                    Variables     3
    ## 3               Missing values     1
    ## 4                 Blank values     1
    ## 5              Infinite values     0
    ## 6              Duplicated rows     0
    ## 7           Constant variables     0
    ## 8 Near-zero variance variables     0
    ## 9        Variables with issues     2

``` r
report$variables
```

    ##         Variable     Class Missing MissingPercent Blank Infinite Unique
    ## ID            ID character       0              0     0        0      3
    ## Species  Species character       0              0     1        0      4
    ## Age          Age   numeric       1             25     0        0      3
    ##         Constant NearZeroVariance Issue
    ## ID         FALSE            FALSE FALSE
    ## Species    FALSE            FALSE  TRUE
    ## Age        FALSE            FALSE  TRUE

``` r
report$duplicates
```

    ##   Row Duplicate
    ## 1   1     FALSE
    ## 2   2     FALSE
    ## 3   3     FALSE
    ## 4   4     FALSE

``` r
report$summary
```

    ##   Rows Variables VariablesWithIssues DuplicateRows TotalIssues RuleViolations
    ## 1    4         3                   2             0           2              0
    ##   OverallIssues
    ## 1             2

## Individual checks

Specific checks can also be performed independently.

``` r
missing_check(dat)
```

    ##   Variable Total Missing Complete MissingPercent CompletePercent
    ## 1       ID     4       0        4              0             100
    ## 2  Species     4       0        4              0             100
    ## 3      Age     4       1        3             25              75

``` r
blank_check(dat)
```

    ##         Variable TextVariable Missing Blank BlankPercent HasBlank
    ## ID            ID         TRUE       0     0            0    FALSE
    ## Species  Species         TRUE       0     1           25     TRUE
    ## Age          Age        FALSE       1     0            0    FALSE

``` r
duplicate_check(dat)
```

    ##   Row Duplicate
    ## 1   1     FALSE
    ## 2   2     FALSE
    ## 3   3     FALSE
    ## 4   4     FALSE

``` r
range_check(
  dat,
  variable = "Age",
  min = 0,
  max = 30
)
```

    ##   Row Value BelowMinimum AboveMaximum OutOfRange
    ## 1   1     5        FALSE        FALSE      FALSE
    ## 2   2    40        FALSE         TRUE       TRUE
    ## 3   3    -2         TRUE        FALSE       TRUE
    ## 4   4    NA        FALSE        FALSE      FALSE

``` r
category_check(
  dat,
  variable = "Species",
  allowed = c(
    "Dog",
    "Cat"
  )
)
```

    ##   Row Value Missing ValidCategory InvalidCategory
    ## 1   1   Dog   FALSE          TRUE           FALSE
    ## 2   2 Horse   FALSE         FALSE            TRUE
    ## 3   3         FALSE         FALSE            TRUE
    ## 4   4   Cat   FALSE          TRUE           FALSE

## Validation rules

Reusable validation rules can be defined with `audit_rules()`:

``` r
rules <- audit_rules(
  range = list(
    Age = c(0, 30)
  ),
  category = list(
    Species = c(
      "Dog",
      "Cat"
    )
  ),
  unique = "ID",
  required = "Species",
  pattern = list(
    ID = "^A[0-9]{3}$"
  )
)
```

Apply the rules directly:

``` r
rule_results <- apply_audit_rules(
  dat,
  rules
)

rule_results
```

    ## DataAudit Rule Validation
    ## =========================
    ## 
    ## Status:           Violations detected
    ## Total violations: 8
    ## 
    ## Rule summary
    ## ------------
    ##      Rule Violations
    ##     Range          2
    ##  Category          2
    ##    Unique          2
    ##  Required          1
    ##   Pattern          1
    ## 
    ## Variables with rule violations
    ## ------------------------------
    ## - Range: Age
    ## - Category: Species
    ## - Unique: ID
    ## - Required: Species
    ## - Pattern: ID

## Integrated audit

Custom validation rules can be incorporated directly into the general
audit:

``` r
report <- audit_data(
  dat,
  include_nzv = FALSE,
  rules = rules
)

report
```

    ## DataAudit Report
    ## ================
    ## 
    ## Rows:                  4
    ## Variables:             3
    ## Variables with issues: 2
    ## Duplicated rows:       0
    ## Total issues:          2
    ## Rule violations:       8
    ## Overall issues:        10
    ## 
    ## Variables with issues
    ## ---------------------
    ##  Variable     Class Missing MissingPercent Blank Infinite Unique Constant
    ##   Species character       0              0     1        0      4    FALSE
    ##       Age   numeric       1             25     0        0      3    FALSE
    ##  NearZeroVariance Issue
    ##             FALSE  TRUE
    ##             FALSE  TRUE
    ## 
    ## No duplicated rows detected.
    ## 
    ## Custom rule validation
    ## ----------------------
    ##      Rule Violations
    ##     Range          2
    ##  Category          2
    ##    Unique          2
    ##  Required          1
    ##   Pattern          1
    ## 
    ## Total rule violations: 8

The report now combines automatic auditing and custom validation:

``` r
report$summary
```

    ##   Rows Variables VariablesWithIssues DuplicateRows TotalIssues RuleViolations
    ## 1    4         3                   2             0           2              8
    ##   OverallIssues
    ## 1            10

``` r
report$rules
```

    ## DataAudit Rule Validation
    ## =========================
    ## 
    ## Status:           Violations detected
    ## Total violations: 8
    ## 
    ## Rule summary
    ## ------------
    ##      Rule Violations
    ##     Range          2
    ##  Category          2
    ##    Unique          2
    ##  Required          1
    ##   Pattern          1
    ## 
    ## Variables with rule violations
    ## ------------------------------
    ## - Range: Age
    ## - Category: Species
    ## - Unique: ID
    ## - Required: Species
    ## - Pattern: ID

A compact summary can be obtained with:

``` r
summary(report)
```

    ## $dataset
    ##   Rows Variables VariablesWithIssues DuplicateRows TotalIssues RuleViolations
    ## 1    4         3                   2             0           2              8
    ##   OverallIssues
    ## 1            10
    ## 
    ## $variables_with_issues
    ##         Variable     Class Missing MissingPercent Blank Infinite Unique
    ## Species  Species character       0              0     1        0      4
    ## Age          Age   numeric       1             25     0        0      3
    ##         Constant NearZeroVariance Issue
    ## Species    FALSE            FALSE  TRUE
    ## Age        FALSE            FALSE  TRUE
    ## 
    ## $duplicate_rows
    ## [1] Row       Duplicate
    ## <0 rows> (or 0-length row.names)
    ## 
    ## $rules
    ##       Rule Violations
    ## 1    Range          2
    ## 2 Category          2
    ## 3   Unique          2
    ## 4 Required          1
    ## 5  Pattern          1
    ## 
    ## attr(,"class")
    ## [1] "summary.DataAuditReport" "list"

## Data-quality score

Calculate an overall data-quality score with:

``` r
score <- audit_score(report)

score
```

    ## DataAudit Quality Score
    ## =======================
    ## 
    ## Score:               54.55 / 100
    ## Quality:             Poor
    ## 
    ## Dataset
    ## -------
    ## Rows:                4
    ## Variables:           3
    ## Data cells:          12
    ## 
    ## Issues
    ## ------
    ## Automatic issues:    2
    ## Rule violations:     8
    ## Overall issues:      10

The score ranges from **0 to 100**, with higher scores indicating fewer
detected data-quality issues relative to dataset size.

DataAudit classifies scores as:

- **Excellent**
- **Good**
- **Moderate**
- **Poor**
- **Critical**

The score is intended as a compact audit summary and should be
interpreted alongside the underlying diagnostics and the intended use of
the dataset.

## Main functionality

DataAudit currently provides functions for:

- general data auditing;
- missing and blank values;
- whitespace problems;
- infinite values;
- duplicate and unique values;
- constant and near-zero variance variables;
- outliers;
- ranges and categories;
- data types;
- regular-expression patterns;
- dates and identifiers;
- character lengths and case;
- sequences and grouped sequences;
- dependencies;
- cross-variable consistency;
- reusable validation rules; and
- data-quality scoring.

## Typical workflow

``` text
Raw data
   |
   v
audit_data()
   |
   +----> Automatic checks
   |
   +----> audit_rules()
              |
              v
        Rule validation
   |
   v
DataAuditReport
   |
   +----> summary()
   |
   +----> audit_score()
```

## Authors

**Vinodh Kumar Obli Rajendran**  
ORCID: 0000-0002-7232-4122

**Keerthi Aaradhana**

## License

DataAudit is released under the MIT License.
