make_dpp() now computes the deviation from perfect
performance as Smith, Lampinen, Wells, Smalarz and Mackovichova (2019)
define it: DPP(c) = 1 - Culprit(c) + Innocent(c) at each ROC operating
point, averaged over the operating points (their Equation 2). The
previous statistic, 1 - AUC(observed) / AUC(perfect) over the observed
false-ID range, was a different quantity under the same name, and it
cited an unrelated paper. On a single operating point with culprit rate
.8 and innocent-suspect rate .1, the old function returned .6; the
published measure is .3. The new version reproduces Smith et al.’s Table
4 (.864 and .816). The return value now has dpp,
dpp_by_point and best_point in place of
auc_observed, auc_perfect,
auc_gap, perfect_roc and max_fa.
Optional alpha and beta arguments give their
weighted Equation 3.compare_dpp() now bootstraps the difference in average
DPP (stratified by target presence within each procedure) and returns
its standard error and a percentile interval, with a print method.
plot_dpp() plots DPP at each operating point;
plot_dpp_comparison() overlays two procedures and loses its
layout argument. plot_dpp() arguments are now
show_mean, show_chance and
show_dpp.innocent_suspect argument: "designated",
"estimated" (filler IDs / lineup size), or
"auto" (the default). The app’s Data tab has a matching
control. See the bug fix below.make_cacdata(), make_cac(),
ppv_by_confidence(), ppv_range_by_confidence()
and make_ppv_range() gain base_rate.
Count-based accuracy, the Mickes (2015) definition, stays the default,
but it equals the probability of guilt only when the target-present and
target-absent samples are the same size: with HR = .6 and FAR = .1 it
gives .857, .923 or .667 for TP:TA splits of 100:100, 200:100 and
100:300. These functions now say so in a message when the arms differ.
base_rate = 0.5 computes accuracy from the hit and false-ID
rates instead, which gives .857 for all three; other values apply Bayes’
rule at that prior. The rate-based CAC standard error is a delta-method
approximation. Existing results are unchanged.fit_max_sdt() could report non-convergence at a
correctly found minimum. It chose the best of its L-BFGS-B starts by
value and kept that start’s convergence code, so a start that stopped
with a line-search code at a flat minimum was reported as not converged
even when the Nelder-Mead polish converged to the same value. This
happened with R 4.6.2 on Linux, where compare_max_sdt()
then refused to compare two fits to identical conditions. A polish that
converges without being worse than the best start is now the reported
solution, and compare_max_sdt() gives the convergence codes
when it refuses. Estimates are unchanged.
make_rocdata() and
make_utility_curves() took a confidence criterion from
every response, rejections included. The extra criteria only duplicated
existing operating points, so the ROC path and partial AUC were
unaffected, but averages over points were reweighted by how rejection
confidence happened to be coded. Changing only the rejection confidences
(from 0 to values between 51 and 99) moved average DPP from .417 to .647
and equal-variance z-ROC d’ from 1.879 to 1.398. The criteria are now
the confidence levels of the responses that can move an operating point:
target-present suspect IDs, plus target-absent suspect IDs with a
designated suspect or filler IDs without one. Both codings now give DPP
.475 and d’ 1.757. Results change for data in which rejections or filler
IDs carry confidence levels that no counted ID uses;
lineup_example is unaffected. make_rocdata(),
make_dpp() and make_utility_curves() gain
thresholds to set the criteria, and
make_rocdata() and make_dpp() return the
criteria used. compare_dpp() and the
fit_sdt_roc() bootstrap reuse the full-sample criteria in
every replicate, so that all replicates estimate the same
average.
Target presence coded 0/1, which the validator accepts, broke the
bootstraps in compare_pauc() (an error) and
fit_sdt_roc(), where every replicate failed silently and
the intervals were NA with n_successful = 0.
Both now resample 0/1 and logical coding identically.
fit_sdt_roc() also warns when bootstrap samples fail,
giving the number and the first error.
func_size_report() bootstrapped functional size,
1/p, and boot.ci() silently dropped the resamples with no
target choices, where it is infinite. With one target choice among 30
mock witnesses, 353 of 1,000 resamples were dropped and the percentile
interval was [10, 30]. The bootstrap now resamples p and inverts its
limits, which gives the same percentile and BCa intervals when every
resample is finite and an unbounded upper limit when it is not ([10,
Inf] here). The result gains an exact interval (inverted
Clopper-Pearson, [5.81, 1185.44] here) and n_infinite, the
number of resamples without target choices. The app’s Bias & size
tab shows both, and displays infinite limits instead of leaving them
blank.
standardize_lineup_data() kept only the columns it
maps, so it dropped suspect_pos and
lineup_member. A designated suspect who was never chosen
then fell back to the filler estimate after standardization (false-ID
rate .10 instead of 0), and effective-size-corrected PPV lost its member
choices (.833 instead of .50 when target-absent choices fall on one
member). Every column it does not map is now carried through
unchanged.
compare_multid() differenced the two bootstrap
matrices row by row. The conditions are independent samples, but fits
made with the same seed have correlated draws, so the standard error
collapsed. Comparing counts (60, 25, 15, 10, 30, 60) with (60, 27, 13,
10, 30, 60), both fitted with seed 1, gave d’(GFp) a standard error of
.014 and p < .0001; the correct values are .347 and .86. Identical
inputs gave a zero-width interval. The standard error is now the square
root of the sum of the two bootstrap variances, and the interval comes
from randomly re-paired draws. compare_multid() gains
seed.
get_face_landmarks() failed with every current
MediaPipe release. The image was read into R and passed back as an int32
array, which OpenCV and MediaPipe reject, and the function used
mediapipe.solutions.face_mesh, which current MediaPipe no
longer has. The image now stays a uint8 numpy array, and when
solutions is absent the MediaPipe Tasks
FaceLandmarker is used. Its model is downloaded once to
tools::R_user_dir("r4lineups", "cache"). Both routes return
the 478-point mesh (468 face points plus 10 iris points); the
documentation said 468. check_python_deps() now records in
a "face_mesh" attribute whether landmarks can actually be
extracted, not only whether MediaPipe imports. An opt-in test
(R4LINEUPS_RUN_FACE=true) exercises the whole
path.
In the app’s Bias & size tab, ticking “Small-sample corrected
E’” changed the reported estimate but left its intervals on the plug-in
estimator, so the value box could show a corrected E’ with a plug-in
interval that did not contain it. esize_boot_dist() gains
unbiased = FALSE, which bootstraps the corrected estimator
(dropping, with a warning, resamples where it is unbounded). The app now
bootstraps the corrected E’ when it reports it, and lists the plug-in
estimate with its own normal-theory and bootstrap intervals on
separately labelled rows.
Target-absent data with a designated innocent suspect who was
never chosen were treated as having no designated suspect, so the
false-ID rate was estimated from filler IDs instead of being zero. With
100 target-absent lineups, 60 filler IDs and no suspect IDs, the ROC
reached a false-ID rate of .10 instead of 0. "auto" now
treats the lineups as designated when any target-absent row is a suspect
ID or the data carry a non-missing suspect_pos
column (as simulate_lineup_data() output and
lineup_example do). Otherwise it falls back to the filler
estimate with a message. Bootstraps (compare_pauc(),
fit_sdt_roc(), bootstrap_anri(),
compare_anri()),
make_calibration_by_condition(),
make_utility_difference(),
ppv_range_by_confidence() and the
make_multid() confidence breakdown now resolve the design
once on the full data, so a resample that happens to contain no suspect
IDs no longer switches estimator.
ppv_by_confidence() dropped every confidence bin
that contained no target-absent trials, and with it that bin’s
guilty-suspect IDs, so both the bin and the overall PPV were wrong. With
90 guilty IDs at confidence 100 and all target-absent responses at 50,
the overall PPV was .50 instead of .909. Such bins are now kept, with no
innocent-suspect IDs (PPV 1) and an NA error rate and
effective size. Only bins empty in both arms are dropped.
make_ppv_range() and its plots inherit the fix.
boot_winter_2ht(method = "bca") failed with a
dimnames error, because the interval matrix was transposed
into the wrong shape, and even before that its acceleration was fixed at
0, so the “BCa” interval was only bias-corrected. The acceleration is
now the jackknife estimate (one refit per non-empty response cell,
weighted by its count), the bias correction handles ties and stays
finite, and both methods return a parameters x (Lower, Upper)
matrix.
The posterior means reported by func_size_bayes(),
diag_ratio_T_bayes() and
diag_ratio_T_bayes_compare() were averages of reciprocal
draws. When the relevant Beta posterior has shape a <= 1 (for
example, no suspect choices under the Jeffreys prior), the true mean is
infinite and the average never settled: it was about 11 million, 3
million and 36 million for 10^3, 10^4 and 10^5 draws of the same
functional size. The means are now computed exactly,
(a + b - 1) / (a - 1) for 1/p, and are
Inf when they do not exist. Print methods say so; a
difference of two infinite means is NA. Medians, intervals
and threshold probabilities are unchanged.
compare_pauc() overstated partial AUC whenever the
ROC had a vertical segment, that is, when a confidence level added
correct IDs but no false IDs. Its integrator kept only the upper
endpoint of each such segment, so the next sloping segment started from
too high a correct-ID rate. A curve through (0, 0), (0, .5), (.1, .5)
and (.1, 1) has area .05; compare_pauc() reported .075
while make_rocdata() reported .05. Both now use one
internal integrator that keeps every endpoint and interpolates within
the segment that crosses the cutoff, and their results agree. The
compare_pauc() bootstrap and
simulate_power_analysis() use the same integrator.
make_utility_curves() stored the reject-all strategy
at a criterion of min(confidence) - 1. Read as “act on
suspect IDs with confidence at or above the criterion”, that value
accepts every identification, the opposite of the strategy. With 16/20
target-present and 2/20 target-absent suspect IDs at confidence 80 and a
base rate of .01, the optimum was reject-all (expected utility .49) but
was reported as criterion 79. Reject-all now sits at
max(confidence) + 1, every row of utility_data
and max_utility carries a strategy
("threshold" or "reject_all"), and the plot
labels the reject-all point. Expected utilities are unchanged.
esize_T_bayes() now warns when
threshold is at or above the number of lineup members.
Because E’ cannot exceed k and equals it only at exact uniformity, P(E’
< k | data) is 1 for any data, so such a threshold carries no
information.
simulate_power_analysis() returns an
n_total column. A sample size of n simulates n
target-present and n target-absent witnesses in each of the two
conditions, 4n in all; the documentation and plot axis now say
so.
winter_2ht_model, model_comparison);unbiased = TRUE uses the unbiased estimator of
the concentration, not of E’ (Vignette,
?esize_T);suspect_pos, so all
three corrections gave 1; it now gives .83, .77 and .45;plot.model_comparison() draws its panels with a shared
compact theme, so titles no longer overrun, and clips the 2-HT Wald
intervals to [0, 1] instead of dropping bars that cross zero.control = list(objective = "pywitness") turns the
correction off to reproduce pyWitness itself. The installed Python
environment is unchanged. Observed-denominator fits reject zero observed
cells. Engine provenance records the correction.The first argument of esize_T(),
esize_m(), i_esize_T(),
gen_linevec(), lineup_prop_tab(),
allfoilbias(), datacheck3(),
esize_T_boot(), esize_T_bayes() and
esize_T_ci_n() is renamed from lineup_table to
lineup_tab, so that it no longer collides with the new
lineup_table() function (see below). Code that calls these
functions positionally is unaffected; code that passes the argument by
name (e.g. esize_T(lineup_table = x)) must be updated to
esize_T(lineup_tab = x).
simulate_power_analysis() now performs an actual
power analysis and its arguments have changed accordingly. It takes
d_prime_null (the comparison condition),
n_bootstrap, lineup_size, seed
and verbose; the former stat_function argument
is replaced by test_function, which takes two dataframes
and returns a p-value. The returned columns are now
sample_size, power, power_se,
mean_effect, sd_effect and
n_valid. See the bug-fix entry below for why this was
necessary.
lineup_example has been regenerated and now has 1400
rows rather than 200, with lineup_member and
suspect_pos columns. Numeric results computed from it will
differ from previous versions.
simulate_power_analysis() computed power as the
proportion of simulations in which the pAUC exceeded zero. A partial
area under a curve is always positive, so that proportion was
always exactly 1, for every sample size and every
effect size, and the resulting power curve was a flat line at the top of
the plot. Power is now the proportion of simulated studies in which a
test of the difference between two conditions reaches significance,
using compare_pauc() by default. The single test covering
this function was skipped and additionally asserted a field the function
never returned, so the fault could not have been caught; it has been
replaced with tests that run.
make_roc(), make_cac(),
make_rac(), make_fullroc(),
make_calibration() and compute_dpp() set
$plot to NULL when called with
show_plot = FALSE, so suppressing display also destroyed
the plot object. result$plot was then NULL and
nothing could be drawn from it — the cause of several “the plot does not
show” reports. The plot is now always built, and show_plot
controls only whether it is displayed immediately, which is what its
documentation always said it did.
compare_pauc() chose its default region of
integration as the larger of the two conditions’
maximum false-ID rates. Since neither curve extends past its own maximum
and no extrapolation is performed, the curve that stopped sooner was
integrated over a smaller interval than the other, so the two “partial
areas” covered different regions and were not comparable. In testing,
this understated a real difference by around 14%. The default is now the
largest region both curves attain, a message reports it
when the two differ, and an explicit max_false_id_rate
beyond either curve’s reach now warns.
make_fullroc() and make_fullroc_data()
now report that the default order = "diagnosticity" is
optimistically biased. Because the ordering of evidence categories is
estimated from the same data used to compute the area, data containing
no signal at all return a mean AUC of roughly 0.65 at 100 lineups per
arm, 0.60 at 200 and 0.55 at 1000, against a true value of 0.5. A-priori
ordering is unbiased at every sample size tested. The default is
unchanged, since it is the ordering Smith et al. describe, but a message
now points to order = "apriori" for inferential
use.
fit_winter_2ht() failed with
non-finite finite-difference value when the data contained
almost no lineup rejections. The detect-absence parameter
dA is identified by rejections, so the likelihood is nearly
flat without them. It now stops with a message naming the actual problem
and the likely cause — a decision criterion so liberal that witnesses
never reject. compare_models() had silently omitted the
2-HT model in this situation, so every downstream section that referred
to it produced no output.
format_comparison_table(format = "markdown")
produced a table that rendered as a single run-on paragraph. It returns
a knitr_kable object, which is a character vector with one
element per line, and the vignette printed it with cat(x) —
joining the lines with spaces. The vignette now uses
cat(x, sep = "\n") and the return value is documented as
line-per-element. The markdown and latex forms also called
kable() without row.names = FALSE, so row
names inherited from the named parameter vectors (“dP”, “dA”, …)
appeared as a spurious unlabelled first column.
plot_2ht_parameters() rejected
winter_2ht_boot objects, although its documentation offered
confidence intervals “if bootstrapped”. It now accepts them and draws
the bootstrap percentile intervals. For a plain fit it still draws
normal approximations, but clamped to [0, 1]: all four parameters are
probabilities, and on lineup_example the interval for
b previously ran to -0.023.
plot.esize_T_bayes() packed the credible interval
into the plot title using an embedded newline, producing a two-line
74-character title laid out as a single text grob — the usual cause of a
heading running off the top of a rendered figure. The interval is now a
subtitle. The other five Bayesian plot methods were checked and none had
the same fault.
Effective size is no longer referred to eponymously as “Tredoux
effective size” in documentation, printed output or plot labels; it is
now “effective size” with the citation (Tredoux, 1998). Affects
esize_T(), i_esize_T(),
esize_T_ci_n() and the esize_T_bayes()
family.
fit_max_sdt() left bootstrap replicates that failed
to converge as NA rows in boot_params without
recording how many, so sd(fit$boot_params[, 1]) returned
NA unexplained. It now reports boot_n_used and
boot_n_failed, warns when more than 10% fail, and errors
rather than returning meaningless intervals if all of them do. Its
documentation also gained a “Provenance” section crediting the R scripts
of A. M. Smith from which the implementation follows, and a “Relation to
Smith’s scripts” section explaining why estimates differ: this function
uses five mutually exclusive response cells, whereas those scripts fit
total target-present choices alongside culprit identifications, counting
the latter twice.
install_r4lineups_python() failed with reticulate’s
opaque
Suitable Python installation for creating a venv not found
when python_version named a version not present on the
machine, without saying how to obtain one. It now checks first, reports
which versions it did find, and gives the
reticulate::install_python() call needed.
Vignette.Rmd drew random numbers – simulated ages
and every bootstrap – without a seed, so its output changed on every
build. Seeded once in setup.
make_bayes_curves() now warns when a response
category has no observations in one arm. The likelihood ratio is then
unbounded and the prior-posterior curve is pinned flat at 0 or 1, which
looks like a plotting fault rather than a data problem.
The member-level branch of the effective-size calculation in
ppv_by_confidence() counted rejections as a lineup position
and did not tabulate over all positions, so unchosen members were
dropped. This branch was previously unreachable, as no dataset supplied
a lineup_member column.
plot_lineup_bias_distribution() and
plot_esize_distribution() no longer draw ragged histograms
with structural gaps. A bootstrapped lineup proportion is a count over a
fixed denominator, so the replicates fall on a lattice with spacing
1/n; binning them at a fixed 30 bins left bins that
contained no attainable value empty regardless of the data, and made
bins that straddled two attainable values twice as tall as their
neighbours. The bin width is now chosen to match the lattice, giving one
bar per attainable value. Estimates that are effectively continuous
still use 30 bins. Both functions gained bins and
binwidth arguments for explicit control.
plot_fullroc() labelled the plot
"Ordering method: " with nothing after it when given the
output of make_fullroc(), because the ordering method is
recorded at $order_used on a
make_fullroc_data() object but at
$summary$order_method on a make_fullroc()
object. Both shapes are now accepted, and an object carrying neither
raises an informative error.
plot() now works on the ROC and CAC objects.
make_rocdata(), make_cacdata() and
make_fullroc_data() returned unclassed lists, so
plot(x) dispatched to plot.default() and
failed; the package registered print methods for these
objects but no plot methods. Added
plot.lineup_roc(), plot.lineup_cac(),
plot.lineup_fullroc() and methods for the corresponding
*_data classes, all reusing the existing
make_roc_gg(), make_cac_gg() and
plot_fullroc() builders.
install_r4lineups_python() pinned no Python version,
so reticulate could select an interpreter newer than any for which
TensorFlow publishes wheels, and the install failed partway through with
No matching distribution found for tensorflow. It now
checks the interpreter before downloading anything and stops with an
actionable message, and gained a python_version argument
for building the virtual environment from a supported
interpreter.
New make_multid() implements Lee and Penrod’s (2019)
Multi-d’ model. From target-present and target-absent lineup data
(trial-level or counts) it computes d’(GI), d’(GFp), d’(IFa), d’(FaFp)
and d’(RaRp), the TP and TA lineup-bias indices, the criterion and the
diagnosticity ratio, with optional bootstrap intervals and a cumulative
breakdown by confidence. Because the model uses total filler rates, a
fair lineup does not give d’(IFa) = 0, so the result also reports the
d’(IFa) a perfectly fair lineup would give at the observed rejection
rate. compare_multid() compares two conditions, and
plot.multid() draws the decomposition of d’(GI), the
indices, or the indices by confidence. The tests reproduce Lee and
Penrod’s Tables 3, 4 and
9. See vignette("multid_model").
esize_T() gained an unbiased argument.
The default (FALSE) is the plug-in estimator as before, so
no existing result changes. unbiased = TRUE applies the
small-sample correction N(N-1) / sum(o(o-1)). The plug-in
form is biased downward, by approximately
k(k-1)/(N + k - 1): a perfectly fair six-person lineup
assessed with 50 choices returns about 5.47 rather than 6, and with 20
choices about 4.87. The correction removes essentially all of it in
simulation, at a variance cost that disappears by about 100 choices. The
bias, the evidence and the choice between the two forms are documented
in ?esize_T, with cross-references from
?esize_m and ?esize_T_ci_n.
New classify_lineup_choices() derives identification
outcomes (suspect ID, foil ID, rejection) from the raw record of which
lineup member each witness chose. Lineup data should record the member
chosen, with the collapsed label derived from it rather than stored in
its place, because foils are not interchangeable and a foil nobody ever
chooses is what makes a lineup unfair.
simulate_lineup_data() gained a
foil_bias argument, giving each foil its own mean memory
strength so that foils differ in plausibility rather than being drawn
from a common distribution. The default (NULL) preserves
the previous behaviour exactly.
simulate_lineup_data() now returns
lineup_member (the position chosen, 0 for a rejection) and
suspect_pos alongside identification. The
chosen position was already computed internally and discarded.
plot.power_analysis() shows a Monte Carlo interval
around the power curve, so that apparent dips between adjacent points
can be recognised as simulation noise, and returns the ggplot object
invisibly for further modification.
homog_diag() and homog_diag_boot() now
report the diagnosticity ratio and a confidence interval for each lineup
pair, not just the pooled mean and the chi-square. The chi-square is
Cochran’s Q, which measures how far the k ratios lie from their pooled
value; that is a different question from whether the pooled ratio
differs from 1, and the two can disagree sharply. Ratios of 6.0, 1.0 and
0.44 give Q = 45.6 (p < .0001) while pooling to 1.06, because they
cancel on the log scale — an output that reads as “highly significant,
but no diagnosticity” unless the per-pair values are visible. Both
functions now print a note stating whether the ratios are heterogeneous
and, when they are, that the pooled estimate averages quantities that
are not estimating a common parameter and should not be reported on its
own. The returned list gained a heterogeneous flag, and
homog_diag_boot() gained df and
p_value; the parameters element now carries
d, ln_d, var_lnd,
wi, lower and upper alongside the
existing cell counts. Both functions gained a conf.level
argument, which also sets the level for homog_diag_boot()’s
percentile intervals (default 0.95, unchanged from the previous
hard-coded value).
Added lineup_table(), which tabulates lineup choices
across all k positions so that null foils — members no mock
witness chose — appear as explicit zeroes rather than being silently
dropped by table(). Its print method names the null foils
and marks the nominated suspect. The result inherits from
table, so it can be passed to esize_m(),
esize_T() and the other table-taking functions
unchanged.
run_r4lineups_app() has been
rebuilt on bslib, with one module per analysis. Every
number it shows now comes from an exported r4lineups function; the app’s
own copy of the Malpass effective-size formula is gone..rds file.reticulate is
unavailable.run_r4lineups_app() checks that shiny,
bslib and DT are installed. bslib
and shinytest2 join Suggests; shinyWidgets and
jsonlite leave.make_racdata() labels response-time bins in plain
notation, e.g. [2390,4810] rather than
[2.39e+03,4.81e+03].
ROC and CAC print methods now say
"Plot: use plot(x), or save the object and extract x$plot"
rather than the cryptic "Plot available in $plot".
print.lineup_roc() and
print.lineup_fullroc() now report
"Target-present trials (N)" rather than
"Target-present lineups", which read as a count of lineups
when it is a count of witnesses.
make_roc_gg() now names the ROC variant in the
title, so a partial ROC is not mistaken for a full ROC, and records the
data source in the caption. It gained title and
data_source arguments.
eff_size_per_foils() had no @return
documentation at all. Documented that it returns a count of lineup
members, so it changes only in whole steps as conf varies
and is flat across ranges of conf in which no member’s
interval crosses 1/k.
eff_size_per_foils() and
lineup_boot_allprop() both described conf as
“Desired level of alpha”; it is a confidence level. Documented also that
lineup_boot_allprop() applies no multiplicity correction
across the k intervals it returns.
The main vignette has been substantially revised: corrected a
typo (line_df$linep_vec), a reference to a nonexistent
sample_data dataset, an inverted functional size formula
(D/N for N/D), a misstated proportion (“0.14%”
for 14%), a misattributed sentence (“He instead proposed” for “Tredoux
instead proposed”), and inline code spans that reflowed into a paragraph
instead of forming a code chunk. Added sections on null foils,
normal-theory versus bootstrap intervals, multiplicity across lineup
members, functional size exceeding nominal size, the
boot.ci() warnings, the *_data()/wrapper
distinction in the ROC and CAC functions, and raw data format for ROC
analysis. The homogeneity section now separates the two questions the
output answers — whether the k ratios agree with one another
(chi-square) and whether the average ratio differs from 1 (interval on
the pooled ratio) — with a worked heterogeneous example in which every
ratio is diagnostic, followed by a cautionary example in which ratios
pointing in opposite directions give a very large chi-square while
pooling to roughly 1. The unsupported version 1 face_sim()
section has been retired in favour of a pointer to the face similarity
vignette, and the vignette closes with a linked index of the other
vignettes.
install_pywitness() and
check_pywitness_deps() for an explicit, isolated
r4lineups-pywitness virtual environment. Installation uses
the immutable archive for audited pyWitness revision
e726dcfc09423d0e0ff7f46c8e3a711040293eba; package loading
and ordinary CRAN checks never install or initialize it.fit_lineup_models() for fair simultaneous
lineups, with Independent Observations, Ensemble, Integration, and
optional BEST-Rest fits; equal- and unequal-variance paths; zero or
estimated shared variance where identified; starting values; optimizer
controls; and explicit convergence diagnostics.lineup_model_comparison
objects with extracted parameters, observed and expected response cells,
Pearson statistics, degrees of freedom, p-values, confidence binning,
engine metadata, and comparison/fit/ROC plots. No live Python object is
retained.NA rather than
invalidly treating it as an MLE.Major release: a substantial expansion of the package from the 0.1.x fairness-measure toolkit into a full analysis suite for eyewitness identification research, plus a package-wide statistical audit.
pos_list now denotes the
actual suspect position for each TP/TA pair (or a two-element TP/TA
position pair). Homogeneity inference requires positive suspect-ID
cells, as in Tredoux’s uncorrected log-risk procedure.1 / lineup_size suspect and
(lineup_size - 1) / lineup_size filler components.lme4, images, or
other optional environments have explicit dependency/error-path coverage
and opt-in smoke-test exemptions."best_rest" remains accepted for compatibility
but is identified as a scale-equivalent parameterization of
Ensemble.fit_max_sdt() as a restricted, aggregate,
single-criterion, equal-variance Independent-Observations fit estimated
by minimum Pearson chi-squared. It is not a maximum-likelihood fit and
is not the complete confidence-based Wixted et al. (2018) model.This release implements high-priority features from pyWitness (Mickes et al., 2024), significantly expanding r4lineups’ analytical capabilities for eyewitness identification research.
compare_models(): Unified interface for fitting and
comparing multiple models
print(), summary(),
plot() for model comparison objectsformat_comparison_table(): Publication-ready comparison
tablescompare_pauc(): Rigorous statistical testing of ROC
curve differencessimulate_lineup_data(): Generate lineup identification
data
k / (k - 1) times Ensemble, so the
two fit identically when criteria are rescaled and must not be treated
as independent model evidencesimulate_power_analysis(): Determine required sample
sizes
fit_sdt_roc(): Extract d’, criteria, and variance ratio
from ROC data
print(), plot(),
summary() methodsmake_rocdata() for seamless
workflowvalidate_lineup_data(): Comprehensive data validation
standardize_lineup_data(): Convert data from various
formats
create_example_lineup_data(): Generate example data for
testing
make_rac(): Response time-accuracy characteristic
analysismake_racdata(): Compute RAC data with time binsmake_rac_gg(): Publication-ready RAC plotsprint.lineup_rac(): Print method for RAC objectsmake_fullroc() / make_fullroc_data() /
plot_fullroc(): full ROC analysis using all lineup
responses (Smith & Yang, 2020), with diagnosticity-ratio or a-priori
orderingcompute_eig() / make_eig() /
make_eig_data() / plot_eig(): Expected
Information Gain analysis (Starns et al., 2023)ppv_by_confidence() /
ppv_range_by_confidence() / make_ppv_range() /
plot_ppv_range(): positive predictive value across
confidence levels, with nominal- and effective-size correctionsfit_winter_2ht(): two-high-threshold MPT model for the
full 2x3 outcome table (Winter et al., 2022), with print(),
summary(), and plot() methodsboot_winter_2ht(): bootstrap confidence intervals for
2-HT parametersmake_calibration() /
make_calibration_by_condition() /
make_calibration_data(): confidence-accuracy calibration
(C, O/U, NRI)compute_anri() / bootstrap_anri() /
compare_anri(): Adjusted Normalized Resolution Index with
bootstrap inferencemake_bayes_curves(), make_bree_curve():
Bayesian prior-posterior curvesmake_utility_curves(),
make_utility_difference(), compare_utility():
expected-utility analysis (Lampinen et al., 2019)make_dpp(), compare_dpp(): Deviation from
Perfect Performance (Smith et al., 2018)esize_T_bayes(), func_size_bayes(),
diag_ratio_T_bayes(), calibration_bayes()sdt_compare(): Bayesian comparison of SDT parameters
between conditionspmax_filler(), dmax_filler(),
qmax_filler(), rmax_filler(),
max_filler_moments(), estimate_msdt_params()
(method-of-moments from rejection rates)fit_max_sdt() / compare_max_sdt():
restricted aggregate Independent-Observations/MAX fitting by minimum
Pearson chi-squared, with bootstrap CIs and nested-constraint
comparisonssdt_summary_from_counts(),
sdt_summary_variance(), compare_sdt_summary():
SDT comparisons when only summary counts are available
(Miller/Gourevitch or bootstrap variance)fit_sdt_glm() / fit_sdt_glmm() /
extract_sdt_metrics(): SDT parameter estimation via
probit/logit (mixed) modelsface_similarity(), lineup_similarity(),
batch_embeddings(), and related helpers compute facial
similarity with deep embeddings (ArcFace, FaceNet, …) through Python’s
deepface library (optional; requires reticulate)install_r4lineups_python() /
check_python_deps() for setuprun_r4lineups_app(): interactive Shiny interface to the
main analyseslineup_boot_allprop(), esize_boot_dist() and
friends)standardize_lineup_data())A component-by-component numerical audit of the statistical routines was completed for this release. Fixes:
fit_max_sdt()):
replaced the single boundary-prone optimizer start with a deterministic
multistart grid plus Nelder-Mead polish; corrected the chi-square
objective to use a proper outcome partition (correct-ID / filler-ID /
reject); floored expected counts instead of dropping cells; corrected
goodness-of-fit degrees of freedom (df = 3 x n_conditions - n_params);
convergence is now reported honestlyorder = "apriori" with
conf_bins (labels were coerced to NA, silently
breaking within-decision ordering)ppv_by_confidence(correction = "effective") computed the
wrong effective size in the no-member-data fallback; now uses the
pseudo-table counts directlyconfidence_scale = c("auto", "0-1", "0-100") argument to
make_calibration_data() and all callers; auto
now warns when confidence looks like a Likert/0-10 scale instead of
silently dividing by 100summary() now reports a saturation note and the
observed-vs-expected discrepancy as a convergence checkln_diag_ratio()entropy() definition that
relied on file collation ordersimulation_power_analysis.Rmd: Complete guide to data
simulation and power analysismodel_comparison.Rmd: Comprehensive model comparison
workflowspauc_statistical_comparison.Rmd: Statistical testing of
ROC curvesrac_analysis.Rmd: Response time-accuracy analysisfullroc_analysis.Rmd: Full ROC methodologyinformation_error_rate_analysis.Rmd: EIG and PPV-range
analysiscalibration_decision_analysis.Rmd: Calibration,
utility, DPP, and ANRIwinter_2ht_model.Rmd: Winter 2-HT modelbayesian_inference.Rmd: Bayesian posterior inference
for lineup measuresmsdt_model.Rmd: Multi-item signal detection theorysdt_summary_comparisons.Rmd: SDT comparisons from
summary countssdt_glm_analysis.Rmd: SDT via GLM/GLMMface_similarity.Rmd: Deep-learning face similarityAll vignettes include: * Complete workflows from start to finish * Real-world examples and use cases * Best practices and common pitfalls * Publication-ready guidance * References to original literature
model_comparison_example.R: 8 comprehensive
examplespauc_comparison_example.R: 8 comprehensive examples
with interpretationsimulation_power_analysis_example.R: Complete
tutorialrac_example.R: RAC analysis demonstration.extract_counts_from_df() in
winter_2ht.R to properly handle dataframe inputs
unname() to ensure clean namestest-model-comparison.R: Model comparison framework
teststest-pauc-comparison.R: pAUC statistical comparison
teststest-simulation.R: Data simulation and power analysis
testsgridExtra to Imports (used by
plot.model_comparison())testthat (>= 3.0.0) to Suggests for unit
testingNew features based on:
Mickes, L., Seale-Carlisle, T. M., Chen, X., & Boogert, S. (2024). pyWitness 1.0: A python eyewitness identification analysis toolkit. Behavior Research Methods, 56, 1533-1550.
Winter, K., Menne, N. M., Bell, R., & Buchner, A. (2022). Experimental validation of a multinomial processing tree model for analyzing eyewitness identification decisions. Scientific Reports, 12, 15571.
Starns, J. J., Cohen, A. L., & Rotello, C. M. (2023). A complete method for assessing the effectiveness of eyewitness identification procedures: Expected information gain. Psychological Review, 130(3), 677–719.
Smith, A. M., Yang, Y., & Wells, G. L. (2020). Distinguishing between investigator discriminability and eyewitness discriminability. Perspectives on Psychological Science, 15(3), 589-607.
Seale-Carlisle, T. M., Wetmore, S. A., Flowe, H. D., & Mickes, L. (2019). Designing police lineups to maximize memory performance. Journal of Applied Research in Memory and Cognition, 8(4), 420-428.
Previous release. See earlier documentation for details.