Benchmark timings for saem.mmkin

Each system is characterized by operating system type, CPU type, mkin version, saemix version and R version. A compiler was available, so if no analytical solution was available, compiled ODE models are used.

Every fit is only performed once, so the accuracy of the benchmarks is limited.

For the initial mmkin fits, we use all available cores.

n_cores <- parallel::detectCores()

Test data

Please refer to the vignette dimethenamid_2018 for an explanation of the following preprocessing.

dmta_ds <- lapply(1:7, function(i) {
  ds_i <- dimethenamid_2018$ds[[i]]$data
  ds_i[ds_i$name == "DMTAP", "name"] <-  "DMTA"
  ds_i$time <- ds_i$time * dimethenamid_2018$f_time_norm[i]
  ds_i
})
names(dmta_ds) <- sapply(dimethenamid_2018$ds, function(ds) ds$title)
dmta_ds[["Elliot"]] <- rbind(dmta_ds[["Elliot 1"]], dmta_ds[["Elliot 2"]])
dmta_ds[["Elliot 1"]] <- NULL
dmta_ds[["Elliot 2"]] <- NULL

Test cases

Parent only

parent_mods <- c("SFO", "DFOP", "SFORB", "HS")
parent_sep_const <- mmkin(parent_mods, dmta_ds, quiet = TRUE, cores = n_cores)
parent_sep_tc <- update(parent_sep_const, error_model = "tc")

t1 <- system.time(sfo_const <- saem(parent_sep_const["SFO", ]))[["elapsed"]]
t2 <- system.time(dfop_const <- saem(parent_sep_const["DFOP", ]))[["elapsed"]]
t3 <- system.time(sforb_const <- saem(parent_sep_const["SFORB", ]))[["elapsed"]]
t4 <- system.time(hs_const <- saem(parent_sep_const["HS", ]))[["elapsed"]]
t5 <- system.time(sfo_tc <- saem(parent_sep_tc["SFO", ]))[["elapsed"]]
t6 <- system.time(dfop_tc <- saem(parent_sep_tc["DFOP", ]))[["elapsed"]]
t7 <- system.time(sforb_tc <- saem(parent_sep_tc["SFORB", ]))[["elapsed"]]
t8 <- system.time(hs_tc <- saem(parent_sep_tc["HS", ]))[["elapsed"]]

anova(
  sfo_const, dfop_const, sforb_const, hs_const,
  sfo_tc, dfop_tc, sforb_tc, hs_tc) |> kable(, digits = 1)

	npar	AIC	BIC	Lik
sfo_const	5	796.3	795.3	-393.2
sfo_tc	6	798.3	797.1	-393.2
dfop_const	9	709.4	707.5	-345.7
sforb_const	9	710.0	708.1	-346.0
hs_const	9	713.7	711.8	-347.8
dfop_tc	10	670.1	668.0	-325.0
sforb_tc	10	662.9	660.8	-321.4
hs_tc	10	667.2	665.1	-323.6

The above model comparison suggests to use the SFORB model with two-component error. For comparison, we keep the DFOP model with two-component error, as it competes with SFORB for biphasic curves.

illparms(dfop_tc)

## [1] "sd(log_k2)"

illparms(sforb_tc)

## [1] "sd(log_k_DMTA_bound_free)"

For these two models, random effects for the transformed parameters k2 and k_DMTA_bound_free could not be quantified.

One metabolite

We remove parameters that were found to be ill-defined in the parent only fits.

one_met_mods <- list(
  DFOP_SFO = mkinmod(
    DMTA = mkinsub("DFOP", "M23"),
    M23 = mkinsub("SFO")),
  SFORB_SFO = mkinmod(
    DMTA = mkinsub("SFORB", "M23"),
    M23 = mkinsub("SFO")))

one_met_sep_const <- mmkin(one_met_mods, dmta_ds, error_model = "const",
  cores = n_cores, quiet = TRUE)
one_met_sep_tc <- mmkin(one_met_mods, dmta_ds, error_model = "tc",
  cores = n_cores, quiet = TRUE)

t9 <- system.time(dfop_sfo_tc <- saem(one_met_sep_tc["DFOP_SFO", ],
    no_random_effect = "log_k2"))[["elapsed"]]
t10 <- system.time(sforb_sfo_tc <- saem(one_met_sep_tc["SFORB_SFO", ],
    no_random_effect = "log_k_DMTA_bound_free"))[["elapsed"]]

Three metabolites

For the case of three metabolites, we only keep the SFORB model in order to limit the time for compiling this vignette, and as fitting in parallel may disturb the benchmark. Again, we do not include random effects that were ill-defined in previous fits of subsets of the degradation model.

illparms(sforb_sfo_tc)

three_met_mods <- list(
  SFORB_SFO3_plus = mkinmod(
    DMTA = mkinsub("SFORB", c("M23", "M27", "M31")),
    M23 = mkinsub("SFO"),
    M27 = mkinsub("SFO"),
    M31 = mkinsub("SFO", "M27", sink = FALSE)))

three_met_sep_tc <- mmkin(three_met_mods, dmta_ds, error_model = "tc",
  cores = n_cores, quiet = TRUE)

t11 <- system.time(sforb_sfo3_plus_const <- saem(three_met_sep_tc["SFORB_SFO3_plus", ],
    no_random_effect = "log_k_DMTA_bound_free"))[["elapsed"]]

Results

Benchmarks for all available error models are shown. They are intended for improving mkin, not for comparing CPUs or operating systems. All trademarks belong to their respective owners.

Parent only

Constant variance for SFO, DFOP, SFORB and HS.

CPU	OS	mkin	saemix	t1	t2	t3	t4
Ryzen 7 1700	Linux	1.2.0	3.2	2.140	4.626	4.328	4.998
Ryzen 7 1700	Linux	1.2.2	3.2	2.427	4.550	4.217	4.851
Ryzen 9 7950X	Linux	1.2.1	3.2	1.352	2.813	2.401	2.074
Ryzen 9 7950X	Linux	1.2.2	3.2	1.328	2.738	2.336	2.023
Ryzen 9 7950X	Linux	1.2.3	3.2	1.118	2.036	2.010	2.088
Ryzen 9 7950X	Linux	1.2.3	3.2	1.419	2.374	1.926	2.398
Ryzen 9 7950X	Linux	1.2.4	3.2	0.972	2.550	1.987	2.055
Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Linux	1.2.6	3.2	2.998	6.523	6.126	4.721
Ryzen 9 7950X	Linux	1.2.6	3.2	1.135	2.025	2.406	2.478

Two-component error fits for SFO, DFOP, SFORB and HS.

CPU	OS	mkin	saemix	t5	t6	t7	t8
Ryzen 7 1700	Linux	1.2.0	3.2	5.678	7.441	8.000	7.980
Ryzen 7 1700	Linux	1.2.2	3.2	5.352	7.201	8.174	8.401
Ryzen 9 7950X	Linux	1.2.1	3.2	2.388	3.033	3.532	3.310
Ryzen 9 7950X	Linux	1.2.2	3.2	2.341	2.968	3.465	3.341
Ryzen 9 7950X	Linux	1.2.3	3.2	2.159	3.584	3.307	3.460
Ryzen 9 7950X	Linux	1.2.3	3.2	2.348	3.134	3.253	3.530
Ryzen 9 7950X	Linux	1.2.4	3.2	2.127	3.587	3.433	3.595
Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Linux	1.2.6	3.2	5.070	8.464	8.525	7.599
Ryzen 9 7950X	Linux	1.2.6	3.2	2.161	3.325	3.669	3.153

One metabolite

Two-component error for DFOP-SFO and SFORB-SFO.

CPU	OS	mkin	saemix	t9	t10
Ryzen 7 1700	Linux	1.2.0	3.2	24.465	800.266
Ryzen 7 1700	Linux	1.2.2	3.2	25.193	798.580
Ryzen 9 7950X	Linux	1.2.1	3.2	11.247	285.216
Ryzen 9 7950X	Linux	1.2.2	3.2	11.242	284.258
Ryzen 9 7950X	Linux	1.2.3	3.2	11.796	216.012
Ryzen 9 7950X	Linux	1.2.3	3.2	12.841	292.688
Ryzen 9 7950X	Linux	1.2.4	3.2	12.160	265.934
Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Linux	1.2.6	3.2	30.168	748.675
Ryzen 9 7950X	Linux	1.2.6	3.2	12.007	286.757

Three metabolites

Two-component error for SFORB-SFO3-plus

CPU	OS	mkin	saemix	t11
Ryzen 7 1700	Linux	1.2.0	3.2	1289.198
Ryzen 7 1700	Linux	1.2.2	3.2	1312.445
Ryzen 9 7950X	Linux	1.2.1	3.2	489.939
Ryzen 9 7950X	Linux	1.2.2	3.2	482.970
Ryzen 9 7950X	Linux	1.2.3	3.2	392.364
Ryzen 9 7950X	Linux	1.2.3	3.2	483.027
Ryzen 9 7950X	Linux	1.2.4	3.2	456.252
Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Linux	1.2.6	3.2	1235.028
Ryzen 9 7950X	Linux	1.2.6	3.2	480.577

Johannes Ranke

Last change 17 February 2023 (rebuilt 2023-11-16)

Test data

Test cases

Parent only

One metabolite

Three metabolites

Results

Parent only

One metabolite

Three metabolites