Benchmark timings for mkin

Johannes Ranke

Last change 13 May 2020 (rebuilt 2022-05-18)

Source: vignettes/web_only/benchmarks.rmd

Each system is characterized by its CPU type, the operating system type and the mkin version. Currently only values for one system are available. A compiler was available, so if no analytical solution was available, compiled ODE models are used.

Test cases

Parent only:

FOCUS_C <- FOCUS_2006_C
FOCUS_D <- subset(FOCUS_2006_D, value != 0)
parent_datasets <- list(FOCUS_C, FOCUS_D)

t1 <- system.time(mmkin_bench(c("SFO", "FOMC", "DFOP", "HS"), parent_datasets))[["elapsed"]]
t2 <- system.time(mmkin_bench(c("SFO", "FOMC", "DFOP", "HS"), parent_datasets,
    error_model = "tc"))[["elapsed"]]

One metabolite:

SFO_SFO <- mkinmod(
  parent = mkinsub("SFO", "m1"),
  m1 = mkinsub("SFO"))
FOMC_SFO <- mkinmod(
  parent = mkinsub("FOMC", "m1"),
  m1 = mkinsub("SFO"))
DFOP_SFO <- mkinmod(
  parent = mkinsub("FOMC", "m1"),
  m1 = mkinsub("SFO"))
t3 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D)))[["elapsed"]]
t4 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D),
    error_model = "tc"))[["elapsed"]]
t5 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D),
    error_model = "obs"))[["elapsed"]]

Two metabolites, synthetic data:

m_synth_SFO_lin <- mkinmod(parent = mkinsub("SFO", "M1"),
                           M1 = mkinsub("SFO", "M2"),
                           M2 = mkinsub("SFO"),
                           use_of_ff = "max", quiet = TRUE)

m_synth_DFOP_par <- mkinmod(parent = mkinsub("DFOP", c("M1", "M2")),
                           M1 = mkinsub("SFO"),
                           M2 = mkinsub("SFO"),
                           use_of_ff = "max", quiet = TRUE)

SFO_lin_a <- synthetic_data_for_UBA_2014[[1]]$data

DFOP_par_c <- synthetic_data_for_UBA_2014[[12]]$data

t6 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a)))[["elapsed"]]
t7 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c)))[["elapsed"]]

t8 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a),
    error_model = "tc"))[["elapsed"]]
t9 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c),
    error_model = "tc"))[["elapsed"]]

t10 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a),
    error_model = "obs"))[["elapsed"]]
t11 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c),
    error_model = "obs"))[["elapsed"]]
mkin_benchmarks[system_string, paste0("t", 1:11)] <-
  c(t1, t2, t3, t4, t5, t6, t7, t8, t9, t10, t11)
save(mkin_benchmarks, file = "~/git/mkin/vignettes/web_only/mkin_benchmarks.rda")

Results

Benchmarks for all available error models are shown.

Parent only

Constant variance (t1) and two-component error model (t2) for four models fitted to two datasets, i.e. eight fits for each test.

t1 t2
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.48.1 3.610 11.019
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.1 8.184 22.889
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.2 7.064 12.558
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.3 7.296 21.239
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.4 5.936 20.545
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.2 1.714 3.971
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.3 1.752 4.156
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.4 1.786 3.729
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.0.3 1.881 3.504
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.0.4 1.867 3.450
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.1.0 1.938 3.517
Linux, Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz, mkin version 1.1.0 1.621 3.123

One metabolite

Constant variance (t3), two-component error model (t4), and variance by variable (t5) for three models fitted to one dataset, i.e. three fits for each test.

t3 t4 t5
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.48.1 3.764 14.347 9.495
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.1 4.649 13.789 6.395
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.2 4.786 8.461 5.675
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.3 4.510 13.805 7.386
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.4 4.446 15.335 6.002
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.2 1.402 6.174 2.764
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.3 1.430 6.615 2.878
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.4 1.397 7.251 2.810
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.0.3 1.430 6.344 2.798
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.0.4 1.415 6.364 2.820
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.1.0 1.470 6.508 2.894
Linux, Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz, mkin version 1.1.0 3.319 18.209 7.773

Two metabolites

Constant variance (t6 and t7), two-component error model (t8 and t9), and variance by variable (t10 and t11) for one model fitted to one dataset, i.e. one fit for each test.

t6 t7 t8 t9 t10 t11
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.48.1 2.623 4.587 7.525 16.621 8.576 31.267
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.1 2.542 4.128 4.632 8.171 3.676 5.636
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.2 2.723 4.478 4.862 7.618 3.579 5.574
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.3 2.643 4.374 7.020 11.124 5.388 7.365
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.49.4 2.635 4.259 4.737 7.763 3.427 5.626
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.2 0.777 1.236 1.332 2.872 2.069 2.987
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.3 0.858 1.264 1.333 2.984 2.113 3.073
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 0.9.50.4 0.783 1.282 1.486 3.815 1.958 3.105
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.0.3 0.763 1.244 1.457 3.054 1.923 2.839
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.0.4 0.785 1.252 1.466 3.091 1.936 2.826
Linux, AMD Ryzen 7 1700 Eight-Core Processor, mkin version 1.1.0 0.810 1.264 1.503 3.096 1.984 2.847
Linux, Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz, mkin version 1.1.0 2.348 3.579 4.254 11.315 5.981 8.368