Each system is characterized by its CPU type, the operating system type and the mkin version. Currently only values for one system are available. A compiler was available, so if no analytical solution was available, compiled ODE models are used.

Test cases

Parent only:

FOCUS_C <- FOCUS_2006_C
FOCUS_D <- subset(FOCUS_2006_D, value != 0)
parent_datasets <- list(FOCUS_C, FOCUS_D)

t1 <- system.time(mmkin_bench(c("SFO", "FOMC", "DFOP", "HS"), parent_datasets))[["elapsed"]]
t2 <- system.time(mmkin_bench(c("SFO", "FOMC", "DFOP", "HS"), parent_datasets,
    error_model = "tc"))[["elapsed"]]

One metabolite:

SFO_SFO <- mkinmod(
  parent = mkinsub("SFO", "m1"),
  m1 = mkinsub("SFO"))
FOMC_SFO <- mkinmod(
  parent = mkinsub("FOMC", "m1"),
  m1 = mkinsub("SFO"))
DFOP_SFO <- mkinmod(
  parent = mkinsub("FOMC", "m1"),
  m1 = mkinsub("SFO"))
t3 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D)))[["elapsed"]]
t4 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D),
    error_model = "tc"))[["elapsed"]]
t5 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D),
    error_model = "obs"))[["elapsed"]]

Two metabolites, synthetic data:

m_synth_SFO_lin <- mkinmod(parent = mkinsub("SFO", "M1"),
                           M1 = mkinsub("SFO", "M2"),
                           M2 = mkinsub("SFO"),
                           use_of_ff = "max", quiet = TRUE)

m_synth_DFOP_par <- mkinmod(parent = mkinsub("DFOP", c("M1", "M2")),
                           M1 = mkinsub("SFO"),
                           M2 = mkinsub("SFO"),
                           use_of_ff = "max", quiet = TRUE)

SFO_lin_a <- synthetic_data_for_UBA_2014[[1]]$data

DFOP_par_c <- synthetic_data_for_UBA_2014[[12]]$data

t6 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a)))[["elapsed"]]
t7 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c)))[["elapsed"]]

t8 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a),
    error_model = "tc"))[["elapsed"]]
t9 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c),
    error_model = "tc"))[["elapsed"]]

t10 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a),
    error_model = "obs"))[["elapsed"]]
t11 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c),
    error_model = "obs"))[["elapsed"]]
mkin_benchmarks[system_string, paste0("t", 1:11)] <-
  c(t1, t2, t3, t4, t5, t6, t7, t8, t9, t10, t11)
save(mkin_benchmarks, file = "~/git/mkin/vignettes/web_only/mkin_benchmarks.rda")

Results

Currently, we only have benchmark information on one system, therefore only the mkin version is shown with the results below. Timings are in seconds, shorter is better. All results were obtained by serial, i.e. not using multiple computing cores.

Benchmarks for all available error models are shown.

Parent only

Constant variance (t1) and two-component error model (t2) for four models fitted to two datasets, i.e. eight fits for each test.

mkin version t1 [s] t2 [s]
0.9.48.1 3.610 11.019
0.9.49.1 8.184 22.889
0.9.49.2 7.064 12.558
0.9.49.3 7.296 21.239
0.9.49.4 5.936 20.545
0.9.50.2 1.714 3.971
0.9.50.3 1.752 4.156
0.9.50.4 1.786 3.729
1.0.0 1.770 3.703
1.0.3 1.582 3.476

One metabolite

Constant variance (t3), two-component error model (t4), and variance by variable (t5) for three models fitted to one dataset, i.e. three fits for each test.

mkin version t3 [s] t4 [s] t5 [s]
0.9.48.1 3.764 14.347 9.495
0.9.49.1 4.649 13.789 6.395
0.9.49.2 4.786 8.461 5.675
0.9.49.3 4.510 13.805 7.386
0.9.49.4 4.446 15.335 6.002
0.9.50.2 1.402 6.174 2.764
0.9.50.3 1.430 6.615 2.878
0.9.50.4 1.397 7.251 2.810
1.0.0 1.373 7.127 2.762
1.0.3 1.417 6.418 2.823

Two metabolites

Constant variance (t6 and t7), two-component error model (t8 and t9), and variance by variable (t10 and t11) for one model fitted to one dataset, i.e. one fit for each test.

mkin version t6 [s] t7 [s] t8 [s] t9 [s] t10 [s] t11 [s]
0.9.48.1 2.623 4.587 7.525 16.621 8.576 31.267
0.9.49.1 2.542 4.128 4.632 8.171 3.676 5.636
0.9.49.2 2.723 4.478 4.862 7.618 3.579 5.574
0.9.49.3 2.643 4.374 7.020 11.124 5.388 7.365
0.9.49.4 2.635 4.259 4.737 7.763 3.427 5.626
0.9.50.2 0.777 1.236 1.332 2.872 2.069 2.987
0.9.50.3 0.858 1.264 1.333 2.984 2.113 3.073
0.9.50.4 0.783 1.282 1.486 3.815 1.958 3.105
1.0.0 0.775 1.269 1.467 3.767 1.919 2.942
1.0.3 0.783 1.258 1.611 3.095 1.951 2.859