## Bayesian Repeated Measures ANOVA: What n do you use?

*in*Literature

Bayesian statistics are interesting as they give credence to the idea that stats can be interpretable and statistically meaningful without having to do post-hoc corrections and analyses. Two papers (and one review), one from Masson (2011) and one from Wagenmakers (2007) (with a nice review of the current debate by Jarosz & Wiley (2014)), discuss how to compute Bayesian Factors using sums of squares, which were computed through a traditional repeated measures analysis of variance. Three parts of computing a Bayesian Factor are: 1) all of your sums of squares, 2) your *k* degrees of freedom (for me, this is the amount of different conditions), and 3) your *n* number of participants/subjects (I study humans, so I use the word "participants").

Wagenmakers makes the argument for simply using your *n* value in the Bayesian rmANOVA. Masson believes that you need to account for each participant in every free condition (aka n X *k *- 1, where *k* is the amount of conditions in your experiment), which increases your *n* value.

Now, there is no set rule as to how to compute a Bayesian rmANOVA. So I tried both methods using the values from Masson's paper example 1. Here's a screenshot:

Using Masson's *n*, I end up with something similar to his result: BIC = -18.5; BF = 0.000096 (Masson reports BIC = -14.1, BF = 0.00086). When I convert this into probability, I get p(Ha) > 0.99 (Masson reports the same probability).

Using Wagenmakers' *n*, I get BIC = -7.75; BF = 0.021; p(Ha) > 0.98

In this example, the probabilities are so near each other, this particular example doesn't really argue for or against either Masson's or Wagenmakers' *n* value.

So I tried doing this with some of my own data. I have 30 participants going through 2 tasks divided at 2 different time epochs. If I am doing this right, *k* - 1 = 3. For reference: *F* = 10.27, *p* < .005, partial *η*^2 = 0.268, SSwithin = 0.00000082, SSbetween = 0.0012, SSresidual = 0.0033 (making SStotal = 0.0046).

Using Masson's *n*: BIC = -19.12; BF = 0.000071; p(Ha) > 0.99

Using Wagenmaker's *n*: BIC = -2.57; BF = 0.28; p(Ha) > 0.78

This is definitely different. One of the advantages to Bayesian statistics is the ability to give a more "straight-forward" report of a situation. In my situation, I'm asking *are there group differences**?* and my Bayesian factors are both strong in support of my hypothesis. However, Masson's *n* indicates the probability of group differences given this experiment are practically 100%, whereas Wagenmaker's *n* says group differences given this experiment are more like 78%.

Still, both percentages are very high -- especially for fNIRS data. But this is all driven by whether you consider your participants to be in every condition separately (Masson's *n*) or if you consider your participants to be simply participants (Wagenmaker's *n*).

Since my traditional statistical value is significant, and my partial η^2 is a moderate-ish 0.27, I should be expecting a strong but moderate Bayes Factor and probability. Since my Wagenmaker's BF = 0.28 (making my inverse BF = 3.61), and since my probability for my alternative hypothesis in this experiment 78%, and since my interpreted support for my alternative hypothesis is "positive" and "substantial" (based on Jarosz & Wiley's interpretation table), I am more comfortable calculating the more conservative of the two *n*'s, the Wagenmaker's *n*.

On the other hand, the interpretation for Masson's *n* would be an inverse BF > 100000, with "very strong" and "decisive" support for my alternative hypothesis. Based on my traditional effect size, I can't really see the leap from a moderate-at-best interpretation to a guaranteed-or-your-money-back interpretation.

Really, either way has some strong logic behind it. I personally side on the side of caution when trying to discuss my results. I don't know how kosher it would be to use both of these numbers as a range (e.g. my likelihood ranges between 78% and 99%), but that could be a fair way to incorporate both computations. Again, I'm not a stats expert, nor a Bayesian expert, so I don't really know. I would suggest reading Masson's paper, Wagenmaker's paper, and the review from Jarosz & Wiley if you want a stronger grasp on Bayes Factor computation! Good easy reads!