The Kruskal-Wallis test is for independent samples. So, your use of the phrase "pairwise comparison" is a bit confusing. I wonder if you should be using the Friedman Two-Way ANOV? Alternatively, you might use several paired t-tests with the results adjusted for multiple tests.

Hi James C. Miller , this was using the Dunn's test for multiple comparisons

(https://journals.sagepub.com/doi/pdf/10.1177/1536867X1501500117#:~:text=Dunn's%20test%20is%20the%20appropriate%20procedure%20following%20a%20Kruskal%E2%80%93Wallis,framework%20of%20the%20hypothesis%20test.).

I have attached an example of the table i am referring to below.

Much clearer. The Dunn's Test tells you which paired comparisons caused the Kruskal-Wallis test result to be significant. Use the adjusted significance to control for multiple comparisons. Does your software tell you what method was used for adjustment: Bonferroni, Scheffe, or Tukey? That method should be reported. So, I see several paired comparisons that were statistically significant (adjusted), affecting the Kruskal-Wallis significance: text-photo (p < 0.001), text-photo carousel (p = 0.002), video-photo (p < 0.001), video-photo carousel (p = 0.001), text-long form video ( p < 0.05), and video-long form video ( p < 0.05) (please double-check my list).

Discuss each of the six statistically significant differences. First, were the absolute differences for each of these six paired comparisons meaningful? Explain those six differences, be they large or small. Then explain why these six differences may have occurred. Suggest designs for follow-on investigations of these six differences to help validate your findings. Then, go back and discuss, in general, the other 25+ comparisons that were not significantly different. How did the natures of the statistically significant comparisons differ from those of the non-significant comparisons? Why were some significant and not others? This is, of course, a bit of a guessing game, but take a shot anyway.

The phrase "pairwise comparisons" is clear here, and Dunn test (1964) is an appropriate post-hoc test for the Kruskal-Wallis test.

In reality, it's the choice of the analyst whether to report unadjusted or adjusted p-values.

Adjustments generally fall under two categories: those for family-wise error rate (FWER) and those for false discovery rate (FDR).

Essentially, using the adjusted p-values, you are avoiding making too many "type-I errors", or "false-positives" *, when looking at all the hypothesis tests for the pairwise comparisons. These increase just by chance as make more and more hypothesis tests. But doing so, you may miss potentially significant comparisons.

On the other hand, using the unadjusted p-values prioritizes finding more potentially significant comparisons over avoiding type-I errors.

There are situations where either approach is desirable. Presenting your reasoning and justification is helpful for the reader.

I know, not very helpful guidance, but that's the reality. It's like deciding into which investment vehicle to put your money. You're gambling against being too conservative on the one hand and not getting the full potential return, or being too risky on the other hand and not getting the best return.

The only way out of this trap in the statistics example is to have a larger sample size, avoid more confounding factors, use more precise measurements, and so on.

_____

* Scare-quotes here, since these terms might not quite represent what the actual statistical analysis is telling us.

Hi James C. Miller and Sal Mangiafico , thank you both for your answers! It has helped create some much appreciated clarity!