2024-11-28: Traci Sitzmann and Stefanie Johnson

• Discussion leader: Elin Kronander

• Date: 2024-11-28

• Paper: [sitzmann_and_johnson_2014] Traci Sitzmann and Stefanie Johnson. "The paradox of seduction by irrelevant details: How irrelevant information helps and hinders self-regulated learning." Learning and Individual Differences (2014): 1-11. Download page

Notes by Richel

My questions

• What grade would you give the paper on scale from 1 to 10?
• Do we all know a teacher that uses seductive details?
• Do we really believe that they came up with their model and hypotheses first, after which all was confirmed to be true?
• How would you summarize the paper?
• Do you believe the findings in this paper?
• Do you think SATs are useless, as per the authors. Here is their quote: 'the fact that learning has declined and stagnated during the twenty-five or so years that higher education has relied on student opinion as a measure of ‘good’ teaching speaks for itself'

My scribbles

First sentence:

Instructors often rely on seductive details, such as jokes, stories, and video clips, to keep trainees entertained

Is this the case? How often? Is this to entertain the trainees or is it a way of teachers to enjoy teaching more?

From the Visible Learning table, only humor is directly related to this paper, with an effect size of 0.04, which is low.

Rank	Influence	ES dec 2017	ES aug 2017	Subdomain	Domain
12	Teacher credibility	0.9	0.9	Teacher attributes	TEACHER
67	Interactive video methods	0.54	0.54	Implementations using technologies	TEACHING: Focus on implementation method
178	Teacher personality attributes	0.23	0.23	Teacher attributes	TEACHER
182	Teacher verbal ability	0.22	0.22	Teacher attributes	TEACHER
230	Humor	0.04	0.04	Teaching/instructional strategies	TEACHING: Focus on teaching/instructional strategies

Main findings: Seductive details ...

• indirectly improved learning performance by reducing negative affect
• indirectly hindered learning performance by increasing the speed of reviewing
• indirectly hindered learning performance by decreasing time on task.

First sentence in the introduction:

Trainers and educators are typically evaluated based on whether trainees enjoy their courses and feel the course is useful

I think that asking a student for satisfaction is an uninteresting metric. So this paper assumes that teachers use seductive details to improve a half-useless metric?

[...] including irrelevant information in training makes learning more engaging, but impairs retention, problem solving, and training transfer [many refs] Seductive details have garnered interest from researchers in education and psychology because of the counterintuitiveness of the effect.

Why is this counterintuitive? When I spend less time on problem solving (e.g. because I tell a fun anecdote), I expect the learners to be worse on problem solving. What is counterintuitive in that?

In education, affect is broadly defined as the attitudes, emotions, and values present in an educational environment

Do we really believe that they came up with their model and hypotheses first, after which all was confirmed to be true? For example, the model assumes a reduction in negative affect. Why not use 'affect' instead of 'negative affect'? If 'negative affect' is used, why not add 'positive affect?'

quickly paging through training content is one of the least effective learning strategies (Sitzmann, 2012; Sitzmann & Johnson, 2012a, 2012b).

Interesting!

H4. Seductive details have a positive within-person effect on the number of slides reviewed per minute, such that trainees page through the content quicker in modules that contain seductive details than in modules without seductive details.

I could argue the other way around too: seductive details cause trainees to page through the content slower, because they look for those fun distractions.

However, “the fact that learning has declined and stagnated during the twenty-five or so years that higher education has relied on student opinion as a measure of ‘good’ teaching speaks for itself”

Does it? What about governmental financial support, status of the teacher, tendency of politicians to do another education reform? The suggestion that using student opinions to judge good teaching is the cause of a decline in learning does not speak for itself to me.

trainee reactions tell us more about the trainees than about the course they are rating (Adams, 1997; Goldman, 1993). [...] Thus, ineffective trainee behavior is related to dissatisfaction with the course, and it may be time for aca- demic and training institutions to realize that trainee reactions are a poor indicator of instructional effectiveness.

So no 'satisfaction' on evaluations :-)

They missed the opportunity to quote [Stroebe, 2020] with the all-saying title: 'Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation'.

Table 1: there are 9 variables, hence ((99)-9=) 72 tests being conducted. Using a alpha of 0.05 this means that one expects to find (0.05 72 = 3.6 =) 4 significant relations by chance. It happened 24x that a p values was below 0.05, but the actual p values were not given. There is no correction for multiple testing. There is no mention about the 4 expected false positives either.

p-values were not supplied, I asked at Nov 2nd.

The paper cannot be found in Google Scholar?

Figure 2 is quite deceiving. Please start you y axis at zero, else it shows you try to deceive.

Variables used:

• Seductive details
• Pretraining knowledge:
• Trainee reactions
• Negative affect
• Time on task
• Slides reviewed per minute
• Attention
• Learning performance
• Attrition

My summary:

The paper investigates the effect of seductive details. It provides weak evidence that seductive details have a small negative effect on some outcomes.

Next paper

• How to do an effective SET ('Student Evaluation of Teacher')
• How to effectively evaluate a teacher
• How to effectively teach online

References

• [Stroebe, 2020] Stroebe, W. (2020). Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis. Basic and Applied Social Psychology, 42(4), 276–294. https://doi.org/10.1080/01973533.2020.1756817