2026-03-04¶

Date: 2026-03-04
Lead discussion: Richel Bilderbeek
Paper: Lockman, Alison S., and Barbara R. Schirmer. "Online instruction in higher education: Promising, research-based, and evidence-based practices." Journal of Education and e-Learning Research 7.2 (2020): 130-152. Article

Talk¶

Why I chose this paper

Answers

I wanted a paper that is useful for me, so I searched for 'evidence based practices in online teaching'. This was the best match that was most recent.

What grade on a scale from 1 (worst) to 10 (best) would you give this paper?

Answers

R: 7
...
...
...

How would you summarize this paper in one line?

Answers

R: The most up-to-date overview of the literature regarding evidence-based best practices in online instructional practice, concluding that only 'faculty feedback' is a practiced that is evidence-based.
...
...
...

How would you praise the paper?

Answers

R: It is open access. It dares to draw an uninspiring conclusion
...
...
...

How would you criticise the paper?

Answers

R: In 2026, it feels outdated. I feel it is quite a dull read. I want to read about the earlier evidence-based best practices too
...
...
...

A review paper gives only a first general impression: some studies will be false/misleading. Was there a finding presented that, upon closer investigation, you think is misleading?

Answers

R: [Uttl et al., 2017] is a meta analysis that shows that a relation between course quality and student satisfaction is likelier to be found in studies that (1) have less learners, (2) have less rigorous analyses, (3) are in pedagogy journals. This relation disappears for bigger studies with a more rigorous analysis. It convinced me there is no relationship between course quality and learner satisfaction. When this paper present correlations to course satisfaction, I feel this just a correlation by chance
...
...
...

The only true finding is that only faculty feedback is proven to be effective teaching. Why do you think that there are so few studies that have actually concluded something real?

Answers

R: I feel that there are so many variables in teaching, that the change of 1 of these often has little effect. However, I think this is false: we know from Hattie [Hattie, 2012] the effect size of some interventions on grades, with 0.4 standard deviations being the average. I feel detecting a grade shift of 0.4 would be detectable
...
...
...

The only true finding is that only faculty feedback is proven to be effective teaching. Can we use this in our setting directly? If not, how to tweak this to make this work for you? Do you already do this?

Answers

R: I cannot, as I do not receive written work from the learners. I do typically see my (online) learners working from a shared screen, where I can give feedback on how they work. I do do this, as this part of the teaching cycle I use

The paper mentions [Quality Matters, 2018]:

We preface these [i.e. the few implications for practice from research] by noting that organizations such as Quality Matters (2018) offer standards for online course designs that can assure at least minimal quality.

[Quality Matters, 2018] (a propietary method) offers rubrics that one can follow. This paper state that these rubrics 'can assure at least minimal quality'. Do you agree with this statement?

Answers

R: To my own surprise, I do! I enjoy rubrics! As [Quality Matters, 2018] is proprietary, I do not read it. But I do enjoy a the rubric by [McCann et al, 2012], where they classified different teaching styles (see below). Note that 'following rubrics' is a hallmark of 'Satisfactory Teaching'.

Teaching Qualities	Satisfactory Teaching	High-Quality Teaching	Disjointed Teaching
Goals -> Outcome	Tightly-coupled	Coupled	Loosely-coupled
Valued Outcomes	Mastery	Proficiency	Creativity
Decision Making	Rule-bound	Experience	Eclectic
Knowledge	Standardized	Interpreted	Constructed
Intelligences Valued	Intellectual	Social -> emotional -> Intellectual	Emotional
Risks	Minimize	Calculate	Maximize
Valued Materials	Textbooks	Problems	Technique of the day; current events
Tasks-Assigned	Low variety	Reasonable variety	High variety/low variety
Tasks Outcomes	Predictable	Developmental	Unpredictable
Teaching	Routine	Reflective	Intuitive
School Outcomes	Credential/grade	Responsible citizen and productive worker	Meeting student needs

Table 2.1 from [McCann et al, 2012]

Apparently, it is hard to measure what is good/better teaching in online settings. However, this paper also shows gaps in the literature of its timespan. What do you feel is missing?

Answers

R: I wish it was about online workshops, instead of online course websites. I miss the very basics on how to do online teaching: how long can an online monologue be, should the cameras be on/off, should the instructor have his/her camera on, should a live lecture be recorded, how to best make learners work in breakout rooms, etc.

In section '3.3.5. Qualities of Faculty Online Teaching', the paper about 'faculty feedback' is probably hidden. I could not find it. I wondered how big that effect is. Could you find it?

Answers

R: Nope :-)

In the paper, we can read:

Most of the strategies with promising effectiveness in the online environment [that are] Unique to the online environment are user-friendly technology tools, orientation to online instruction, opportunities for synchronous class sessions, and incorporation of social media.

First, what is 'orientation to online instruction'? I have no idea.

Second, will these strategies with promising effectiveness inspire you to do the same?

Answers

R: Not without more information: my tools are fixed (i.e. Zoom), no idea what 'orientation to online instruction' is, online workshops are synchronous and I predict social media has no positive effect

Should we do what is in the paper?

Answers

We identified only faculty feedback as an evidence-based practice in online instruction

Yes, if we think it is important to listen to the literature.

How does this paper make us a better teacher?

Answers

R: it inspires me to combine research with teaching

References¶

[Bell, 2020] Bell, Mike. The fundamentals of teaching: A five-step model to put the research evidence into practice. Routledge, 2020.
[Hattie, 2012] Hattie, John. Visible learning for teachers: Maximizing impact on learning. Routledge, 2012.
[McCann et al, 2012] McCann, Thomas M., Alan C. Jones, and Gail A. Aronoff. Teaching matters most: A school leader’s guide to improving classroom instruction. Corwin Press, 2012.
[Quality Matters, 2018] Quality Matters. (2018). Higher education course design rubric. Retrieved from: https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric
[Uttl et al., 2017] Uttl, Bob, Carmela A. White, and Daniela Wong Gonzalez. "Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related." Studies in Educational Evaluation 54 (2017): 22-42.