Conclusion

RQ1: What is the history of the ELIXIR evaluation questions?

The ELIXIR student evaluation is based on two evaluations. Neither evaluation describes (1) how its questions were developed, (2) by which criteria the best questions were selected.

From these two questionnaires, some (but not all) questions were selected to be put in the ELIXIR evaluation. Additionally, ELIXIR added new questions, regarding student satisfactions. The criteria for neither not copying questions for the earlier questionnaires, not for adding adding questions were not written down.

As none of these evaluations describe the process on how questions were developed, it comes as no surprise that neither evaluation refers to evidence-based best practices in the literature.

RQ2: Can the current ELIXIR evaluation questions be criticized?

The mandatory ELIXIR evaluation questions with the goal of evaluating course quality can be criticized, with

• 1 question that can be answered before taking the course (question 5)
• 1 question that requires predicting the future (question 6)
• 1 question having some research findings hinting it may be a valid way to assess course quality (question 7)
• 1 question which has no relation to course quality according to a meta-analysis (question 8)
• 1 question that gives no information, except the learner being willing to be contacted in the future (question 9)

RQ3: Which ELIXIR evaluation questions are concluded from a fully transparent process?

TODO

RQ4: How different are the newly suggested questions from the current ones?

TODO