Written by Peter Joffe
“If you search for a black cat in a dark room for quite a long time, then, finally, you will find it …even if it does not exist.”
Rephrasing of Confucius.
In this blog, I will give an example of how a dubious interpretation of the facts leads to questionable conclusions. However, if these conclusions are packed into a nice “scientific” wrapper, a theory can be born. Such a theory may be presented to the scientific community and the general public, and even becomes the “mainstream.”
Many scientists dream of being new Einsteins and of inventing a grand theory. I think that is nothing wrong with it. At the end of the day, human ambitions are one of the engines of progress. However, generally, Science moves forward with the small steps, and most of the theories that claimed greatness failed.
Creators and apologists of such theories very often passionately defend their brainchild and tirelessly looking for the facts which could support it.
Unfortunately, this passion often leads to finding “non-existed black cat.” I mean that scientific exploration is biased and all facts are interpreted exclusively in favour of one theory.
Quiet Eye.
One of the examples may be “Quiet Eye” (QE) theory, which was invented by Joan Vickers back in 1992.
In short, the idea is that experts and non-experts in aiming sports (e.g., archery) and tasks (e.g., basketball free throws) differ in how they look at a target. Experts look longer and with a more focused gaze.
Vickers used special device which can track eye pupil movements for training athletes where to look and how long. She reported great improvements.
Since then this nice theory attracted a lot of sports scientists.
Numerous studies in support of QE have been published, and hundreds of theses have been defended.
I can say that, with the help of the popular press, it becomes mainstream now.
I met the QE during study at the University of Exeter. My tutors, Mark Wilson, and Samuel Vine were passionate supporters for this theory and made numerous studies on the topic.
I always was skeptical. In my opinion, “greatness” of QE theory is overestimated, and professor Vickers studies often lack statistical and logical robustness. I believe this is “sincere bias” due to her passionate devotion to QE. Science, however, should be unbiased.
Study at basketball free throws.
One of the most cited papers at the Quiet Eye training is the study of the basketball team (K. Harle, Shawnee & Vickers, Joan2001). It was mentioned in the link above. However, many important details that could change the conclusions were not outlined in this article by the BBC, confirming my opinion that popular journalism is not always good for science.
Let’s have a closer look at this famous paper.
The researchers conducted a QE training in the Canadian Inter-Universities Athletic Association women’s basketball team, and this increased the accuracy of free throws by 22.6% after two competitive seasons. Really impressive and not surprising that this study is considered one of the most reliable evidence supporting the QE theory.
Well, significant and fast gains are quite possible for novices when actually every training brings progress. However, at a higher level when the athletes are close to their limit, every further improvement become more and more difficult; thus every percent counts. Level of Canadian University league is probably mediocre; nevertheless, 23% is really impressive, and if all this improvement was due to Quiet Eye training, it is a really revolutionary method.
If a theory claims to be a significant improvement in the good level sport, check it carefully. Most likely, something is wrong.
Natural variability.
There is a simple rule: check improvements against natural variability of the parameter.
The team was behind two other teams taken for comparison by 7-13% after season 1 despite having one year of Quiet Eye training. Another team improved by 12.6% from season 1 to season 2 despite having no Quiet Eye training.
We can suggest that at least 12% differences between teams and between seasons are quite common.
Additionally, the authors presented a range of accuracy for six players. It was massive: 43.6-100%! So from six players, one scored everything and another just four from ten.
We can conclude that at this level, variability in accuracy between individuals is really high.
Despite improving by 22.6 % from season 1 to season 2, QE team still showed nothing extraordinary and their shooting was at approximately the same level with one of two teams taken for comparison. Thus, even if Quiet Eye training has any effect, it still does not guarantee success.
Dose-Response relationship.
Another trait which can distinguish successful intervention is the dose-effect relationship.
If the training method is effective, people who train longer get better results. Of course, this is not always the case, but to some extent, this rule works.
In this study, six new players (half of the QE training receivers) came to the team between season 1 and season 2, and they obviously received less QE training. However, their accuracy was better!
This absence of dose-effect relationship may be interpreted as indirect evidence that theory does not work.
However, researchers made, in my opinion, strange conclusion that QE training is especially beneficial for newcomers. Hm…
Variability in training effect.
The range of accuracy was 43.8-100% for these six newcomers.
So despite all had the same QE training, the results were hugely different. Thus even if QE training had any effect, not all players benefited from it.
If novel training is not successful for some of the players, this can make its efficacy doubtful.
Procedure.
Studies of Joan Vickers, in my opinion, often lack the robustness of procedure. In this study as well: why did researchers pick up these two teams as controls? Based on their position in the league was the answer. How this connected with free throws? Usually success in a league weakly correlated with the accuracy of free throws (see article). Even more strikingly control teams were chosen post-hock when results of the experimental team were already known. It’s a bias.
Summary of controversial facts:
1. Despite one season of Quiet Eye training team was 7-13% behind two other teams. Is one season not enough?
2. However, new players despite having less Quiet Eye training, overall shoot better than players who had two-seasons training.
3. Significant variability (43.8-100%) in accuracy between players received the same Quiet Eye training.
4. Another team improved accuracy by 12.6% without Quiet Eye training.
5. Choice of controls may be biased.
More simple explanation.
If the research has led to some unusual results, check out a simpler explanation, which may be on the surface, before turning to a complex scientific discussion.
I can suggest a more simple explanation for Vickers et al. findings. Although I am not a specialist in basketball, so players and coaches, please correct me if I am wrong.
Significant individual differences in accuracy, at least at this level, make overall team results hugely dependent on who is shooting. If the good scorers made the most of the free throws, overall team result is excellent. If not, accuracy goes down. In football, the team trusts penalties to the best penalty takers. However, in basketball, free throw must be made by the player who was fouled. Thus if good throwers were fouled more, the team has higher chances to show good free throw percentage. However, who was fouled depends on many factors and at this level may vary significantly from season to season.
Possibly in season 2 good shooters were fouled more and consequently made more free throws than in season 1.
Can QE training be beneficial?
Yet still, a lot of papers reported a beneficial effect of QE training.
Can this be true?
Yes, it can, and I am not against Quiet Eye training. The question is: why and how it may help?
Perhaps this is not because crucially important is where to look and how long. It is not that athlete collected some valuable visual information during a longer look as QE theory states. What can basketball player additionally know staring longer at the hoop? Nothing.
There may be another explanation for QE effect.
Definitely, free throws, penalties, archery, and other aiming activities depend on athletes mental state.
QE training can facilitate attainments of needed mental state.
This is because of its procedure itself. Concentration on gaze routine and fluidity of movements can help the athlete to avoid unnecessary thoughts and reduce pressure. This is not unique for gaze. For example, the same positive effect may be found for exercises at breathing control.
Adepts of Eastern meditation practices new this long ago.
You can try to train mental state through the breathing, self-talk, or pre-shooting routine. QE may be one of the methods you can try; why not? Nothing “scientifically great ” however.
My mentors finally admitted this, as well. After investing in QE theory for many years they said that “the positive QE training results might be telling us more about generic psychological and physiological changes that occur via the taught pre-performance routine, rather than any specific role for QE itself” (Wilson, Mark & Wood, Greg & Vine, Samuel. 2016).
Epilogue.
I remember in 2011 group of scientists made a “discovery” that neutrino can exceed the speed of light. This makes Einstein theory incorrect. It was announced to the scientific community and not without help from popular press became a world sensation.
The advice from scientists to these revolutionaries was “to check the wires” before making such announcements. And indeed, wires were not connected properly. Neutrino travels with the speed of light. Einstein theory still stands.
Maybe it is time for “quiet eyers” to check the wires as well?
References.
K. Harle, Shawnee & Vickers, Joan. (2001). Training Quiet Eye Improves Accuracy in the Basketball Free Throw. The Sport Psychologist. 15. 289-305. 10.1123/tsp.15.3.289.
Wilson, Mark & Wood, Greg & Vine, Samuel. (2016). Say it quietly, but we still do not know how Quiet Eye training works – comment on Vickers. Current Issues in Sport Science. 2016. 10.15203/CISS_2016.117.