香京julia种子在线播放

There is ample literature reporting on the effects of feedback in motor learning tasks. From the first attempts to experimentally test Thorndike's theory of learning (Trowbridge and Cason, 1932), to more recent work (Newell, 1974; Salmoni et al., 1984; Schmidt et al., 1989; Winstein, 1991), studies consistently show how feedback during motor learning increases the rate of improvement over trials. However, the effects on retention and long-term learning are less clear. Approaches providing infrequent feedback have shown improvements in retention phases usually performed 24 h after the experiment (for a review of feedback studies see Winstein, 1991). Still, it is important to consider that this kind of research has focused on studying the effect of feedback in controlled environments where the effect of intrinsic feedback (e.g., visual, auditory, proprioceptive) pertaining to the outcome movement is minimized. This kind of experimental design may imitate the process of learning in a person with sensory deficits who is unable to use intrinsic feedback and depends on the extrinsic feedback (i.e., related to the result of the action) given by the experimenter. On the other hand, motivation is very important in learning (Elwell and Grindley, 1938). Some researchers have attempted to control the motivational effects of feedback in experimental setups where subjects were asked to improve their performance or were given explicit goals (Locke, 1966; Locke and Bryan, 1966) in order to find a significant goal effect. Although those results may relate with the learning of motor skills (e.g., in sports, music), extrinsic feedback could play a different role depending on the task performed, so more specific research is needed in order to understand better the impact of feedback technologies in music students.

Mastering the violin and other bowed-string instruments require special considerations compared with other musical instruments. As opposed to the piano, for instance, pitch control in the violin is continuous and correct intonation is an important issue. In addition, the process of good sound generation in the violin is a notorious complex task which requires precise spatiotemporal control of bowing gestures (Schoonderwaldt and Demoucron, 2009). Acquiring correct bowing motor skills require many hours of practice in which aural feedback is crucial for students to adjust their motor gestures to generate good sound. According to Konczak and Jaeger (2009) novice players need approximately 700 practice hours to achieve bowing skills comparable to those of experts. Moreover, string players have the highest risk of playing-related musculoskeletal injuries/disorders (PRMDs) with the neck and shoulder being the main body parts affected (Middlestadt and Fishbein, 1989).

A recent survey on Australian higher education music students showed how the use of Youtube and self-recording has become common practice among them (Zhukov, 2015). Youtube offers videos of professional musicians performing music repertoire pieces as a model for students while self-recording has become an important tool for self-evaluation. Previous research (Kepner, 1986; Bundy, 1987) found that high school instrumentalists were more able to identify musical errors when hearing tape recordings of their own performances than when actually performing the pieces. In particular, Bundy (1987) explains the obtained results by a sensory blocking theory which hypothesizes that when musicians are concerned with monitoring a big number of sensory aspects involved in performance (like sight-reading or finger movements) the sense of hearing, which is perceived to be of lesser importance, is blocked. However, recent research (Hewitt, 2001) studied the effects of listening to a model (i.e., an expert reference performance), listening to oneself on audiotape, and self-evaluation on junior high school instrumentalists, concluding that there is a significant interaction effect for modeling and self-evaluation. However, self-evaluation (which in the case of the study consisted on the Woodwind Brass Solo Evaluation Form Saunders and Holahan, 1997) or self-recording on their own were not found to be effective strategies for improving music performances. Although self-recording may be important, in absence of a teacher it requires the student to be his/her own judge which may be problematic. The superiority of highly trained musicians encoding spectral and temporal features of music-sound compared with non-musicians has been found in a large number of neuroscientific studies (Besson et al., 1994; Koelsch et al., 1999; Pantev et al., 2001; Tervaniemi et al., 2005; Hutchins and Peretz, 2012). For example, Koelsch et al. (1999) demonstrated, using electrophysiological and behavioral data, that highly trained violin players are able to detect automatically undetectable pitch differences for nonmusicians.

Current music technologies provide us with objective measures of student improvement in specific music tasks. Thus, such technologies can allow us to monitor the learning process of music students in order to provide better and personalized learning strategies. In addition, objective measures about music students' performance may serve as additional information which could complement the verbal feedback given by the teacher. In the past, the role of feedback in music learning has been addressed mainly to study the effects of real-time visual feedback (RTVF) in singing. Welch et al. (1989) studied the effect of a feedback system called SINGAD (Singing Assessment and Development) in 32 primary school children aged 7 years. The system provided a real-time F0 trace plotted against time together with the target notes displayed in order to guide time and pitch accuracy. The study reported improved pitch accuracy by using the system. Previous research has studied the effect of using different kinds of interfaces and different kinds of feedback in singing voice (Thorpe, 2002; Welch et al., 2004; Wilson et al., 2008; Leong and Cheng, 2014), trombone (Schlegel and Gregory Springer, 2018), piano (Hamond, 2017), and violin (Wang et al., 2012). Although there are differences in the way RTVF may improve performance, most of the previous studies reported beneficial effects of RTVF in learning. An extensive review on feedback and technology applied to music learning can be found in Hamond (2017). The same author also investigated the nature and application of combined visual-auditory feedback generated by technology systems in higher education piano learning and teaching contexts. As suggested by self-reports collected from music students, the feedback provided could increase conscious-awareness of their own performance. As related by one of those students: “Sometimes you know in your mind what you want to do, […] but sometimes you do not realize exactly what you're doing in practice[…]. So, when you hear, you can clearly see what you are doing and what you're not” (Hamond, 2017, p. 278).

Regarding violin learning, special efforts have been done to offer different kinds of feedback, not only on the produced pitch but also on timbre, good posture, and bowing technique. The i-Maestro project (Ng and Nesi, 2008) was one of the first steps in that direction offering tools based on gesture analysis and audio processing. More recently the TELMI project has developed tools for providing feedback on timbre quality, pitch and timing accuracy, posture and bowing techniques, and musical expression (Ortega et al., 2017; Dalmazzo et al., 2018; Giraldo et al., 2018; Zacharias et al., 2018). Optical motion capture combined with sensors has also been used to extract bowing parameters from violin performance (Schoonderwaldt and Demoucron, 2009; Deutsch, 2011) allowing to study and compare the motor patterns of professional and student violinists. Tracking violin performance using low-cost methods has also been investigated by Perez Carrillo and Wanderley (2012) through the sole use of audio signal and a system trained on empirical data previously collected with a highly accurate sensing system. Pardue et al. (2015) also explored low-cost methods using a resistive fingerboard and four optical reflectance sensors placed on the bow stick. Some attempts have been done in order to evaluate motion capture techniques to teach violin skills. For example, Van Der Linden et al. (2011) used a wearable system to teach good posture and bowing technique to novice violin students and found a larger improvement when compared with a control group of subjects who received the same number of training sessions using conventional teaching techniques. One possible limitation of the previously mentioned study is that the quality of generated sound is not taken into account, while in violin learning the production of a good sound is one of the main reasons for learning a correct bowing technique.

The work of Romaní et al. (2015) aimed to identify audio descriptors, extracted from the recordings of professional musicians while playing single notes, maximally correlated with their own subjective opinions about the quality of the produced sound. Some of the features that showed higher correlations were those characterizing pitch stability and dynamic stability. This research led to the implementation of Cortosia (Korg, 2018) an app owned by the Korg company, which aims to provide students with visual feedback about the quality of their produced sound. More recently, Giraldo et al. (2018) investigated the application of machine learning techniques to obtain sound quality model and implemented a real-time feedback system for enhancing violin learning. However, no studies have been done until now to evaluate the pedagogical effectiveness of such systems.

One could be tempted to offer simultaneous real-time feedback in violin learning environments (e.g., violin-bow orientation, bowing trajectory, and timbre quality). However, a common concern found in user studies offering several simultaneous feedback is that participants usually have difficulties dealing with them (Van Der Linden et al., 2011; Johnson et al., 2012; Johnson, 2014). Delivering the different feedback separately at different times and as requested by the user could be one possibility to resolve that problem, as has been the approach in the TELMI project. Another common concern is the potential dependency that feedback systems could create on students.

Recent research (Brandmeyer et al., 2011) has evaluated the effects of RTVF on expressive percussion performance interpreting their results using the Cognitive Load Theory (CLT) (Paas et al., 2003). In their work, they differentiate between three different kinds of cognitive load: intrinsic, extraneous and germane. Intrinsic cognitive load is associated with the difficulty of the particular task whereas extraneous cognitive load relates to the manner in which information is received. On the other hand, germane cognitive load relates to the mental resources involved in learning in general, independently of the task. Brandmeyer et al. (2011) found empirically that too many visual elements can create a high extraneous cognitive load in participants, dividing their attention and leading to poorer learning outcomes. However, apart from behavioral measures, no other measures were used to evaluate the amount of cognitive load participants were experiencing. Physiological measures can provide objective measures of the mental work a person is experiencing while learning. Recently, the neural activity associated with learning tasks has been investigated by the neuroscientific community using both functional neuroimaging and electroencephalography (EEG) techniques. In particular, EEG is the most common technique used to study cognitive load from brain activity and one of the most feasible among other electro-physiological measures (Miller, 2001).

Event-related (de)synchronization (ERS/ERD) is a well-established measure for the quantification of changes in different frequency bands of the EEG signal. It reflects the decrease (desynchronization) or increase (synchronization) in a band power during a test (time period where the subject is performing a specific task which demands cognitive load) compared with a reference baseline (time period without any task demands). This is usually done for each electrode. A positive ERD/ERS value means a decrease in a band power (desynchronization, ERD) while a negative value indicates an increase in band power (ERS). It has been reported repeatedly for several researchers that alpha and theta band activity (8–13 Hz and 4–7 Hz, respectively) is very sensitive to task difficulty or cognitive load in a wide variety of task demands (Klimesch, 1999; Gevins and Smith, 2003; Neubauer et al., 2006). Generally, as cognitive load increases, frontal midline theta band increases, and posterior alpha band decreases. Larger alpha band ERD has been associated with highly intelligent subjects and good performance (Jaušovec and Jaušovec, 2004). Explanations of this phenomenon are usually delegated to the neural efficiency hypothesis which assumes that high alpha band power reflects cortical inhibition. On the other hand, theta has been investigated for its implications in memory performance (Raghavachari et al., 2006) showing strong increases in the frontal area during the encoding and retention period (Maurer et al., 2015). Thus, an alpha band power decrease at posterior sites (larger alpha band ERD) and a frontal theta increase represent a general index for cognitive demands. Some research also highlights the importance of gamma band waves (30–100 Hz) which its enhancement is observed within a task-specific spatial distribution (Fitzgibbon et al., 2004) and seems to be correlated with cognitive load in humans (Howard et al., 2003). The temporal binding model gives gamma band the responsible role of integrating (binding) information processed in distributed cortical areas. Task complexity demands more cognitive resources, more binding and thus, gamma band power enhancement. Interestingly, some research has found that subjects with musical training show enhanced induced gamma band activity (Shahin et al., 2008; Trainor et al., 2009) suggesting it reflects a superior binding of acoustical features (e.g., pitch, timbre, harmony) and processes also thought to be enhanced by music training, e.g., anticipation, expectation and attention (Bhattacharya et al., 2001; Sokolov et al., 2004; Gurtubay et al., 2006).

The viability of the use of EEG to test the effectiveness of learning materials designs has been provided by some studies (Antonenko and Niederhauser, 2010; Antonenko et al., 2010). Thanks to the measure of participants' cognitive load it is possible to assess which learning strategy seems to work better in concrete situations. On the other hand, some studies have also started to investigate the potential of real-time monitoring of mental workload to improve human performance. For instance, Kohlmorgen et al. (2007) describes a system to reduce distractions while driving by monitoring mental workload.

EEG has also been used to improve music performance through the use of an increasingly popular technique called neurofeedback. It consists of learning, through visual or auditory feedback, how to modify voluntarily your own mental activity. Several studies have reported improvements in the music performance of those musicians who received a neurofeedback session on the theta /alpha protocol (i.e., learning how to maximize the theta to alpha ratio) before a performance, compared with other groups who received different kinds of relaxing techniques like the Alexander technique or different neurofeedback protocols (Bazanova et al., 2009; Gruzelier, 2009). Similar results have also been found for dancers (Raymond et al., 2005). According to the authors, the production of theta waves with eyes closed is related to the hypnogogic process which at the same time is associated with an improvement of the creative process and well-being of users.

Other studies have tried the use of theta-EEG and EMG biofeedback with violinists while they perform, with positive results (Silvana et al., 2008). The pre-recorded sound of applauses as feedback gave the musician the opportunity to recognize which is the adequate mental and muscular state needed for optimum performance. The reason to train theta during the performance was that some investigations have found enhanced theta activity in highly-skilled professional musicians (Klimesch et al., 1997; Bazanova and Aftanas, 2006). According to the neural efficiency hypothesis experts should show lower brain activation (which means higher theta power and more efficient networks), and thus, training students to learn how to use their brain more efficiently could lead to an enhancement of their performance.

The relationship between EEG power changes and proficiency have also been reported in sports activities such as rifle marksmanship (Haufler et al., 2000; Kerick et al., 2004), archery (Salazar et al., 1990; Landers et al., 1994) and golf (Crews and Landers, 1993; Babiloni et al., 2008). This research shows how the most predictive data of expertise is recorded before the skilled movements occur, in what is called the “pre-shot routine.” For instance, it has been shown that the magnitude of the increase in theta power before the shot is correlated with the accuracy of the shot. Berka et al. (2010) tracked the learning process of beginners in rifle marksmanship while firing a total of 40 shots and correlated the accuracy of the results with the EEG power activity, finding increases in theta and high theta Bands (6–7 Hz) just as experts showed during all their trials. They also compared the results of the learning group with another one which, additionally, received a neurofeedback training based on the same frequency bands showing how the neurofeedback group obtained significantly better results. Similar results were also found by Gentili et al. (2008) where subjects had to learn and interact with new tools. They found increases in alpha and theta band power in the frontal and temporal lobes during movement planning (i.e., just before the movement, like in the pre-shot routine).

However, in a recent study (Gutierrez and Ramírez-Moreno, 2016) changes in brain activity associated with the progression of the learning experience were estimated with different results. They monitored the process of learning to typewrite using the Colemark keyboard layout, which is an alternative to the QWERTY layout, finding a decreasing trend of the beta and gamma bands. They interpreted beta band decrease as a result of long-duration repetitive hand movements, similar to results found by as Niemann et al. (1991) and Erbil and Ungan (2007), and explained the gamma band decrease as a consequence of the temporal binding model previously mentioned, which associates gamma band activity with coupling perception and learning, as reported by Gruber and Müller (2005).

The aim of this work is to contribute to the understanding of the effects of feedback in music learning from an electrophysiological point of view. For this purpose, we have evaluated the effectiveness of using a sound quality visual feedback system (SQVFS) to improve the quality of sound produced by of novice violin players while their EEG activity and the violin sound they produced was recorded. These recorded data provides non-invasive biomarkers of motor learning in a musical task. Participants (with no previous experience with violin or any other bowed string instrument) were asked to produce a stable and sustained violin sound on an open string (i.e., the second string in the violin). The choice of using an open string was to allow participants to exclusively concentrate their attention to control the bow movement. This task requires to control and change the pressure of the bow along the whole movement due to the fact that bow pressure requires to be heavier at the frog and lighter at the tip. If the pressure of the bow is not constant along the movement both pitch and energy of the produced tone could change. For that reason, we hypothesized that the use of dynamic stability and pitch stability audio descriptors, as Romaní et al. (2015) did, to measure sound quality among trials would allow us to track improvement through the session. We also offered the numerical result of the descriptors as feedback to the participants (i.e., the SQVFS).

Participants were divided into two groups. Both of them had access to learning materials and reference videos during the experiment, but in addition one of the groups received offline feedback about the quality of their performance given by the SQVFS. The quality of the produced sound, as well as the EEG activity of each participant, was recorded during 4 blocks of 5 trials each (20 trials in total). An additional group of violin experts was considered in the experiment for comparative purposes. Data recollected in this study is publicly available in Zenodo (Casares and Ramírez, 2018) and the code to analyze it in Github (Blanco, 2018).

The study was carried out in the recording studio located in the Information and Communication Technologies Engineering (ETIC) department of the Universitat Pompeu Fabra, Barcelona and included the participation of twenty-five right-handed subjects. Participants conceded their written consent and procedures were approved by the Conservatoires UK Research Ethics committee on 04/04/2017, following the guidelines of the British Psychological Society. Participants provided information about their musical skills, main instrument and years of music training. Those with extensive experience in violin playing were included in the expert group [EG;6 male, 1 female; mean age: 35.2 (9.01); mean years studying violin: 7.6 (2.19)]. Participants with no violin (or viola, double-bass or cello) experience were included in the beginner's group. This last group, was randomly divided in two groups: the first group [BF; 6 male, 3 female; mean age: 27.57 (4.46); all of them were musicians with several years of experience, mean: 9 (5.07)] practiced with instructional videos and offline feedback from the SQVFS reflecting the quality of their produced sound, while the second group [BNF; 8 male, 1 female; mean age: 27.2 (2.28)] practiced with the instructional videos only. All participants were musicians with several years of experience, mean: 10.8 (4.65).

EEG data were acquired using the Emotiv EPOC EEG device. The Emotiv EPOC consists of 16 wet saline electrodes, located at the positions AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4 according to the international 10-20 system (see Figure 1). The two remaining electrodes located at P3 and P4 are used as reference. The data acquired were digitized using the embedded 16-bit ADC with 128 Hz sampling frequency per channel and sent to the computer via Bluetooth. The Emotiv Control Panel software was used to monitor visually the impedance of the electrodes contact to the scalp. The data were recorded using the OpenViBE platform (Renard et al., 2010) and later processed in EEGLAB (Delorme and Makeig, 2004) under the Matlab environment (MATLAB, 2010).

A Zoom H4N handy recorder was used to record the audio of each trial which was processed in Matlab using the “Yin pitch estimation toolbox” (Llimona, 2015) in order to extract audio features for assessing sound quality and provide feedback to participants. Yin is a widely used algorithm to estimate fundamental frequency both in speech and music (De Cheveigne and Kawahara, 2002).

Visual feedback provided to the BF group consisted of graphs generated in Matlab showing the sound quality score in the y-axis and the trial number in the x-axis. Feedback was intended to allow participants to monitor their progress and compare their performance to that of an expert participant who previously did the experiment (also plotted in the feedback screen) (see Figure 2).

Instructional videos about basic violin playing techniques, e.g., stance, violin position, bow position, and grip, were used to provide participants with basic information. The videos were collected from the web (Sassmannshaus, 2018) (see Figure 3). In addition, we recorded a reference video of the requested task performed by a professional violin player. The produced video was shown to all participants to explain the task to be performed. The video can be found in Zenodo (Casares and Ramírez, 2018).

EEG acquisition and audio processing were performed on different laptops (PC1 and PC2, respectively). To synchronize audio and EEG data PC2 sent markers to OpenVibe in PC1 through OSC everytime a new trial began and ended. The experimenter controlled the display of instructional videos and the reference expert video for both BNF and BF groups and sound quality visual feedback for the BF group (see Figure 4).

Due to the nature of the experiment, it was not possible to conduct a double-blind study. In order to avoid unconscious bias during the instructions given to participants, both beginner groups (i.e., BNF and BF) watched the same set of instructional videos on violin and bow position and stance with a total duration of 10 min (Sassmannshaus, 2018). Participants watch the videos while the EEG device was positioned on their heads. Once setup of the EEG device and the videos were finished, participants proceeded to perform the violin bowing exercise which consisted in the alternation of eight up and down bowing movements using the full length of the bow with the goal of producing a sound in the A open string. Participants were asked to produce a stable and sustained sound at the same tempo of the reference video. Participants were also asked to minimize blinking and facial movements during the exercise to avoid artifacts in the EEG signal.

The blocks of trials were named as follows: early block (trials from 6 to 10), middle block (trials from 11 to 15), and late block (trials from 15 to 20). In Figure 5 we can see all the steps that were involved in the processing of audio and EEG data. First, audio and EEG data were processed separately to extract meaningful descriptors. Posterior EEG and audio analysis allows us to study changes over time and between groups. Finally, the correlation analysis allows us to measure the correlation between sound quality features and EEG features. We also did a behavioral analysis to study participants' learning patterns, i.e., number of times they consulted the learning materials. The total duration of the experiment was approximately 45 min. The first block of trials, where both groups of beginners did not have the option to rewatch instructional videos or offline feedback from the SQVFS, was used as a baseline to compute the amount of change in both sound quality and EEG waves around the rest of blocks. From the early block on, BF and BNF had the option to rewatch both instructional and/or reference expert videos as many times as they wanted for the rest the trials. In addition, the BF group had the opportunity to receive offline feedback from the SQVFS visualizing the dynamic stability, pitch stability and aperiodicity scores of their performance for each trial. The number of times a participant requested the learning materials were also recorded.

The results of IGAE ranked the Sound instability descriptor which included aperiodicity as the most important one to differentiate between beginners and experts with a value of 0.668. It was followed by pitch instability with 0.599; dynamic instability with 0.577; Sound instability without aperiodicity with 0.549 and aperiodicity with 0.514. A Shapiro-Wilk test for normality was performed on the data showing significant results leading us to use non-parametric statistical tests. A Wilcoxon rank-sum test was performed for each audio descriptor comparing experts and beginners showing significant results (p < 0.00001 for all the descriptors). Beginners showed higher values than experts, i.e., beginners produced more unstable sound (see Figure 6).

Percentage changes of Sound instability were computed for each block and for each group using the first block as a baseline. After adjusting the p-value for three tests using the Bonferroni method, significant changes through blocks were found for both beginners groups but not for the experts (Wilcoxon sign-rank test, BNF: p = 0.00001; BF: p = 0.000005; EG: p = 0.1396). In order to detect differences between the blocks, three more Friedman's tests were performed, one for each group. Only the BF group showed significant results (p = 0.0011). On average, the BF group showed a higher amount of change between the three blocks compared with the BNF group together with a higher variability especially during the Middle and Late period (see Figure 7). At the end of the session and during the Last period (trials 16–20), the BF group showed, on average, 200% percent more than the BNF group on the scores of Sound instability together with a standard deviation 4.7 times higher., although This difference was not found to be significant (Wilcoxon rank-sum test: p = 0.2973).

The results of IGAE ranked beta and gamma band power at frontal sites as the most important features to discriminate between beginners and experts with a value of 0.01931 and 0.01905, respectively, followed by gamma band power at posterior sites with 0.00828; right frontal beta with 0.00811; right frontal gamma with 0.00562; left frontal gamma with 0.00424; left posterior gamma and beta with 0.00297 and 0.00269, respectively.

A Shapiro-Wilk test for normality was performed in the distributions of each group showing significant results leading us again to use non-parametric statistical tests. Eight Wilcoxon rank-sum tests were performed for each feature comparing experts and beginners adjusting the p-value with Bonferroni correction. Only beta and gamma band power at frontal sites showed significant results (p < 0.00001 both). Figure 8 shows the differences in beta and gamma band power between beginners and experts. On average, Beginners showed an amount of 190% more of power than experts in the beta band and a 16% more in the gamma band.

ERD/ERS was computed for the rest of the blocks using the first block as a baseline. A Wilcoxon sign-rank test was performed for each cluster and frequency band to determine those sensor clusters and frequency bands where variations occurred through the rest of blocks. The p-value threshold chosen after the Bonferroni correction for 156 tests was p < 0.00032. Results showed an overall desynchronization of gamma band across the scalp for all the experimental groups and all the blocks with certain differences between them. Significant desynchronizations were obtained for gamma band at frontal electrodes in both groups of beginners (see Figure 9) but not for experts (p < 0.00032). Significant desynchronizations for gamma band were also found at right parietal and left temporal in all the groups including experts. Only the expert group showed significant changes at the right temporal cluster reflecting a synchronization of beta band.

Significant results were found when comparing the amount of desynchronization at frontal sites of the BNF group with the BF (p = 0.057) and with the EG(p = 0.025). Beta synchronization found at the right temporal cluster in experts showed significant results when comparing it with the BNF group (p = 0.000041) and with the BF group (p = 0.0149). In order to detect differences between the blocks, three more Friedman's tests were performed, one for each group. No significant changes were found in the amount of ERD/ERS between blocks.

Four Pearson's correlations were performed in total. Four of them between each one of the four frequency bands filtered (theta, alpha, beta, gamma) from frontal electrodes and Sound instability across the whole session. After adjusting the p-value for four statistical tests results showed only one statistically significant linear correlation at gamma band (R² = 0.70, p = 0.00001). In Figure 10 Sound instability results averaged for each trial for both beginners and expert groups can be seen in comparison with the averaged gamma band power at frontal electrodes obtained for each trial.

The number of times each participant requested each learning material (instructional videos, reference video or their score evaluated with audio descriptors) was recorded and compared between the two different beginners groups (BF and BNF). A Shapiro-Wilk test for normality was performed in the distributions of the number of times each group requested a learning material showing significant results for the BNF group. Two statistical tests were performed in total adjusting the p-value with Bonferroni correction (p = 0.016). Two Mann-Whitney U-test were performed to find differences between both beginners group (BF + BNF) in each one of the distributions. No significant differences were found, although results show some different tendencies in terms of the number of reference video requests (p = 0.0203). On average, the BF group requested the reference video 25.8% more times than the BNF group. In Figure 11, we can see the differences found in each distribution for each group. The BF group also had the possibility to request the audio-based automatic evaluation of their performance produced by the system. A paired sampled t-test was performed between the number of times the BF group requested the reference video with the number of times they requested the audio evaluation. No significant differences were found (p = 0.37).

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.