E4D-Compare Software for Dental Analysis

E4D-Compare Software for Dental Analysis ABSTRACT Aim: The E4D-Compare software is an innovative tool that provides immediate feedback to students’ projects and competencies. It should provide consistent scores even when different scanners are used which may have inherent subtle differences in calibration. This study aimed to evaluate potential discrepancies in evaluation using the E4D Compare software based on four different NEVO scanners in dental anatomy projects. Additionally, correlation between digital and visual scores was evaluated. Methods: Thirty-five projects of maxillary left central incisors were evaluated. Among these, thirty wax-ups were performed by four operators and five consisted of standard dentoform teeth. Five scores were obtained for each project: one from an instructor that visually graded the project and from four different NEVO scanners. A faculty involved in teaching the dental anatomy course blindly scored the thirty-five projects. One operator scanned all projects to four NEVO scanners (D4D Technologies, Richardson, TX, USA). The images were aligned to the gold standard, and tolerance set at 0.3 mm to generate a score. The score reflected percentage match between the project and the gold standard. One-way ANOVA with repeated measures was used to determine whether there was a significant difference in scores among the four NEVO scanners. Paired-sample t-test was used to detect any difference between visual scores and the average scores of the four NEVO scanners. Pearson’s correlation t est was used to assess the relationship between visual and average scores of NEVO scanners. Results: There was no significant difference in scores among four different NEVO scanners (F(3, 102)=2.27, p=0.0852 one-way ANOVA with repeated measures). Moreover, the data provided strong evidence that a significant difference existed between visual and digital scores (p=0.0217; a paired-sample t-test). Mean visual scores were significantly lower than digital scores (72.4 vs. 75.1). Pearson’s correlation coefficient of 0.85 indicated a strong correlation between visual and digital scores (p Conclusion: The E4D Compare software provides consistent scores even when different scanners are used and correlates well with visual scores. KEYWORDS Computer-assisted learning/computer-assisted simulation (CAL/CAS), Dental anatomy, E4D Compare software CLINICAL SIGNIFICANCE The use of innovative digital assessment tools in dental education is promising with the E4D Compare software correlating well with visual scores and providing consistent scores even when different scanners are used SHORT RUNNING TITLE Innovative Digital Assessment Tool INTRODUCTION Dental anatomy is a course given to the first year dental students at the University of Iowa College of Dentistry Dental Clinics (UICOD) with the purpose of providing students with basic knowledge on dental anatomy and dental terminology to serve as a solid foundation for the following courses throughout their school years. It is designed to develop the psychomotor skills necessary to proficiently reproduce tooth contours in wax to be able to apply these skills in their future clinical practice. Students are challenged with four waxing projects during the course. Evaluation of these projects and providing proper feedback for self-assessment to establish life-long learning are still major issues that need to be addressed in the dental anatomy course. Traditionally faculty members involved in the course have used a pre-established checklist (Table 1) to provide ongoing feedback during the course, and also to determine the grade by adding points related to specific criteria. This visua l evaluation is time consuming and limited by grader fatigue, subjectivity, and inconsistency.1 The inconsistency among different faculty members is also the most frequently noted concern from student feedback after the course that may negatively affect students’ learning process and performance.2 Computer assisted learning and grading tools have been introduced recently and have shown great promise in providing not only objectivity to the grading system, but also reducing the workload of faculty.3 A digital device using blue laser technology (Nevo scanner, D4D Technologies, Richardson, TX, USA) offers high scanning image quality at a fast video-rate speed and ease of use. A laptop computer connected to the E4D Design Center provides rapid processing of the scans. Through this new technology, students can generate a high quality 3-D virtual model of their project and receive immediate feedback on discrepancies by comparing it to the gold standard determined by the course director.4 An ideal ratio in a dental classroom setting is to accommodate one scanner for every ten students.3 Therefore multiple scanners are usually used to facilitate proper teaching and learning experience for all students. It is expected that consistent scores will be obtained even when different scanners are used which may have inherent subtle differences in calibration. However, with the recent introduction of this digital technology, there is a lack of information on how accurately different scanners relate to each other, as well as lack of evidence on how well visual scores by faculty correlate to scores generated by the digital assessment technique. This study aimed to explore potential discrepancies in evaluation in dental anatomy projects using the E4D Compare software based on four different NEVO scanners. Additionally, correlation between digital and visual scores was evaluated. The hypotheses tested were that: first, the E4D Compare software will provide consistent grades even wh en different scanners are used, and second, there will be no correlation between visual and digital scores. MATERIALS AND METHODS The study was determined to be a non-human subject research project by the Institutional Review Board of the University of Iowa. Dental Anatomy Wax-Up of Tooth #9: The maxillary left central incisor (#9) was used for the evaluation of this study. Four operators (Junior dental students) independently waxed up a total of thirty projects based on the wax-addition technique described in the UICOD Dental Anatomy Manual. The maximum time allowed for the wax-up was 3 hours, which is also the time allotted to students during their waxing competency in the Dental Anatomy course. Additionally five dentoform teeth of tooth #9 (Kilgore model, Nissin Dental Corp., Tokyo, Japan) were included for evaluation. The original Kilgore typodont tooth also served as the gold standard for evaluation purposes. Five scores were obtained for each project: one from an instructor and four from NEVO scanners 1 to 4 Visual Evaluation of Wax-ups and Typodont Teeth: A faculty member involved in teaching the dental anatomy course was blinded and scored all thirty-five projects based on the check-list of the UICOD Dental Anatomy Course (Table 1). The check-list consisted of three categories including: contact areas, embrasure and line angles; facial and lingual contour; incisal edge and surface finish, adding up to a total of 23 criteria. Based on the addition of correct criteria the final score was converted to a percentage score for evaluation purposes. Scanning and Digital Evaluation: One operator started the set-up of the files and scanned all projects to four NEVO scanners (NEVO 1 to 4, D4D Technologies, Richardson, TX, USA). The images were checked for proper data density and then aligned to the gold standard, with the tolerance level set at 0.3 mm to generate a general score (Figs 1-6). This score reflected the percentage match between the project and the gold standard as calculated by the software. Data Analysis: One-way ANOVA with repeated measures was used to determine any significant differences in scores among four NEVO scanners. Paired-sample t-test was conducted to detect any difference between visual grades and the average grades of four NEVO scanners. Pearson’s correlation test was used to assess the relationship between visual grades and average grades of NEVO scanners. All tests utilized a 0.05 level of significance, and statistical analyses were performed using the statistical package SASÂ ® System version 9.3 (SAS Institute Inc., Cary, NC, USA). RESULTS Thirty-five teeth were used for this study. Five scores were obtained from each tooth with five different methods; one from an instructor and four from the NEVO scanners. Table 2 presents a summary of descriptive statistics on scores by the five evaluation methods. (1) Detecting the difference in scores among the four NEVO scanners The data were analyzed using the one-way ANOVA with repeated measures. This analysis revealed no significant difference in scores among the four scanners (p=0.0852). (2) Detecting the difference in scores between visual and NEVO evaluation methods There was a significant difference in scores between visual and NEVO evaluation methods (p=0.0217; a paired-sample t-test). The data revealed that the mean scores from visual evaluation method were significantly lower than those obtained from the NEVO evaluation method (mean score: 72.4 vs. 75.1) (Table 2). Note that the average scores of four scanners (NEVO_AVE) were used for the comparisons. (3) Assessing the correlation between visual and NEVO grading scores Based on both Pearson’s correlation test, the data provided strong evidence that there was a significant correlation between visual and NEVO evaluation scores (p DISCUSSION Digitization through innovative technology has become an integral part of contemporary dental practice. This trend is manifested by the increased use of computer-aided design and computer-aided manufacturing (CAD/CAM) technology that promotes the same day preparation and same day delivery concept. Despite intense initial debate, the adoption of restorations fabricated using this approach has profoundly benefited both clinical dentistry and dental laboratories, and epitomizes one of the earliest successes of digital dentistry.5,6 Given the success of CAD/CAM approaches in the clinical setting, many dental institutions are currently striving to incorporate computer-assisted learning and computer-assisted simulation (CAL/CAS) systems into their curriculum. This is to teach dental students to be abreast with the latest technology and keep them prepared for the dynamic changing environment they may face in their future practice. At the UICOD, active incorporation of new technology has bee n perceived to be of high priority. A Technology Committee has been established to plan for active implementation of digital dentistry into the curriculum to provide appropriate vertical and horizontal integration of these concepts as students progress from freshmen to senior years. As part of the planning process, importance has been placed on accumulating evidence on the validity and benefits of using digital technology in the pre-clinic simulation clinic. The study reported here attempted to address these issues, and was designed to evaluate the validity of using multiple scanners and the correlation between visual evaluation performed by faculty and the evaluation by the digital assessment tool. Based on the results, our first hypothesis was accepted: the E4D Compare software provided consistent scores even when different scanners were used. This reflects that the four NEVO scanners used were properly calibrated and the operator was able to create good data density virtual models that generated consistent scores for each project evaluated. This is relevant to other studies that evaluated the repeatability of digital evaluation with the use of the same scanner at different time points and showed excellent intra class correlation of 0.93 to 0.98.3,4 However, excellent repeatability does not necessarily indicate valid evaluation as good samples from the visual paradigm could consistently be graded as poor samples in the digital paradigm and vice versa. Therefore, the correlation of visual and digital evaluation scores was used to examine this possibility. Based on the results, our second hypothesis was rejected; there was strong correlation between the visual and digital scores . This partly supports the validity of the use of digital technology in assessing students’ projects and suggests that this tool could also be used for official grading in competencies as high grades were consistently graded as high and low grades as low across the two methods. It is important to note that there was a significant difference in scores between visual and NEVO evaluation methods with scores from the visual evaluation being lower than that obtained from the digital evaluation method. Changing the tolerance level and giving more leeway for the digital evaluation method could compensate for this difference. Implementation of digital assessment tools in the dental pre-clinic seems promising. However, there are still many issues to be addressed to justify the financial investment and time needed to introduce this system into the curriculum. A systematic review of the effectiveness of computer-aided, self-instructional programs in dental education has revealed disparate outcomes, with some studies finding no difference between CAL and other learning strategies, and others indicating the opposite thus that CAL provided a significant advantage in terms of knowledge gain.7 This is an important issue that needs to be explored in future studies with regards to the use of digital assessment tools as a self directed learning tool. Will the use of these devices indeed increase the technical performance of students? Furthermore, in order to integrate new technology into the curriculum, institutions need to formulate a strategic plan and find resources for proper faculty training so that instructors involved with teaching this new technology can share a positive enthusiasm, and not perceive this as an additional burden in terms of their efforts and time. Lastly, it is critical that students are encouraged to take responsibility for their own learning.8 As such, students’ perspective in new technology implementation is vital and should be taken into consideration. Therefore, more information on whether students regard this technology helpful in preparing them for their future professional life and contributes significantly to their development remains to be probed. CONCLUSION Within the limitations of this study it can be concluded that the E4D Compare Software provides consistent grades even when different scanners are used and correlates well with visual scores.