Skip to main content
JDRS · Open access journal
Joint Diseases and
Related Surgery
Online manuscript submissionSubmit a manuscript e-ISSN 2687-4792

Original Article · Open access

Evaluating deepresearch and deepthink in total knee arthroplasty patient education: ChatGPT-4o excels in comprehensiveness, Deepseek R1 leads in clarity and readability of orthopedic information

PDF Cite this article

Onur Gultekin

Department of Orthopedics and Traumatology, University of Health Sciences, İstanbul Fatih Sultan Mehmet Training and Research Hospital, İstanbul, Türkiye

ORCID: 0000-0002-0836-076X

Michael T. Hirschmann

Department of Orthopedic Surgery and Traumatology, Kantonsspital Baselland, Bruderholz, Switzerland Department of Clinical Research, Research Group Michael T. Hirschmann, Regenerative Medicine & Biomechanics, University of Basel, Basel, Switzerland

ORCID: 0000-0002-4014-424X

Halil İbrahim Arıkan

Department of Orthopedics and Traumatology, Viranşehir State Hospital, Şanlıurfa, Türkiye

ORCID: 0009-0007-5787-7785

Bekir Eray Kilinc

Department of Orthopedics and Traumatology, University of Health Sciences, İstanbul Fatih Sultan Mehmet Training and Research Hospital, İstanbul, Türkiye

ORCID: 0000-0003-1229-9815

Baris Yilmaz

Department of Orthopedics and Traumatology, University of Health Sciences, İstanbul Fatih Sultan Mehmet Training and Research Hospital, İstanbul, Türkiye

ORCID: 0000-0003-2023-267X

Süleyman Abul

Department of Orthopedics and Traumatology, İstanbul Kartal Dr. Lütfi Kırdar City Hospital, İstanbul, Türkiye

ORCID: 0000-0002-3197-5770

Jumpei Inoue

Department of Orthopaedic Surgery, Nagoya Tokushukai General Hospital, Kasugai, Aichi, Japan

ORCID: 0000-0003-3953-5646

Mahmut Enes Kayaalp

Department of Orthopedics and Traumatology, University of Health Sciences, İstanbul Fatih Sultan Mehmet Training and Research Hospital, İstanbul, Türkiye Department of Orthopaedics and Traumatology, University Hospital Brandenburg/Havel, Brandenburg Medical School Theodor Fontane, Brandenburg/Havel, Germany Brandenburg Medical School Theodor Fontane, Faculty of Health Sciences Brandenburg, Brandenburg/Havel, Germany

ORCID: 0000-0002-9545-7454

Abstract

Objectives: This study aims to directly compare ChatGPT and DeepSeek, both equipped with DeepResearch/DeepThink capabilities, based on their responses to frequently asked questions (FAQs) on total knee arthroplasty (TKA).

Materials and methods: Thirty frequently asked questions related to TKA were compiled from validated patient education sources, including American Academy of Orthopaedic Surgeons (AAOS) OrthoInfo, National Institute for Health and Care Excellence (NICE) guidelines, and popular patient discussion forums, and verified for clinical relevance by two independent arthroplasty surgeons. Two orthopedic surgeons, blinded to model identity, evaluated each response using a five-point Likert scale across five domains: accuracy, comprehensiveness, readability, relevance, and ethical and safety considerations. The maximum total score per response was 25. Readability was also assessed using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease Score (FRES). Inter-rater and intra-rater reliability were calculated using intraclass correlation coefficients (ICCs).

Results: The ChatGPT-4o scored significantly higher in comprehensiveness and clinical detail, whereas DeepSeek R1 produced responses with superior readability, indicated by a lower FKGL (7.5 vs. 10.2) and higher FRES (62.3 vs. 45.6) (p < 0.05). Both models demonstrated high accuracy and safety, with no factual errors identified. Intra-rater reliability was excellent (ICC > 0.81), and inter-rater agreement ranged from fair to substantial (ICC 0.31 to 0.63).

Conclusion: Both ChatGPT-4o and DeepSeek R1 are capable of generating accurate, ethically sound, and clinically relevant educational content for patients undergoing TKA. While ChatGPT-4o offers more comprehensive information, DeepSeek R1 provides content that is more accessible to patients with lower health literacy. Model selection should be tailored to the target population to optimize educational effectiveness in clinical practice. The ability of real-time data retrieval to incorporate the most current clinical evidence and guideline updates may further enhance the educational quality, reliability, and clinical relevance of AI-generated patient information.

Declarations

Author Contributions

M.E.K., M.T.H., J.I.: Idea/concept, analysis; O.G., H.İ.A., B.E.K., B.Y.: Design; H.İ.A., S.A.: Data collection; O.G., M.E.K.: Literature review; O.G., M.E.K., H.İ.A., S.A.: Writing the article; B.Y., M.E.K., M.T.H., J.I., B.E.K. Critical review.

Conflict of Interest

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Financial Disclosure

The authors received no financial support for the research and/or authorship of this article.

Data Sharing Statement

The datasets generated and analyzed during the current study (AI-generated responses and blinded evaluator scoring sheets) are available from the corresponding author.

AI Disclosure

The authors declare that artificial intelligence (AI) tools were not used, or were used solely for language editing, and had no role in data analysis, interpretation, or the formulation of conclusions. All scientific content, data interpretation, and conclusions are the sole responsibility of the authors. The authors further confirm that AI tools were not used to generate, fabricate, or ‘hallucinate’ references, and that all references have been carefully verified for accuracy.

References

  1. Bujnowska-Fedak MM, Węgierek P. The ımpact of online health information on patient health behaviours and making decisions concerning health. Int J Environ Res Public Health 2020;17:880. DOI: 10.3390/ijerph17030880.
  2. Badarudeen S, Sabharwal S. Assessing readability of patient education materials: Current role in orthopaedics. Clin Orthop Relat Res. 2010;468:2572-80. DOI: 10.1007/s11999-010-1380-y.
  3. Fahy S, Oehme S, Milinkovic D, Jung T, Bartek B. Assessment of quality and readability of information provided by ChatGPT in relation to anterior cruciate ligament ınjury. J Pers Med 2024;14:104. DOI: 10.3390/jpm14010104.
  4. Sadoghi P, Koutp A, Prieto DP, Clauss M, Kayaalp ME, Hirschmann MT. The projected economic burden and complications of revision hip and knee arthroplasties: Insights from national registry studies. Knee Surg Sports Traumatol Arthrosc 2025;33:3211-7. DOI: 10.1002/ksa.12678.
  5. Cieremans DA, Arraut J, Marwin S, Slover J, Schwarzkopf R, Rozell JC. Patellar component design does not impact clinical outcomes in primary total knee arthroplasty. J Arthroplasty 2023;38:1493-8. DOI: 10.1016/j.arth.2023.01.061.
  6. Dahmen J, Kayaalp ME, Ollivier M, Pareek A, Hirschmann MT, Karlsson J, et al. Artificial intelligence bot ChatGPT in medical research: The potential game changer as a double-edged sword. Knee Surg Sports Traumatol Arthrosc 2023;31:1187-9. DOI: 10.1007/s00167-023-07355-6.
  7. Fayed AM, Mansur NSB, de Carvalho KA, Behrens A, D'Hooghe P, de Cesar Netto C. Artificial intelligence and ChatGPT in orthopaedics and sports medicine. J Exp Orthop 2023;10:74. DOI: 10.1186/s40634-023-00642-8.
  8. Kaarre J, Feldt R, Keeling LE, Dadoo S, Zsidai B, Hughes JD, et al. Exploring the potential of ChatGPT as a supplementary tool for providing orthopaedic information. Knee Surg Sports Traumatol Arthrosc 2023;31:5190-8. DOI: 10.1007/s00167-023-07529-2.
  9. Ollivier M, Pareek A, Dahmen J, Kayaalp ME, Winkler PW, Hirschmann MT, et al. A deeper dive into ChatGPT: History, use and future perspectives for orthopaedic research. Knee Surg Sports Traumatol Arthrosc 2023;31:1190-2. DOI: 10.1007/s00167-023-07372-5.
  10. Kayaalp ME, Ollivier M, Winkler PW, Dahmen J, Musahl V, Hirschmann MT, et al. Embrace responsible ChatGPT usage to overcome language barriers in academic writing. Knee Surg Sports Traumatol Arthrosc 2024;32:5-9. DOI: 10.1002/ksa.12014.
  11. Ayık G, Ercan N, Demirtaş Y, Yıldırım T, Çakmak G. Evaluation of ChatGPT-4o's answers to questions about hip arthroscopy from the patient perspective. Jt Dis Relat Surg 2025;36:193-9. DOI: 10.52312/jdrs.2025.1961.
  12. Gültekin O, Inoue J, Yilmaz B, Cerci MH, Kilinc BE, Yilmaz H, et al. Evaluating DeepResearch and DeepThink in anterior cruciate ligament surgery patient education: ChatGPT-4o excels in comprehensiveness, DeepSeek R1 leads in clarity and readability of orthopaedic information. Knee Surg Sports Traumatol Arthrosc 2025;33:3025-31. DOI: 10.1002/ksa.12711.
  13. Kayaalp ME, Prill R, Sezgin EA, Cong T, Królikowska A, Hirschmann MT. DeepSeek versus ChatGPT: Multimodal artificial intelligence revolutionizing scientific discovery. From language editing to autonomous content generationRedefining innovation in research and practice. Knee Surg Sports Traumatol Arthrosc 2025;33:1553-6. DOI: 10.1002/ksa.12628.
  14. Yapar D, Demir Avcı Y, Tokur Sonuvar E, Eğerci ÖF, Yapar A. ChatGPT's potential to support home care for patients in the early period after orthopedic interventions and enhance public health. Jt Dis Relat Surg 2024;35:169-76. DOI: 10.52312/jdrs.2023.1402.
  15. Agharia S, Szatkowski J, Fraval A, Stevens J, Zhou Y. The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard. J Orthop 2023;50:1-7. DOI: 10.1016/j.jor.2023.11.063.
  16. American Academy of Orthopaedic Surgeons. Total knee replacement. OrthoInfo. 2025 [cited 22.09.2025]. Available from: https://orthoinfo.aaos.org.
  17. National Institute for Health and Care Excellence. Total knee replacement guidelines. NICE. 2025 [cited 22.09.2025]. Available from: https://www.nice.org.uk/ guidance.
  18. OpenAI. Deep Research system card. 2025 [cited 22.09.2025]. Available from: https://cdn.openai.com/deep-research-system-card.pdf.
  19. Sun Y, Zhang Y, Gwizdka J, Trace CB. Consumer evaluation of the quality of online health information: Systematic literature review of relevant criteria and indicators. J Med Internet Res 2019;21:e12522. DOI: 10.2196/12522.
  20. Readability Formulas. Free readability tools. 2025 [cited 22.09.2025]. Available from: https://www. readabilityformulas.com.
  21. Flesch R. A new readability yardstick. J Appl Psychol 1948;32:221-33. DOI: 10.1037/h0057532.
  22. Kincaid JP. Derivation of new readability formulas: automated readability index, fog count and Flesch reading ease formula for navy enlisted personnel. Memphis: Chief of Naval Technical Training, Naval Air Station; 1975.
  23. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155-63. DOI: 10.1016/j.jcm.2016.02.012.
  24. Wang D, Liang J, Ye J, Li J, Li J, Zhang Q, et al. Enhancement of the performance of large language models in diabetes education through retrieval-augmented generation: Comparative study. J Med Internet Res 2024;26:e58041. DOI: 10.2196/58041.
  25. Eltorai AE, Ghanian S, Adams CA Jr, Born CT, Daniels AH. Readability of patient education materials on the american association for surgery of trauma website. Arch Trauma Res 2014;3:e18161. DOI: 10.5812/atr.18161.
  26. Dubin JA, Bains SS, Chen Z, Hameed D, Nace J, Mont MA, et al. Using a Google web search analysis to assess the utility of ChatGPT in total joint arthroplasty. J Arthroplasty 2023;38:1195-202. DOI: 10.1016/j.arth.2023.04.007.
  27. Oviedo-Trespalacios O, Peden AE, Cole-Hunter T, Costantini A, Haghani M, Javier E. Rodriguez, JE, et al. The risks of using ChatGPT to obtain common safety-related information and advice. Saf Sci 2023;167:106024.