Skip to main content
JDRS · Open access journal
Joint Diseases and
Related Surgery
Online manuscript submissionSubmit a manuscript e-ISSN 2687-4792

Original Article · Open access

Comparison of ChatGPT and Google in addressing patients’ questions on robot-assisted total hip arthroplasty

PDF Cite this article

Mustafa Fatih Dasci

Department of Orthopedics and Traumatology, University of Health Science, Bağcılar Training and Research Hospital, İstanbul, Türkiye

ORCID: 0000-0002-2443-4802

Serkan Surucu

Department of Orthopaedics and Rehabilitation, Yale University, New Haven, USA

ORCID: 0000-0003-1551-4525

Furkan Aral

Department of Orthopedics and Traumatology, Gazi University Faculty of Medicine, Ankara, Türkiye

ORCID: 0000-0003-4879-6530

Mahmud Aydin

Department of Orthopedics and Traumatology, Memorial Şişli Hospital, İstanbul, Türkiye

ORCID: 0000-0002-2235-1480

Cihangir Turemis

Department of Orthopedics and Traumatology, Çeşme Alper Çizgenat State Hospital, İzmir, Türkiye

ORCID: 0000-0002-5794-6652

N Amir Sandiford

Joint Reconstruction Unit, Southland Hospital, University of Otago, Invercargill, New Zealand

ORCID: 0000-0002-7285-5804

Mustafa Citak

Department of Orthopaedic Surgery, HELIOS ENDO-Klinik Hamburg, Hamburg, Germany

ORCID: 0000-0003-3207-7101

Abstract

Objectives: This study aims to compare ChatGPT (Generative Pre-Trained Transformer) and Google in addressing frequently asked questions (FAQs), answers, and online sources regarding robot-assisted total hip arthroplasty (RATHA).

Materials and methods: On December 15th, 2024, the 20 most FAQs were identified by inputting the search term “Robot-Assisted Total Hip Replacement” into both Google Search and ChatGPT-4o. Twenty FAQs were independently identified using a clean Google search and a prompt to ChatGPT-4o. The FAQs on Google were sourced from the "People also ask" section, while ChatGPT was requested to generate the 20 most often asked questions. All questions, answers, and references cited were recorded. A modified version of the Rothwell system was used to categorize questions into 10 subtopics: special activities, timeline of recovery, restrictions, technical details, cost, indications/management, risks and complications, pain, longevity, and evaluation of surgery. Each reference was categorized into the following groups: commercial, academic, medical practice, single surgeon personal, or social media. Responses were also graded as “excellent response not requiring clarification” (1), “satisfactory requiring minimal clarification” (2), “satisfactory requiring moderate clarification” (3), or “unsatisfactory requiring substantial clarification” (4).

Results: Overall, 20% of the questions that Google and ChatGPT-4o considered as the most FAQ were similar to each other. Technical details (35%) were the most common categories of questions. The ChatGPT provided significantly more academic references than Google search (70% vs. 20%, p=0.0113). Conversely, Google web search cited more medical practice references (40% vs. 0%, p=0.0033), single surgeon websites (20% vs. 0%, p=0.1060), and government websites (10% vs. 0%, p=0.4872) more frequently than ChatGPT. In terms of response quality, 62% of answers were rated as Grade 1-2 (excellent or satisfactory with minimal clarification), while 38% required moderate or substantial clarification (Grades 3-4).

Conclusion: ChatGPT demonstrated comparable results to those of Google searches on information regarding RATHA, with a higher reliance on academic sources. While most responses were satisfactory, a notable proportion required further clarification, emphasizing the need for continued evaluation of these platforms to ensure accuracy and reliability in patient education. Taken together, these technologies have the capacity to enhance health literacy and provide enhanced shared decision-making for patients seeking information on RATHA.

Declarations

Author Contributions

Writing, editing, analysis: M.F.D.; Writing, data collection, statistics: S.S.; Data collection, analysis: F.A.; Data collection, proofreading, editing: M.A.; Analysis, statistics, writing, editing: C.T.; Data collection, analysis, statistics: N.A.S.; Supervision, editing, proofreading: M.C. All authors contributed to the study conception and design. All authors read and approved the final manuscript.

Conflict of Interest

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Financial Disclosure

The authors received no financial support for the research and/or authorship of this article.

Data Sharing Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Yin H, Zhang Y, Hou W, Wang L, Fu X, Liu J. Comparison of complications between total hip arthroplasty following failed internal fixation and primary total hip arthroplasty for femoral neck fractures: A meta-analysis. Jt Dis Relat Surg 2025;36:479-88. DOI: 10.52312/jdrs.2025.2230.
  2. Tsertsvadze A, Grove A, Freeman K, Court R, Johnson S, Connock M, et al. Total hip replacement for the treatment of end stage arthritis of the hip: A systematic review and meta-analysis. PLoS One 2014;9:e99804. DOI: 10.1371/journal.pone.0099804.
  3. Wang Y, Wang R, Gong S, Han L, Gong T, Yi Y, et al. A comparison of radiological and clinical outcomes between robotic-assisted and conventional total hip arthroplasty: A meta-analysis. Int J Med Robot 2023;19:e2463. DOI: 10.1002/rcs.2463.
  4. Nawabi DH, Conditt MA, Ranawat AS, Dunbar NJ, Jones J, Banks S, et al. Haptically guided robotic technology in total hip arthroplasty: A cadaveric investigation. Proc Inst Mech Eng H 2013;227:302-9. DOI: 10.1177/0954411912468540.
  5. Kim K, Kwon S, Kwon J, Hwang J. A review of roboticassisted total hip arthroplasty. Biomed Eng Lett 2023;13:523- 35. DOI: 10.1007/s13534-023-00312-9.
  6. Viswanathan VK, Jain VK, Vaish A, Jeyaraman M, Iyengar KP, Vaishya R. Chatbots and their applications in medical fields: current status and future trends: A scoping review. doi: 101177/09760016241259851 [Internet]. 2024 Jul 25 [cited 2025 Feb 23]; Available at: https://journals.sagepub.com/ doi/full/10.1177/09760016241259851
  7. Chakraborty C, Pal S, Bhattacharya M, Dash S, Lee SS. Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science. Front Artif Intell [Internet]. 2023 [cited 2025 Feb 23];6:1237704. Available at: https://pmc.ncbi.nlm.nih.gov/articles/ PMC10644239
  8. Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncol 2023;9:1437- 40. DOI: 10.1001/jamaoncol.2023.2947.
  9. Chakraborty C, Pal S, Bhattacharya M, Dash S, Lee SS. Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science. Front Artif Intell 2023;6:1237704. DOI: 10.3389/frai.2023.1237704.
  10. Andrikyan W, Sametinger SM, Kosfeld F, Jung-Poppe L, Fromm MF, Maas R, et al. Artificial intelligence-powered chatbots in search engines: A cross-sectional study on the quality and risks of drug information for patients. BMJ Qual Saf 2025;34:100-9. DOI: 10.1136/bmjqs-2024-017476.
  11. Dubin JA, Bains SS, Chen Z, Hameed D, Nace J, Mont MA, et al. Using a Google Web search analysis to assess the utility of ChatGPT in total joint arthroplasty. J Arthroplasty 2023;38:1195-202. DOI: 10.1016/j.arth.2023.04.007.
  12. Rothwell JD. In mixed company: Communicating in small groups. Boston, MA: Cengage Learning. Account Educ 2013. p. 23.
  13. Mika AP, Martin JR, Engstrom SM, Polkowski GG, Wilson JM. Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty. J Bone Joint Surg Am 2023;105:1519-26. DOI: 10.2106/JBJS.23.00209.
  14. Kunze KN, Jang SJ, Fullerton MA, Vigdorchik JM, Haddad FS. What's all the chatter about? Bone Joint J 2023;105-B:587- 9. DOI: 10.1302/0301-620X.105B6.BJJ-2023-0156.
  15. Jabson JM, Patterson JG, Kamen C. Understanding health information seeking on the internet among sexual minority people: Cross-sectional analysis from the Health Information National Trends Survey. JMIR Public Health Surveill 2017;3:e39. DOI: 10.2196/publichealth.7526.
  16. Giorgino R, Alessandri-Bonetti M, Del Re M, Verdoni F, Peretti GM, Mangiavini L. Google Bard and ChatGPT in orthopedics: Which is the better doctor in sports medicine and pediatric orthopedics? The role of AI in patient education. Diagnostics (Basel) 2024;14:1253. DOI: 10.3390/diagnostics14121253.
  17. Estanislau AR, Nobre DM, Naback FDN, Barros TSV, Souza LFM, Penido PCL. Use of the internet and social networks in orthopedics and traumatology and perspective of post COVID telemedicine. Acta Ortop Bras 2022;30:e252728. DOI: 10.1590/1413-785220223005e252728.
  18. Kaicker J, Borg Debono V, Dang W, Buckley N, Thabane L. Assessment of the quality and variability of health information on chronic pain websites using the DISCERN instrument. BMC Med 2010;8:59. DOI: 10.1186/1741-7015-8-59.
  19. Dobbins M, Watson S, Read K, Graham K, Yousefi Nooraie R, Levinson AJ. A tool that assesses the evidence, transparency, and usability of online health information: Development and reliability assessment. JMIR Aging 2018;1:e3. DOI: 10.2196/aging.9216.
  20. Sun Y, Zhang Y, Gwizdka J, Trace CB. Consumer evaluation of the quality of online health information: Systematic literature review of relevant criteria and indicators. J Med Internet Res 2019;21:e12522. DOI: 10.2196/12522.
  21. Vivekanantham A, Protheroe J, Muller S, Hider S. Evaluating on-line health information for patients with polymyalgia rheumatica: A descriptive study. BMC Musculoskelet Disord 2017;18:43. DOI: 10.1186/s12891-017-1416-5.
  22. Megalla M, Hahn AK, Bauer JA, Windsor JT, Grace ZT, Gedman MA, et al. ChatGPT and Google provide mostly excellent or satisfactory responses to the most frequently asked patient questions related to Rotator Cuff Repair. Arthrosc Sports Med Rehabil 2024;6:100963. DOI: 10.1016/j.asmr.2024.100963.
  23. Tharakan S, Klein B, Bartlett L, Atlas A, Parada SA, Cohn RM. Do ChatGPT and Google differ in answers to commonly asked patient questions regarding total shoulder and total elbow arthroplasty? J Shoulder Elbow Surg 2024;33:e429-37. DOI: 10.1016/j.jse.2023.11.014.
  24. Varady NH, Lu AZ, Mazzucco M, Dines JS, Altchek DW, Williams RJ 3rd, et al. Understanding how ChatGPT may become a clinical administrative tool through an investigation on the ability to answer common patient questions concerning Ulnar Collateral Ligament Injuries. Orthop J Sports Med 2024;12:23259671241257516. DOI: 10.1177/23259671241257516.
  25. Oeding JF, Lu AZ, Mazzucco M, Fu MC, Taylor SA, Dines DM, et al. ChatGPT-4 performs clinical information retrieval tasks using consistently more trustworthy resources than does Google Search for queries concerning the latarjet procedure. Arthroscopy 2025;41:588-97. DOI: 10.1016/j.arthro.2024.05.025.
  26. Shen TS, Driscoll DA, Islam W, Bovonratwet P, Haas SB, Su EP. Modern internet search analytics and total joint arthroplasty: What are patients asking and reading online? J Arthroplasty 2021;36:1224-31. DOI: 10.1016/j.arth.2020.10.024.
  27. McCormick JR, Kruchten MC, Mehta N, Damodar D, Horner NS, Carey KD, et al. Internet search analytics for shoulder arthroplasty: What questions are patients asking? Clin Shoulder Elb 2023;26:55-63. DOI: 10.5397/cise.2022.01347.
  28. Subramanian T, Shahi P, Araghi K, Mayaan O, Amen TB, Iyer S, et al. Using artificial intelligence to answer common patient-focused questions in minimally invasive spine surgery. J Bone Joint Surg Am 2023;105:1649-53. DOI: 10.2106/JBJS.23.00043.
  29. Shrestha N, Shen Z, Zaidat B, Duey AH, Tang JE, Ahmed W, et al. Performance of ChatGPT on NASS Clinical Guidelines for the diagnosis and treatment of low back pain: A comparison study. Spine (Phila Pa 1976) 2024;49:640-51. DOI: 10.1097/BRS.0000000000004915.
  30. Price WN 2nd, Gerke S, Cohen IG. How much can potential jurors tell us about liability for medical artificial intelligence? J Nucl Med 2021;62:15-6. DOI: 10.2967/jnumed.120.257196.
  31. Johnson CK, Mandalia K, Corban J, Beall KE, Shah SS. Adequacy of ChatGPT responses to frequently asked questions about shoulder arthroplasty: Is it an appropriate adjunct for patient education? JSES Int 2025;9:830-6. DOI: 10.1016/j.jseint.2025.01.008.
  32. Özbek EA, Ertan MB, Kından P, Karaca MO, Gürsoy S, Chahla J. ChatGPT can offer at least satisfactory responses to common patient questions regarding hip arthroscopy. Arthroscopy 2025;41:1806-27. DOI: 10.1016/j.arthro.2024.08.036.
  33. Slawaska-Eng D, Bourgeault-Gagnon Y, Cohen D, Pauyo T, Belzile EL, Ayeni OR. ChatGPT-3.5 and -4 provide mostly accurate information when answering patients' questions relating to femoroacetabular impingement syndrome and arthroscopic hip surgery. J ISAKOS 2025;10:100376. DOI: 10.1016/j.jisako.2024.100376.
  34. Adelstein JM, Sinkler MA, Li LT, Fortier LM, Vakharia AM, Salata MJ. ChatGPT can often respond adequately to common patient questions regarding femoroacetabular impingement. Clin J Sport Med 2024. DOI: 10.1097/JSM.0000000000001327.
  35. Adelstein JM, Sinkler MA, Li LT, Mistovich RJ. ChatGPT responses to common questions about slipped capital femoral epiphysis: A reliable resource for parents? J Pediatr Orthop 2024;44:353-7. DOI: 10.1097/BPO.0000000000002681.
  36. Li AW, Adelstein JM, Li LT, Sinkler MA, Mistovich RJ. Assessing ChatGPT responses to frequently asked questions regarding pediatric supracondylar humerus fractures. J Pediatr Orthop 2025;45:327-31. DOI: 10.1097/BPO.0000000000002923.
  37. AlShehri Y, McConkey M, Lodhia P. ChatGPT provides satisfactory but occasionally inaccurate answers to common patient hip arthroscopy questions. Arthroscopy 2025;41:1337-47. DOI: 10.1016/j.arthro.2024.06.017.
  38. Shayegh NA, Byer D, Griffiths Y, Coleman PW, Deane LA, Tonkin J. Assessing artificial intelligence responses to common patient questions regarding inflatable penile prostheses using a publicly available natural language processing tool (ChatGPT). Can J Urol 2024;31:11880-5.
  39. Winden F, Bormann M, Gilbert F, Holzapfel BM, Berthold DP. ChatGPT delivers satisfactory responses to the most frequent questions on meniscus surgery. Knee 2025;56:249- 57. DOI: 10.1016/j.knee.2025.05.018.
  40. Li LT, Adelstein JM, Sinkler MA, Mistovich RJ. Artificial intelligence promotes the Dunning Kruger Effect: Evaluating ChatGPT answers to frequently asked questions about adolescent idiopathic scoliosis. J Am Acad Orthop Surg 2025;33:473-80. DOI: 10.5435/JAAOS-D-24-00297.
  41. Kolac UC, Karademir OM, Ayik G, Kaymakoglu M, Familiari F, Huri G. Can popular AI large language models provide reliable answers to frequently asked questions about rotator cuff tears? JSES Int 2024;9:390-7. DOI: 10.1016/j.jseint.2024.11.012.
  42. Tan SS, Goonawardene N. Internet health information seeking and the patient-physician relationship: A systematic review. J Med Internet Res 2017;19:e9. DOI: 10.2196/jmir.5729.
  43. Shen OY, Pratap JS, Li X, Chen NC, Bhashyam AR. How does ChatGPT use source information compared with Google? A text network analysis of online health information. Clin Orthop Relat Res 2024;482:578-88. DOI: 10.1097/CORR.0000000000002995.