Human Experts in AI Clinical Evaluation: An Irreplaceable Role in Global Health Governance

When AI Evaluates AI: The Trust Gap in Clinical Settings

Global healthcare systems are facing the dual pressure of resource scarcity and rising demand, with artificial intelligence seen as a key tool to improve efficiency and expand coverage. However, a recent study published in *npj Digital Medicine* issues a warning: relying entirely on automated evaluation systems for quality assessment of clinical AI outputs may overlook critical local issues.

The research team compared the performance of human expert evaluations with “LLM-as-a-Judge” (large language models as judges) and found that although AI systems can provide consistent and low-cost ratings, local clinicians still identified important concerns missed by the automated assessment. This finding points directly to a core contradiction in global digital health strategies: how to preserve the contextual judgment of human experts while scaling up AI deployment.

The Value of Local Knowledge in Digital Health

This result is particularly important for countries in the Global South. Many low- and middle-income countries are leveraging AI to address physician shortages, but the training data often come from high-income settings, potentially leading to diagnostic biases. The study shows that local clinical experts can identify subtle issues related to disease prevalence patterns, cultural practices, and infrastructure constraints—issues that often lie beyond the scope of general-purpose models.

For example, in sub-Saharan Africa, an AI-assisted malaria diagnostic system may need to account for patients’ multi-infection risks, drug availability, and the testing capacity of local laboratories. Automated evaluation might focus solely on algorithmic accuracy, whereas human experts can point out gaps in usability within real clinical workflows.

ESG Perspective: Social Equity and Governance Responsibility

Viewed through the ESG (Environmental, Social, and Governance) framework, the study highlights three key issues:

• Social (S): AI deployment should enhance, not undermine, health equity. If evaluation systems ignore local voices, marginalized groups may be subjected to unsuitable AI solutions.
• Governance (G): A transparent and accountable AI regulatory framework is essential. The study recommends incorporating ongoing human review throughout the AI system lifecycle, especially in high-risk clinical decisions.
• Environmental (E): While AI itself has a carbon footprint, efficient and accurate diagnostics can reduce resource waste from over-testing, ultimately contributing to sustainable development goals.

Implications for Global Governance

The World Health Organization has already released the *Ethics and Governance of Artificial Intelligence for Health*, emphasizing the “human-in-the-loop” principle. This study provides empirical support for this principle: when human experts and AI collaborate in evaluation, it is possible to leverage AI’s economies of scale while retaining the depth of clinical judgment.

International cooperation organizations should promote the establishment of “hybrid evaluation” standards: 1. Develop localized evaluation toolkits for low-resource settings; 2. Fund data collection for AI training and evaluation in Global South countries; 3. Incorporate human review into performance indicators for development financing projects.## Conclusion: Technological Iteration and Human-Machine Collaboration

The potential of AI in health is undeniable, but technological progress should not come at the expense of local knowledge. This study reminds policymakers, developers, and funders: while pursuing efficiency, investment must also be made in human capacity building and regulatory mechanisms. The true path to achieving Sustainable Development Goal 3 (Good Health and Well-being) is to make artificial intelligence and human intelligence complementary, rather than substitutes for each other.