Clinician and CDSS as Hybrid Intelligence

In our approach to clinical decision-making, we contend that clinical decision-making is, in practice, a complex and intricate reasoning process. We argue that CRSS can play a role in or even improve this reasoning process as a clinical reasoning support system,provided that the system is reliable, that its outcomes are explainable in relevant respects, and that an empirical link can be established between the algorithm and the individual patient. If these requirements are met, clinicians can combine their human intelligence with the artificial intelligence of a CRSS into a hybrid intelligence,30, 11As rules-based systems and data-driven have different capacities, Steels and Lopez de Mantaraz (2018) suggest that “The full potential of AI will only be realized with a combination of these two approaches, meaning a form of hybrid AI.” (ibid. 488) This is a different type of hybrid than we have in mind here. in which both have clearly delineated and complementary tasks. To achieve this, CRSSs must be given highly standardized and trainable epistemic tasks. For example, CRSSs can provide accurate and precise classifications based on their ability to detect patterns that are not discernible by humans. Or they can help search the database for the most suitable procedure, supported by the most up to date scientific evidence. Machine learning algorithms make use of large amounts of data, and therefore are able to establish similarities and correlations between (sub)groups of patients and rare cases. The task of clinicians is to incorporate the outcomes of CRSS into medical reasoning, first of all by hypothesizing about possible causes of the patient’s signs and symptoms, and secondly by selecting the appropriate test to confirm or reject this hypothesis. In addition, clinicians are tasked with determining what data is relevant, collecting that data and entering it into the CRSS, such that the system can use it. In short, the task of a clinician is to ask questions that the CRSS can answer. Moreover, clinicians are tasked with interpreting, integrating and contextualizing the outcome of the CRSS, in order to utilize it for empirical tasks in practice.
Additionally, we have defended that clinical experts need to be closely involved in the development of AI systems. Developing a CRSS that facilitates clinical reasoning in practice means that clinicians, as future users, need to be involved at an early stage of development. They need to ensure that the system is designed to answer questions that are relevant to the clinical reasoning process, and that the data that is collected and used as training data is relevant and suitable to their patient population (e.g., that a CRSS to diagnose skin cancer is not just trained on using data of patients with white skin31 and that the outcomes generated by the CRSS are interpretable. This requires that, along with an advice, the system indicates what factors contributed to arriving at that advice, allowing the user to evaluate whether these factors are indeed medically plausible and applicable to the current patient. In addition, medical education should prepare clinicians to perform the new epistemic tasks required to use CRSS in clinical practice. For example by teaching students how data is collected and processed and by teaching them how to evaluate whether the context in which the data is collected is relevant to the intended application.
In conclusion, a CRSS can aid clinical decision-making and possibly improve it, if clinicians use it in an epistemologically responsible manner. Both the system and their users need to be equipped for this. Clinicians need to develop new cognitive skills necessary to perform specific epistemic tasks related to the use of CRSS. For example, establishing an empirical link between the model and the individual patient, asking appropriate questions (that can be answered by the system), collecting and assessing the required data and evaluating the outcome. CRSS must not only be reliable, in the sense that the performance is scientifically proven to be as good or better than that of medical experts. It must also provide the information necessary to enable the clinician to perform the necessary epistemic activities, in a way that supports the performance of these activities. This entails, for example, to provide insight in the data set that is used to train the algorithm (e.g., which characteristics of patients were included in the data), as well as a precise description of the task that the algorithm is trained to perform (e.g., to use images of skin lesions for identifying melanoma); to give information about the reliability of the outcome (such as confidence intervals) and; to give information about the procedure with which the algorithm arrives at the outcome (i.e. the weightings of the different pieces of information).
If developers and users succeed in meeting the requirements that allow them to combine human and artificial intelligence into hybrid intelligence, CRSS holds great promises for health care by improving the accuracy, speed and consistency of clinical decision-making.