A Modeling Approach for Measuring the Performance of a Human-AI Collaborative Process
- Despite the unabated growth of algorithmic decision-making in organizations, there is a growing consensus that numerous situations will continue to require humans in the loop. However, the blending of a formal machine and bounded human rationality also amplifies the risk of what is known as local rationality. Therefore, it is crucial, especially in a data-abundant environment that characterizes algorithmic decision-making, to devise means to assess performance holistically. In this paper, we propose a simulation-based model to address the current lack of research on quantifying algorithmic interventions in a broader organizational context. Our approach allows the combining of causal modeling and data science algorithms to represent decision settings involving a mix of machine and human rationality to measure performance. As a testbed, we consider the case of a fictitious company trying to improve its forecasting process with the help of a machine learning approach. The example demonstrates that a myopic assessment obscures problems that only a broader framing reveals. It highlights the value of a systems view since the effects of the interplay between human and algorithmic decisions can be largely unintuitive. Such a simulation-based approach can be an effective tool in efforts to delineate roles for humans and algorithms in hybrid contexts.
Author: | Ganesh Sankaran, Marco A. Palomino, Martin KnahlGND, Guido SiestrupGND |
---|---|
URN: | https://urn:nbn:de:bsz:fn1-opus4-90522 |
DOI: | https://doi.org/10.3390/app122211642 |
ISSN: | 2076-3417 |
Parent Title (English): | Applied Sciences |
Document Type: | Article (peer-reviewed) |
Language: | English |
Year of Completion: | 2022 |
Release Date: | 2023/01/16 |
Tag: | Algorithmic decision-making; Machine learning; Simulation modeling; Supply chain planning; System dynamics |
Volume: | 12.2022 |
Issue: | 22 |
Article Number: | 11642 |
Page Number: | 26 |
Open-Access-Status: | Open Access |
Licence (German): | Creative Commons - CC BY - Namensnennung 4.0 International |