Emotional Impact of Moral Action - A Vague Experience from Data Science.
‘What action do you intend to take?’ Is one of the most crucial questions that I ask when evaluating the requirements of a project. As a developer, understanding the intended action allows for the most suitable solution to be built, tailored to the use-case.
Actions invoke a moral response and taking actions that are morally wrong can be emotionally impactful on oneself. Negative moral emotions such as disgust, shame, anger, and guilt can be debilitating and avoiding them is preferable for maintaining a good state of mental health.
I have been involved in projects in which my work was used to facilitate actions that I believe to be immoral. I aim to express how it is possible to be involved in facilitating immoral actions without consent nor knowledge.
Statement of involvement
I cannot speak to my specific direct experiences as they could impact the reputation of my past employer(s). Therefore, I must illustrate revelations from experiences via a reductive and completely made-up example.
Invented Illustration
Three employees, a CEO, Manager, and a Data Scientist. The manager asks the Data Scientist to develop a time-series model that forecasts for each employee their prospective value to the company over time.
The model specification is clear with prescribed and well defined inputs and outputs from the Manager; the Data Scientist’s role is to build the model, develop a testing strategy, and evaluate the error rate.
The Manager tells the Data Scientist that their model will justify actions that will improve the company’s financial situation. The Manager provides a set of example actions that the company wishes to pursue:
- Provide support (training/mental health) to employees who exhibit a predicted declining or flat trend of value to the company.
- Provide motivational gifts or activities during periods in which employees are predicted to provide less value.
- Re-adjust team composition so that high and low value employees work closely to enable passive knowledge and skill transfer.
The actions described by the Manager aim to improve employee wellbeing, morale, skills, and knowledge; this, in turn, is expected to increase company value. The model is intended to be used to take actions that could be considered morally right as they aim to improve the wellbeing of colleagues.
The CEO used the model to decide which employees will receive disciplinary action or redundancy. The Manager was aware this was a possibility but did not disclose this to the Data Scientist during development. While the Manager did not lie as the model will be used to inform the actions they described, they did not express all possible actions. They failed to disclose the immoral action of assigning disciplinary action based on the employee’s projected future value.
Although not directly involved in the decision making process, the Data Scientist was complicit in developing the tools that allowed for such decisions to be justified. As a result, the Data Scientist experiences feelings of guilt for introducing the model that led to the action of his friends and colleagues to suffer disciplinary action, or even redundancy.
Back to reality
A developer can sit at a level in a company hierarchy that is many degrees of separation from the decision maker. As a result, a developer’s work can be used without their knowledge in ways not communicated during development. Whether this is intentional or just a circumstance of complex company hierarchies, the impact on the developer is the same.
I have been involved in projects in which the end action was undisclosed, and my work has been used in a way that has elicited negative feelings. In one situation, my motivation to work all but disappeared for a few weeks as I felt I had been misled and used; I find my feelings of pride and satisfaction on a successful project delivery are now tainted and look back on that work negatively. Would I have still engaged with the project if I had known? Most likely. The sole difference is I would have prepared for the consequences. I cannot suppress emotions, but I can establish a more comfortable environment for experiencing them.
Informed Development
While situations alluded to above are infrequent, they have influenced how I approach most other projects. Rather than looking to merely meet the stakeholders immediate requirements, I now look to understand the actions that they intend to take, both evident and hidden. Fundamentally, the absence of informed development can lead to negative consequences to the individual.
