The first design question any developer (using the word on a wide sense: those who develop things) should ask when facing a new project is if the project should be built at all.
We need to acknowledge the fact that there are technologies that simply should not exist, either because they have too many or too bad unintended consequences, or because they could be too easily used maliciously, or simply because they cause too much damage even when they work as intended.
I know there are no easy or definitive answers about what a developer should or shouldn’t do, but one thing that we have the responsibility to do is to think about what we are doing and to sometimes just say no, to our managers, to prospective clients, and to our own ideas. Just because something can be built doesn’t mean that it should be.
This is an extremely difficult subject that is being intensively discussed, and I don’t have answers, only a lot of questions (and not even all of them), so I want to leave here some of the discussions around the subject that I have found particularly interesting from the last few months:
- There has been a lot of scandals, and corresponding discussion, about bias in AI in the last few years. However, a lot of those discussions are centered around technological fixes to those biases, while the conversation should be much more deep and fundamental. This is a great commentary about the framing of ethical problems in technology:
- This is an excellent thread about an aspect that can be very easily overlooked during the design phases of a product: thinking about how our product can be used for evil. It talks about speculative harm analysis and the concept of “ethical debt” around the discussion over how easy is to maliciously use Voice Tweets, a new feature introduced in by Twitter in June:
- A few of the most basic questions to ask during the conceptualization of an idea: who will benefit and who can be harmed by it. A short thread about that in the context of the scandal of algorithmically grading students in the UK:
- Finally, some ideas simply should not exist. This is an example of a technological product that could be seen as a good idea for the business, but is inherently bad for society:
These are discussions that need to start happening before a technology is out in the world and harming people. Un-inventing a technology is much more difficult to do.