Consider the following projects:<p>* Facebook's Free Basics initiative in India:
http://www.bbc.com/news/technology-35169226<p>* The Stingray phone surveillance device:
https://en.wikipedia.org/wiki/Stingray_phone_tracker<p>* North Korea's Red Star operating system:
https://media.ccc.de/v/32c3-7174-lifting_the_fog_on_red_star_os<p>* Hacking Team's surveillance software (sold to countries with a poor human rights track record):
https://theintercept.com/2015/07/07/leaked-documents-confirm-hacking-team-sells-spyware-repressive-countries/<p>Do you think that the software engineers who consented to work on the above projects acted ethically?
Errr, define "ethically".<p>I'm not trying to trap you. I'm willing to work with whatever definition you specify, and I won't try to play semantic games with the definition if it's at least close enough to something specific to work with. I'm not asking for a universality claim. But without some specification of what you mean the question is vague to the point of unanswerability.<p>The North Korean programmers may well have truly believed in what they were doing. A utilitarian may well truly believe that even if Facebook Free Basics isn't a perfect program, it's a net good for the participants. The "surveillance software" can be seen as just a tool and whether the tool makers are responsible for its misuse is ethically debatable. (And let me be clear I mean that literally, not as an attempt to rhetorically state a position. I could write a coherent argument both ways.) After all, many people even in free countries end up calling for strict regulation of corporations and that same "surveillance software" is pretty much the way you instantiate such regulation, so, is it <i>really</i> clear that it's intrinsically evil?<p>And again let me emphasize the point I'm making here is just the width of possible arguments about ethics that can be made. My previous paragraph is itself ethically incoherent, inasmuch I'm not even trying to take a consistent stand overall, but merely trying to highlight the most obvious problem per issue where ill-defined "ethics" makes it hard to even debate the matter.
How about the engineers who wrote this code? <a href="https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal" rel="nofollow">https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal</a><p>I can't put all the blame on the software engineers involved: the decision to do this surely lies higher up the chain. But did anyone of the coders object or blow the whistle beforehand? Or were they all fine with "just following orders"?<p>Now that more is known, have any come forward or even given an anonymous interview about it afterwards? AFAIK,no.
I've always thought we could benefit from having a Hippocratic Oath for software developers and taking it seriously. The tough part would be agreeing what kinds of projects are and are not ethical to work on.<p>I personally have quit a job in the past because (among sever other important reasons) I felt the projects' primary application (surveillance) was not something I wanted to be associated with. I have also chosen not to voluntarily participate in the patent process for any software I've designed (foregoing those slimy patent "bonuses" other engineers seem to like to gobble up). But not every software developer shares my particular set of ethics. How would you come up with a definitive list of what does and does not violate the "Software Hippocratic Oath"?
Its nearly impossible to answer your question, because the examples you give all have an element of "its bigger than one person".<p>In all of those cases, it it is conceivable that programmers could easily justify the "larger purpose" in their own moral/ethical framework. This a problem inherent in all collective efforts.<p><i>"There's always the same amount of good luck and bad luck in the world. If one person doesn't get the bad luck, somebody else will have to get it in their place. There's always the same amount of good and evil, too. We can't eradicate evil, we can only evict it, force it to move across town. And when evil moves, some good always goes with it. But we can never alter the ratio of good to evil. All we can do is keep things stirred up so neither good nor evil solidifies. That's when things get scary. Life is like a stew, you have to stir it frequently, or all the scum rises to the top."</i>[1]<p>[1]: <a href="http://www.goodreads.com/quotes/12020-there-s-always-the-same-amount-of-good-luck-and-bad" rel="nofollow">http://www.goodreads.com/quotes/12020-there-s-always-the-sam...</a>
How about the extensive surveillance software developed by the NSA (PRISM et. al.)? That's probably the biggest example of ethical and moral failure in modern history.
For an older example, see [0].<p>[0] - <a href="https://en.wikipedia.org/wiki/IBM_and_the_Holocaust" rel="nofollow">https://en.wikipedia.org/wiki/IBM_and_the_Holocaust</a>
There's a lot of ethical failure in failed implementations.<p>I realize these kind of projects are big consulting firms' bread and butter, but if you as an engineer continue to work on a project that's being managed with an eye towards "Failure is okay, because our lawyers wrote the contract to cover that contingency" then that's pretty scummy from an ethical perspective.<p><a href="https://en.wikipedia.org/wiki/Virtual_Case_File" rel="nofollow">https://en.wikipedia.org/wiki/Virtual_Case_File</a><p><i>Edit</i>: On the flipside, I feel a lot more comfortable when employers I've worked for have sat down with customers and had the "Look, this just isn't working out. We recommend you cancel this project and we tie off our relationship, because it's not going to end well for either of us if we continue" talk.
I'd add:<p>- Anyone who works for or supports militaries who wage aggressive wars or military actions. Arguably, depending on your stance on pacifism, anyone who works for or supports the military at all.<p>- Anyone who supports torture or "intensive interrogation".<p>- Anyone who helps to imprison people for victimless crimes, or otherwise supports the injustices of the legal system.<p>- Anyone who works in or supports unsolicited advertising which tries to manipulate people in to buying junk they don't need, often lying in the process.<p>- Anyone who works to support the security, surveillance, or police state aparatus.<p>Given certain viewpoints these are ethically indefensible. From other viewpoints these can be not only ethical, but righteous. A lot depends on where you're coming from and what your politics are.<p>Unfortunately, I don't think most people care or think much about the ethical implications of their work or what they do. And if they do think about it, they usually just throw up their hands and say there's nothing they can do about it, or everyone else is doing it and if they don't do it somebody else will, or that they got to make a living somehow, or that at least they're not doing something even worse, or that they were just following orders. The excuses, even in the rare cases when people recognize there might be an ethical problem with their actions, are legion.
It is subjective, but you can argue that any device ever made with intention to kill or harm people had people involved who knew what they were doing and created it anyway. Everything from Gas chambers in concentration camps (made supposedly by Siemens or Bosch), bombs, guns, even swords.<p>And then there is Java EE...
Why focus on these iffy things rather than say - ransomware, someone basically wrote a nw.js based ransomware as a service tool even. Are these people not professional enough or something?
It's impossible to say without knowing more about the alternative courses of action available to the individuals involved.<p>You can't fault someone for making a morally wrong choice if they never had a morally right option to choose.<p>They might also genuinely believe--perhaps due to misinformation--that they are doing good rather than evil.<p>Besides that, I believe that it is acceptable to perform an action for the benefit of a mutually loyal relationship at the expense of anonymous strangers.<p>It may be that someone who has a moral objection to IMSI catchers would, instead of endangering himself for the sake of strangers, anonymously pass a tip to a defense attorney containing a suggestion for discovery. Would that person then be condemned as unethical for supporting the technology, or expiated by peeling away one of the veils of secrecy? There is never any shortage of skilled workers who can be psychologically manipulated into acting against their own interests, after all. If the morally conflicted person never worked there, the secrecy may have remained intact longer, creating a larger window for potential abuse.<p>It's very difficult to condemn someone for trying to collect a regular wage. "I was only following orders" is not exculpatory, but if the alternative to following the order is sufficiently dire, it is sufficient to discourage me from adding moral condemnation to any of the other consequences that may result.
I know somebody who knows somebody :-) who did the following:<p>In a mad scramble to create some impromptu how-to instructions for a customer's users, the person made screenshots that revealed a bit of personal and confidential information.<p>The person doing that knew it was wrong at the time. This took place in a dodgy ethical milieu, where the customer's management refused to support the publication of high-quality how-to materials, and the customer's users were failing and flailing, and themselves emailing screenshots showing private data.<p>In my belief, this kind of breach of ethics is much more common than colossal systematic moral collapse. The trouble is this: It contributes to a morally slippery workplace.
The engineers working on those projects may grasp that there are problematic sides to the wider-scope of what they do, but they are not the ones running the show and they are just trying to do their job. The ones running the show usually have very good compelling reasons to keep running it (national security, avoid company crisis and keep jobs for many families.. etc) even when things are illegal or borderline, and usually can diffuse responsibility in a larger board, subordinates etc...<p>In short, most people won't perceive their job as unethical at all. Those who see the problems, won't relate them to the tiny bit they work on but to the whole that doesn't depend on them (and are probably right).<p>And those who get enough remorse to stop doing their tiny bit, will be replaced by someone else who will happily continue it anyway.<p>So I don't blame the engineers working on those systems despite the results being so immoral.<p>Appeals to ethics are not very useful. Otherwise most of the Tech giants of this world, not only the NSA, would have an awful time finding someone to work for them.
There are software engineers who follow some definition of ethics, and there are software engineers who ignore some definition of ethics - just like every other profession in the world.<p>What do you hope to achieve by finding real world examples? People do Bad Things for some definition of bad all the time, no matter their profession, but good luck trying to make any claim to absolute wrongdoing without the discussion devolving into semantics.<p>It will be very difficult for you to ascribe a moral position to any real world example because the real world isn't binary. To a first approximation, debates about ethics are usually won by vindicating the majority in-group's opinion about something they disagree with.<p>As you can tell, I'm basically saying this discussion isn't productive. You'll probably get some trendy answers like "NSA" or something to do with "surveillance" but I'll bite - how about the blackhat hackers who are employed by organized crime to de-anonymize "problem people" so those people can be found and "dealt with"? But that's just my opinion.<p>One of the biggest problems with this sort of thing is that no matter how powerfully you might believe someone working at e.g. the NSA is doing something evil, that individual likely feels just as powerfully that they are working to increase the net good in the world. In fact, they probably have an objectively coherent argument in favor of their position.
<a href="https://en.wikipedia.org/wiki/Room_641A" rel="nofollow">https://en.wikipedia.org/wiki/Room_641A</a><p>Well, not really about SOFTWARE engineers, I suppose.
My question is slightly different. Is it the writing of the software that is ethically at issue, or is it how the software is utilized?<p>Are the scientists that develop new drugs which have potential side affects, even death, murders, are they ethically challenged, morally corrupt? Some people died using their invention, some could have even have been intentionally killed or accidentally killed. Rogue governments could use the drugs to hurt or abuse people. I'd say no, the scientists (like engineers) aren't the problem, they did something that served some valid purpose but their invention (creativity) can be abused or misused.<p>Stingray devices, while I totally disagree with how they are used, I can see that validity for their use for some specific law enforcement cases. The problem to me isn't the technology, but the lack of ethics and morales in the people using them. Lying to the judges about the usage, lying about it's capabilities etc.<p>Hacking Team's surveillance software, having read just a little about it in the past. It appeared again that the software was being used appropriately for a genuine and valid purpose, at least at first. However, then it was sold to people who planned to mis-use it and to people who have a track record of abusing human rights. So is it the software's creation that is the problem or the assholes that sold it to dictators and abusers?<p>A syringe isn't an evil device, it is necessary for the medical community to do its' job, but it is also a device that can be used to kill, commit suicide, overdose on drugs etc. Is the person/people who invented it morally corrupt/ethically challenged, or does their invention just have a potential to be abused?
although i would personally consider all of the above unethical to some degree, i think i could understand someone thinking differently about Facebook Free Basics, Stingray, and Red Star. someone might put more weight to giving the poor any "internet", than giving them proper, neutral internet. Stingrays are used by legitimate law enforcement agencies, although i don't know much about how and under what conditions. and Red Star, well, all of that is legal there and not complying with the Kim can cost NK citizens their heads, so i'd cut a lot of slack there.<p>but investing your work in a company which sells surveillance equipment to dictators is downright disgusting. people are being tortured and killed because their governments managed to track them down thanks to that company. that seems like complete moral ambivalence at work. even if one could have somehow rationalized enabling their own government to spy, upon finding out who the company sells to, they must quit. and perhaps also leak everything.
I heard a rumour (well people were talking in abstract terms about "the secret project" so I'd say 90%+ true) that a certain french owned software consultancy was building systems for the Saudi government to monitor homosexuals and journalists. Mass surveillance FTW!
Anyone who worked to keep Tor secure from the US government.<p>Anyone who worked to aid the US government to circumvent Tor.<p>The problem with ethical actions is that it depends upon one's framework. Now there are areas we can tend to all agree on. Taking money to sabotage software to kill someone we agree is innocent is probably a good example. But once you get past the clear cut examples, it begins to be clear that ethical behavior is subjective and our subjective views do not always agree. For example, I can find people who would agree strongly with one of the above two that I listed while disagreeing strongly with the other, and I can find people who would do the opposite.
IBM & nazis: <a href="http://www.nytimes.com/books/first/b/black-ibm.html" rel="nofollow">http://www.nytimes.com/books/first/b/black-ibm.html</a>
There has been a conference by Bertrand Meyer on the topic but I don't know if there is a video of it : <a href="https://bertrandmeyer.com/2015/10/12/danke-sehr/" rel="nofollow">https://bertrandmeyer.com/2015/10/12/danke-sehr/</a>
don't you think that, especially in a large organization, the work can be partitioned in such a way that nobody knows they are working on something objectionable?<p>There are cases where it is obvious and people will quit over it, but if you are given the task, say, to change the firmware on this particular device to execute these external calls for conditions A, B, C, how do you know if this particular check is used for something you don't agree with?<p>What if somebody had you implement a bunch of logging and management interfaces telling you it's for QA when instead it is for mass surveillance? how are you going to know?<p>I think the ethics come more into play after the fact, if you find out that your company has done something you don't agree with (and you might have taken part in it without knowing), are you going to quit over it or not?
Zynga<p>And I would personally argue for Microsoft's <a href="https://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish" rel="nofollow">https://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish</a> program
What about people that work at knife manufacturers? A portion of their product will be used to kill/hurt someone.<p>Most engineers build something for a paycheck and the company paying them ends up using it for something unethical or illegal.
I've always felt a little strange about TEMPEST:<p><a href="https://en.wikipedia.org/wiki/Tempest_(codename)" rel="nofollow">https://en.wikipedia.org/wiki/Tempest_(codename)</a>
Relevant Defcon talk - My life as a spyware developer<p><a href="https://www.youtube.com/watch?v=k2mdUcOXW6I" rel="nofollow">https://www.youtube.com/watch?v=k2mdUcOXW6I</a>
<a href="http://www.wired.com/2010/09/google-spy/" rel="nofollow">http://www.wired.com/2010/09/google-spy/</a> like him?
slightly related, but check out consequences of an insightful algorithm - which speaks to the unintentional ethical failures we make as technology continues to progress and become more powerful.<p><a href="https://www.youtube.com/watch?v=NheE6udjfGI" rel="nofollow">https://www.youtube.com/watch?v=NheE6udjfGI</a>
Nobody mentions software freedom? [0] Arguably any software that restricts user freedom is the result of a moral failure somewhere in the process.<p>[0] <a href="https://en.wikipedia.org/wiki/Free_software" rel="nofollow">https://en.wikipedia.org/wiki/Free_software</a>
Like Patrick Naughton (Java creator) beeing a pedophile? But instead of 13-year-old girl it was an FBI agent.<p><a href="https://en.wikipedia.org/wiki/Patrick_Naughton#Sex_crime_arrest_and_conviction" rel="nofollow">https://en.wikipedia.org/wiki/Patrick_Naughton#Sex_crime_arr...</a>