[5] Concerns have been raised around how biases can be integrated into algorithm design resulting in systematic oppression[6]whether consciously or unconsciously.
A notable example of this is pulse oximetry, which has shown reduced reliability for certain demographic groups due to a lack of sufficient testing or information on these populations.
[7] Additionally, many algorithms are designed to maximize specific metrics, such as engagement or profit, without adequately considering ethical implications.
[12] For example, the sharing of healthcare data can shed light on the causes of diseases, the effects of treatments, an can allow for tailored analyses based on individuals' needs.
For example, the NSA’s collection of metadata exposed in global surveillance disclosures raised concerns about whether privacy was adequately protected, even when the content of communications was not analyzed.
The right to privacy is often complicated by legal frameworks that grant governments broad authority over data collection for “national security” purposes.
Historically, activists involved in the Civil rights movement were frequently targets of government surveillance as they were perceived as subversive elements.
[16] Marginalized communities often lack the resources or education needed to challenge these privacy violations or protect their data from nonconsensual use.
Furthermore, there is a psychological toll, known as the “chilling effect,” where the constant awareness of being surveilled disproportionately impacts communities already facing societal discrimination.
[9] In the European Union, the right to be forgotten entitles EU countries to force the removal or de-linking of personal data from databases at an individual's request if the information is deemed irrelevant or out of date.
[17] According to Andrew Hoskins, this law demonstrates the moral panic of EU members over the perceived loss of privacy and the ability to govern personal data in the digital age.
The case revealed that governments controlled and possessed far more information about civilians than previously understood, violating the principle of ownership, particularly in ways that disproportionately affected disadvantaged communities.
For instance, activists were frequently targeted, including members of movements such as Occupy Wall Street and Black Lives Matter.
[14] This revelation prompted governments and organizations to revisit data collection and storage practices to better protect individual privacy while also addressing national security concerns.
The case also exposed widespread online surveillance of other countries and their citizens, raising important questions about data sovereignty and ownership.
Data was harvested from approximately 87 million Facebook users without their explicit consent and used to display targeted political advertisements.
In some cases, the political ads spread misinformation,[27] often disproportionately targeting disadvantaged groups and contributing to knowledge gaps.
In contrast, users with more resources or digital literacy could better safeguard their data, exacerbating existing power imbalances.