In an era in which data drives decision-making, business and institutional leaders must grapple with significant ethical challenges regarding data collection. The enormous volumes of data gathered from consumers and stakeholders offer unparalleled opportunities for growth and innovation, yet they also pose risks around privacy, consent, and bias.
The growing use of artificial intelligence (AI), particularly in decision-making processes, only compounds concerns. A Pew Research Center study found that 81 percent of U.S. adults believe that AI adoption will result in misuse of their personal information.
Leaders must ensure that data practices are transparent and equitable, balancing the pursuit of institutional objectives with the responsibility to protect individuals’ rights. Navigating these ethical complexities is essential for fostering trust, complying with regulations, and maintaining a positive organizational reputation in a data-driven world.
In today’s business climate, ethical leadership in data collection is no longer an option; it’s a crucial imperative for building consumer trust and adhering to legislative requirements. This article outlines the key considerations for leaders surrounding ethical data collection. It also explores how earning an online Doctorate in Education in Leadership and Innovation from the NYU Steinhardt School of Culture, Education, and Human Development can help prepare leaders to navigate the challenging data privacy and ethics landscape.
Elevate Your Leadership Skills With NYU Steinhardt’s EdD Program
Prepare to Lead with Integrity in the Digital Age
Why Leaders Must Prioritize Ethics in Data Collection
Institutional and business leaders must promote public trust by safeguarding their organizations’ reputations as fair employers, producers of exceptional products and services, and ethical stewards of information. The public seeks assurances that sensitive data will not be used maliciously or exploited without regard for consumers’ privacy.
That is why leaders must uphold ethical standards in data collection by fostering a culture of accountability, transparency, and respect for privacy within their organizations. They must ensure that ethical policies are established, encourage responsible decision-making, and hold their teams accountable for adhering to fair and secure data practices.
Failure to adequately address data privacy risks and eradicate potentially unethical data practices can have dire consequences. Not only can a business suffer legal, reputational, and operational consequences in the case of a data ethics failure, but board members can also be held personally liable for cybersecurity failures.
Failures can be catastrophic. In 2024, National Public Data, a data aggregator that supplies information for background checks, suffered a data breach resulting in the compromise of nearly 3 billion records that included Social Security numbers, names, addresses, email addresses, and phone numbers. Affected individuals faced risks of identity theft and fraud, eroding public trust in a major institution’s ability to safeguard data; the fact that impacted users only learned of the breach when a class action lawsuit was filed further exacerbated the public relations disaster. The incident underscored the critical need for robust data protection strategies and emphasized the importance of transparency and accountability in handling personal information.
Key Considerations in Data Collection
When using, collecting, and transmitting data—or guiding teams that do—leaders across sectors must be mindful of issues such as informed consent, bias, data privacy, and transparency.
Informed Consent and Data Ownership
Informed consent is the process by which individuals are fully informed about how their personal information will be collected, used, stored, and shared before they agree to such practices. Informed consent is a critical component of data collection transparency and is crucial for building trust and upholding ethical standards. It is essential for empowering individuals to make educated decisions about their data privacy and is a cornerstone of ethical data governance. It also typically includes the right for individuals to withdraw their consent at any time, ensuring they retain control of their personal information.
Obtaining meaningful consent can be challenging, especially in the case of mobile apps and IoT devices, as users tend to skip past privacy policies without reading or fully understanding them. For example, a fitness tracker might collect location and health data, but if the consent process is buried in fine print or presented in vague terms, users may not realize the full extent of data sharing with third parties, limiting their ability to make an informed choice.
Organizations can help individuals better understand how their data is used by providing clear, concise, and accessible privacy policies that use simple language instead of legal jargon. Companies can also consider using infographics or videos to explain data use. Furthermore, businesses must make it easy for users to update their privacy settings and withdraw consent if they so choose.
Bias and Fairness in Data-Driven Decision-Making
AI models are trained on data sets to recognize patterns and make decisions. If an AI model is trained on biased data (e.g., sets reflecting historical or representational bias), the system will most likely provide discriminatory outputs.
For example, AI-driven facial recognition technology often performs worse on people of color because darker-skinned faces were underrepresented in the data training set. When used by law enforcement agencies or to protect individual rights, this bias can have dire consequences for individuals.
The NYU Tandon School of Engineering is working to address this challenge by training AI facial recognition models on balanced datasets and researching the outcomes. NYU researchers have open-sourced their code to enable others to reproduce and build upon their work, developing unbiased, high-accuracy facial recognition and analysis capabilities.
Business leaders must be aware of the impact of using AI systems trained on biased data and work to source unbiased and fair technology to prevent discriminatory outcomes.
Data Privacy and Security
Business leaders must guide their organizations to protect consumer and employee data by overseeing the implementation of strong security measures and ensuring regulatory compliance. Key data privacy regulations include the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA).
GDPR applies to all organizations processing the personal data of EU residents, regardless of the company’s location. It requires organizations to ensure robust data protection practices and maintain detailed records of data processes. Non-compliance may result in substantial fines (up to €20 million or 4 percent of global turnover) for the business involved.
The CCPA applies to businesses operating in California and those handling the personal data of California residents. CCPA compliance involves providing individuals with options for how their personal data may be used and the right to opt out of data processing and sales. CCPA penalties for non-compliance range from $2,500 for an unintentional breach to $7,500 for an intentional violation. It’s important to note that these penalties apply to a single offense, such as a data breach involving one person; when a large amount of data is compromised, the financial impact can be substantial.
Transparency and Accountability in AI Systems
As AI tools develop in power and sophistication, “black box” systems grow increasingly common. Black box AI refers to artificial intelligence systems whose internal decision-making processes are not transparent or easily understood by users, often making it difficult to interpret how inputs are converted into outputs.
Some AI models are built to be black boxes, typically to protect intellectual property. In these cases, the AI developers will know how the system works but choose to keep the source code and decision-making process a secret. However, most advanced AI tools, including generative AI, utilize “organic black boxes.” This means that these tools’ creators don’t intentionally obscure their operation. Instead, the deep learning systems that power these tools are so complex that even the creators don’t fully understand how they work.
Black box models pose significant ethical challenges to companies that use them, specifically regarding bias and fairness. The nature of these systems makes it especially difficult to pinpoint the existence of bias and its cause. Black box AI also makes it difficult for an organization to determine whether it complies with data protection regulations like GDPR and CCPA.
These ethical challenges are driving a shift toward more transparent and explainable AI (XAI) systems. With explainable AI, organizations can access the technology’s underlying decision-making process, enabling the company to ensure ethical data use.
Demand is also growing for the ethical auditing of AI systems, which is considered a key mechanism for AI governance. While an audit of a black-box system is possible, white-box access (when the auditor has access to the system’s inner workings) allows for a more thorough assessment of the system in question. This provides actionable insights that allows businesses to fine-tune the technology to address concerns.
Ethics in Data Collection and NYU’s EdD in Leadership and Innovation
NYU Steinhardt’s online EdD in Leadership and Innovation is designed for leaders in all industries who are motivated to create change through education and learning. The program examines – and equips students to navigate – the most pressing issues cross-sectoral leaders face, including ethical dilemmas related to data collection and AI. For example, in the course Management and Ethics of Data, students focus on recognizing the person behind the numbers. Coursework covers research and applied ethics, concepts of privacy and publicity, issues related to data collection and data mining, and the lifecycle of data.
NYU Steinhardt’s online Doctorate in Leadership and Innovation also empowers leaders to tackle the challenge of innovating to enact change. The program fosters a problem-solving mentality throughout its coursework and problem-of-practice project, where students work on solutions to real-world challenges in their organizations.
The Role of Ethical Leaders in the AI Age
As data continues to shape decision-making across industries, leaders must actively ensure ethical and responsible practices. They must enable informed consent, address issues related to bias and fairness, and ensure compliance with data privacy regulations and best practices.
Ultimately, responsible data use isn’t just a technical issue—it’s a leadership imperative. Proactive, informed leaders set the tone for ethical data practices, shaping policies that prioritize fairness, privacy, and accountability. By championing responsible AI and data governance, leaders can build trust, drive innovation, and ensure their organizations navigate the digital future with integrity.
If you’re ready to take the next step in your leadership career, connect with an enrollment advisor to learn more about the NYU Steinhardt online EdD in Leadership and Innovation. Alternatively, start your application today, if you’re ready.