Top 6 Open Source Sensitive Data Discovery Tools
The following tools are selected based on GitHub activity and sorted by GitHub star count in descending order. They cover the main use cases for sensitive data discovery: metadata cataloging with lineage, agentless scanning, and API-based detection of PII, PCI data, and credentials at rest.
One thing worth being upfront about before the list: these six tools do not all do the same thing, and some of the names on this list are not sensitive data discovery tools in the strict sense.
DataHub, Apache Atlas, and Marquez are metadata catalog and data lineage platforms; they help you understand what data you have and where it flows, which is a prerequisite for discovery but is not the same as scanning for PII. OpenDLP is the only tool here purpose-built for finding sensitive data at rest. Piiano Vault is a storage system for data you already know is sensitive. Nightfall is a commercial detection engine with open-source scanner scripts wrapped around it. Each fits a different stage of the data security problem.
Read more: Sensitive data discovery & classification tools, DLP software.
Administrative features
Tool | Graphical dashboard | Search-based | Data lineage | Federated database system |
|---|---|---|---|---|
DataHub | ✅ | ✅ | ✅ | ✅ |
Apache – Atlas | ✅ | ✅ | ✅ | ❌ |
Marquez | ✅ | ✅ | ✅ | Not shared. |
OpenDLP | ❌ | ❌ | ❌ | ❌ |
Piiano Vault – ReDiscovery | ❌ | Not shared. | ❌ | ❌ |
Nightfall AI – Sensitive data scanner | ✅ | ✅ | ❌ | ❌ |
Feature descriptions:
- Graphical dashboard – allows to visualize your data findings.
- Search-based functionality – allows searching for data assets.
- Data lineage – allows users to visualize how data is generated, transformed, transmitted, and used across a system over time.
- Federated database system – maps multiple autonomous database systems into a single federated database.
These functionality (especially data lineage and search capabilities) allow businesses to:
- Uncover the location of their personal information (PII), payment card industry (PCI) data, etc., stored across multiple databases, apps, and user endpoints.
- Comply with industry regulatory data protection and privacy standards such as General Data Protection Regulation (GDPR), and California Consumer Privacy Act (CCPA).
Data security features
Feature descriptions:
- Data masking– allows hiding data by modifying its original letters and numbers, so that it has no value to unauthorized intruders while remaining usable for authorized employees.
- Data loss prevention (DLP) – detects potential data breaches and prevents them by blocking sensitive data.
Categories and GitHub stars
Tool selection & sorting:
- Number of reviews: 10+ GitHub stars.
- Update release: At least one update was released last week as of November 2024.
- Sorting: Tools are sorted by GitHub stars in descending order.
DataHub
DataHub is an open-source metadata platform for data discovery, observability, and governance, originally built by LinkedIn and now maintained by Acryl Data as both an open-source project (Apache 2.0) and a commercial cloud offering. It is the most actively developed tool on this list by a wide margin: 3,000+ organizations run DataHub in production worldwide, including hyperscale tech companies, regulated financial institutions, and healthcare providers.
What DataHub actually does for sensitive data discovery
DataHub is a metadata catalog, not a scanner. It tells you what data assets exist across your stack, who owns them, how they flow through pipelines, and what their quality looks like but it does not scan file systems or databases for PII patterns the way OpenDLP does. The data discovery use case it solves is: “we have 200 data sources, and we do not know what is in them or who is using them.” The sensitive data discovery use case it does not solve is: “we need to find all files containing Social Security numbers across our Windows endpoints.”
Key features:
- Column-level data lineage: Traces data flow from source to consumption across platforms. New support in 2026 includes Snowflake dynamic tables, Sigma stitched back to source warehouses, Glue and Iceberg, Athena mapping to the catalog instead of raw S3 paths, and Power BI via Microsoft’s official M-Query parser.
- 90+ native connectors: Snowflake, BigQuery, Redshift, Hive, Athena, Postgres, MySQL, SQL Server, Looker, Power BI, Tableau, Okta, LDAP, S3, Delta Lake, and others. New connectors shipped in Q2 2026 include Airbyte, Matillion, dltHub, Informatica, and ThoughtSpot. In development: Monte Carlo, SAP Datasphere, AWS QuickSight, and streaming connectors for Firehose and Kinesis.
- MCP server support: Native support for AI agents via MCP, LLM integrations, and context management meaning AI assistants can query DataHub metadata directly without switching tools.
- Google Cloud deepened integration (April 2026): DataHub expanded its Google Cloud Knowledge Catalog connector to support Gemini Enterprise Agent Platform, Bigtable, Spanner, Pub/Sub, and Dataproc Metastore, alongside BigQuery and Google Cloud Storage.
Consideration: DataHub’s architecture runs several interconnected services. Production deployments typically require Kubernetes. Setup complexity is the most frequently cited pain point in the community, and it has not gotten simpler with the addition of AI features. If your team does not have Kubernetes experience, budget time for the learning curve before committing to a production rollout.
Apache – Atlas
Apache Atlas is an open-source metadata management and governance platform, designed primarily for Hadoop and big data ecosystems. It supports classification, lineage tracking, and search across data assets in environments built on Hive, HBase, Kafka, Spark, Sqoop, and Storm.
Key features
- Dynamic classification: Atlas allows creating custom classifications such as PII, EXPIRES_ON, DATA_QUALITY, and SENSITIVE. These can be applied at the entity or attribute level, which is useful for tagging sensitive columns in Hive tables without building a separate catalog.
- Metadata types: The platform provides pre-defined metadata types for Hadoop and non-Hadoop environments, covering HBase, Hive, Sqoop, Kafka, and Storm.
- SQL-like query language (DSL): Atlas supports a domain-specific language that provides SQL-like query functionality for searching entities. Useful for analysts familiar with SQL who need to query the metadata catalog without learning a new syntax.
- Integration with external tools: Apache Hive, Apache Spark, Kafka, and Presto, making it suited for big data environments.
Considerations:
- Configuring Atlas in a multi-cloud environment is complex, particularly when bridging AWS, Azure, and Databricks APIs. Atlas does not have native connectors for these platforms additional configuration is required to record lineage from AWS Redshift or Azure Synapse.
- Cloud-native cataloging services (e.g., AWS Glue) may offer lower-overhead lineage tracking for teams already committed to a single cloud provider.
- Atlas is best suited to organizations running Hadoop, Spark, and Hive at scale. Teams without a Hadoop-centric stack will find its architecture adds unnecessary complexity.
Add as preferred sourceMarquez
Marquez is an open-source metadata service for collecting, aggregating, and visualizing data ecosystem metadata. It was created at WeWork and open-sourced in 2019. Marquez is a graduation-stage project of the LF AI & Data Foundation, having graduated in September 2023.
What Marquez actually does
Marquez focuses purely on lineage tracking, not discovery. It is the reference implementation of the OpenLineage standard, the open specification for how pipelines report lineage metadata. If you use Apache Airflow, Apache Spark, Apache Flink, dbt, or Dagster, those tools can emit OpenLineage events that Marquez collects and visualizes. The result is a lineage graph showing how datasets flow through your pipelines, which jobs produced them, and what runs look like historically.
This makes Marquez useful for: understanding what breaks when an upstream dataset changes, debugging pipeline failures by tracing data provenance, and knowing which jobs consume a given dataset before you deprecate it. It is not a tool for scanning endpoints for PII.
Community activity note
As of early 2026, open issues on Marquez’s GitHub dating back to May 2025 remain unaddressed. Commit velocity has slowed compared to DataHub and OpenMetadata. If you need a fast-moving catalog with responsive maintainers, Marquez is the slower option. If you need a stable, lightweight lineage store that implements the OpenLineage standard without the infrastructure overhead of DataHub, Marquez still does that job well.
Key features:
- OpenLineage reference implementation: Works out of the box with every OpenLineage integration. This is Marquez’s strongest differentiator: no other tool on this list is the canonical implementation of the lineage standard.
- Lineage graph visualization: The web UI provides an interactive view of how datasets are connected and transformed through workflows. Useful for understanding pipeline dependencies and tracing errors.
- REST API and GraphQL: The lineage API allows automating tasks like backfills and root cause analysis by traversing the dependency tree programmatically. GraphQL endpoint is currently in beta.
- Low infrastructure overhead: Marquez runs on PostgreSQL and does not require Kafka, Elasticsearch, or a graph database, unlike DataHub or Atlas. For teams that want lineage tracking without a complex stack, that simplicity is the point.
Example workflow: To inspect lineage metadata, navigate to the Marquez UI and search for a job (e.g., etl_delivery_7_days) using the search box. From the job’s output dataset, you can view the dataset name, schema, description, and upstream inputs. The lineage graph shows everything that feeds into or depends on that dataset.
Piiano Vault – ReDiscovery
Piiano Vault is a privacy vault for storing and securing sensitive personal data within your own cloud environment. It is not a data discovery scanner; this distinction matters enough that it is worth stating plainly before describing what it does.
The use case Vault solves is: “we know which fields are sensitive (credit card numbers, SSNs, names, emails, phone numbers), and we want to move them out of our application databases into a centralized, access-controlled store.” Vault becomes the authoritative store for those fields. The application database holds a token or reference; the actual value lives in Vault. This is a data protection architecture pattern, not a discovery tool.
Vault deploys via Docker or Kubernetes (Helm charts available). SDKs exist for Python (Django ORM), TypeScript, Java, and Go. The vault-releases repository was last updated in August 2025.
Nightfall
Nightfall is a commercial AI-native DLP platform. Its GitHub repositories include open-source scanner scripts (Apache 2.0) that wrap Nightfall’s commercial detection API. This distinction matters for the “open-source” framing: the scanner scripts are open source, but the detection engine that actually identifies PII, credentials, and payment data is Nightfall’s proprietary service. Executing scans requires a Nightfall API key and calls Nightfall’s commercial API. The free tier allows up to 100 scans per month on public and private repositories.
This is the most capable tool on this list for detecting sensitive data in modern environments. Nightfall covers GitHub repositories, S3 buckets, Salesforce exports, and similar sources, but it is not fully open source. If your requirement is zero dependency on a commercial vendor, Nightfall does not meet it.
Open-source scanner capabilities (free tier):
- Scans the full commit history of public and private repositories.
- Detects credentials, secrets, PII, and credit card numbers using Nightfall’s ML detection models.
- Runs up to 100 scans per month at no cost.
- Sends alerts to Slack when violations are detected.
- Pushes results to a SIEM, reporting tool, or webhook endpoint.
Distinct feature: Nightfall can send alerts to Slack when violations are detected and push results to a SIEM, reporting tool, or webhook endpoint.
Example use case: Scan a Salesforce backup to detect sensitive data at rest. The scanner (1) submits backup files to Nightfall’s API for scanning, (2) runs a local webhook server to receive results, and (3) exports findings to a CSV file.
The above URL is provided by Nightfall. It is the temporarily signed S3 URL to retrieve the sensitive findings that Nightfall identified.
Further reading
Cite this research
Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.
@misc{dilmegani2026,
author = {Dilmegani, Cem and Sezer, Sena},
title = {{Top 6 Open Source Sensitive Data Discovery Tools }},
year = {2026},
month = jun,
howpublished = {\url{https://aimultiple.com/open-source-sensitive-data-discovery}},
note = {AIMultiple. Retrieved June 24, 2026}
}Cem's work has been cited by leading global publications including Business Insider, Forbes, Washington Post, global firms like Deloitte, HPE and NGOs like World Economic Forum and supranational organizations like European Commission. You can see more reputable companies and resources that referenced AIMultiple.
Throughout his career, Cem served as a tech consultant, tech buyer and tech entrepreneur. He advised enterprises on their technology decisions at McKinsey & Company and Altman Solon for more than a decade. He also published a McKinsey report on digitalization.
He led technology strategy and procurement of a telco while reporting to the CEO. He has also led commercial growth of deep tech company Hypatos that reached a 7 digit annual recurring revenue and a 9 digit valuation from 0 within 2 years. Cem's work in Hypatos was covered by leading technology publications like TechCrunch and Business Insider.
Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
Be the first to comment
Your email address will not be published. All fields are required. Comments are left in their original language.