In April 2023, Samsung lifted its internal ban on ChatGPT. Twenty days later, engineers had leaked confidential data on three separate occasions. One employee pasted faulty source code from a facility measurement database seeking a fix. Another uploaded program code for identifying defective equipment wanting optimization. A third converted a recording of a company meeting to text and fed it to ChatGPT to generate meeting minutes.
Samsung's semiconductor division had just handed its proprietary code and internal discussions to OpenAI's servers. The company responded by reimposing the ban, limiting prompts to 1024 bytes, and beginning development of an internal AI system. But the damage was done. The data had been transmitted. And depending on OpenAI's settings at the time, it may have been incorporated into the model's training data.
This is the reality every organization faces. ChatGPT offers genuine productivity gains. The risk is equally real. And most people have no idea what actually happens to the documents and data they share.
The short version: If you need to redact sensitive documents before they reach AI systems, PaperVeil handles that layer. The rest of this article explains where it fits in the broader governance architecture.
Where Your Data Actually Goes
When you paste text into ChatGPT or upload a document, that data travels to OpenAI's servers in the United States. What happens next depends on which tier you're using and which settings you've configured.
Consumer tiers (Free, Plus, Pro): By default, OpenAI uses your conversations to train future models. Your prompts, the documents you upload, and ChatGPT's responses all become potential training material. This is how OpenAI improves its models. It's also how your confidential information could theoretically surface in responses to other users.
Business tiers (Enterprise, Business, Edu, API): Data is excluded from model training by default. The organization owns and controls its inputs and outputs. But the data still transmits to OpenAI's infrastructure.
For consumer users, standard chat history is retained indefinitely unless you actively delete conversations. Once deleted, chats are purged within 30 days. For Temporary Chat mode, conversations are automatically deleted within 30 days regardless of user action.
Files follow similar rules. Uploads are retained as long as the conversation exists. Delete the chat, and files are removed within 30 days. None of these files are used for training unless explicitly permitted.
The Training Toggle Myth
You've probably seen the setting: "Improve the model for everyone." Toggle it off, and you've protected your data, right?
Not quite.
The toggle prevents your conversations from being used to train future models. It does nothing to prevent the transmission of your data to OpenAI's servers. It doesn't change the 30-day retention period for abuse monitoring. And it doesn't affect the fundamental architecture where your prompts and documents travel to external infrastructure you don't control.
Here's what actually happens when you disable training:
- Your data still transmits to OpenAI's servers
- OpenAI still retains the data for up to 30 days
- OpenAI employees may still access data for policy compliance reviews
- The data still exists outside your security perimeter
The training toggle is a meaningful privacy control. It's not a security solution. For organizations handling sensitive data, the distinction matters enormously.
A 2025 Stanford study put it bluntly. When asked if users should be worried about privacy, researcher Jennifer King said "Absolutely yes." The study found that sensitive information shared in dialogues with ChatGPT, Gemini, and other frontier models may be collected and used for training, even when uploaded in separate files during conversations.
The Actual Risks, Ranked
Not all privacy risks are equal. Here's what actually threatens organizations using ChatGPT, from most to least common:
1. Inadvertent data exposure (Most Common) Employees paste sensitive information without realizing the implications. This is the Samsung scenario. Customer data, source code, strategic plans, HR records: all transmitted to external servers by well-meaning staff trying to be productive. This happens constantly, in every organization, often without anyone knowing.
2. Retention beyond organizational control Once data transmits to OpenAI, you've lost control over its lifecycle. Even with training disabled, data persists for 30 days minimum. In May 2025, a US Magistrate Judge ordered OpenAI to preserve all ChatGPT conversation logs indefinitely for legal discovery in the New York Times lawsuit. Your data could be caught in similar preservation orders.
3. Training data incorporation If training isn't disabled, your inputs may become part of the model's knowledge. While OpenAI states this data is used responsibly, the theoretical risk exists that patterns from your data could surface in responses to others.
4. Breach of OpenAI's infrastructure (Rare but high impact) No system is invulnerable. A breach of OpenAI's servers could expose retained conversation data. The attack surface is massive given ChatGPT's user base.
5. Internal OpenAI access Terms permit data access for policy compliance, safety research, and legal obligations. This access is necessary for running the service but creates scenarios where your data could be viewed by humans you've never authorized.
What Data Types Are Most At Risk
Understanding what's at stake helps prioritize protection. These categories routinely flow through ChatGPT in organizations that haven't implemented safeguards:
Customer PII: Names, addresses, phone numbers, email addresses. Every customer service interaction, every support ticket summary, every CRM data export. The Samsung incident involved internal data, but the more common exposure involves customer information.
Financial data: Account numbers, transaction details, credit card data, salary information. Finance teams using ChatGPT to draft reports or analyze trends may inadvertently share protected financial information.
Source code and trade secrets: Developers asking ChatGPT for code reviews or debugging help. This was Samsung's specific exposure. Proprietary algorithms, security implementations, and competitive advantages transmitted to external infrastructure.
Strategic documents: Board presentations, M&A plans, product roadmaps. Executives using ChatGPT to refine language or structure inadvertently share strategy with a third party.
Employee records: Performance reviews, compensation data, disciplinary actions. HR teams using AI to draft communications may expose protected employment information.
Healthcare information: Patient records, diagnoses, treatment plans. Healthcare organizations face HIPAA implications on top of general privacy concerns.
Each category has different regulatory implications. Some carry immediate legal liability (PHI under HIPAA, financial data under SOX/GLBA). Others carry competitive risk (source code, strategy). All deserve protection.
Why Private AI Isn't the Answer
The obvious solution seems to be running your own AI. Deploy an open-source model on your infrastructure. Keep everything internal. Problem solved.
Except the economics don't work for most organizations.
Running a capable language model requires significant GPU infrastructure. We're talking about tens of thousands of dollars monthly for the compute to run models that approach GPT-4's capabilities. You need ML engineering talent to manage the deployment. You need security staff to protect the infrastructure. You need ongoing maintenance as models improve and vulnerabilities emerge.
For enterprises with dedicated AI teams and existing GPU clusters, private deployment makes sense. For a law firm with twenty attorneys or a healthcare practice with fifty employees, the cost is prohibitive. They need AI productivity gains. They can't spend six figures annually on infrastructure.
And even with private deployment, you still face the fundamental problem: sensitive data in documents. Whether you're sending data to OpenAI or processing it on your own servers, the challenge remains: how do you get AI assistance on documents that contain information that shouldn't be processed by AI at all?
The Approach That Actually Works
The solution isn't choosing between AI productivity and data protection. It's removing the conflict entirely.
If ChatGPT never sees the sensitive data, there's nothing to train on, nothing to retain, nothing to protect.
This is the redaction-first approach. Before any document touches AI (whether ChatGPT, Claude, Gemini, or your private deployment), strip the identifying information:
Original document:
"This agreement between ABC Corporation (EIN 12-3456789) and John Smith (SSN xxx-xx-5678) for services at 123 Main Street, San Francisco, CA 94102. Contact: [email protected], (415) 555-1234."
After redaction:
"This agreement between [COMPANY] (EIN [REDACTED]) and [NAME] ([REDACTED]) for services at [ADDRESS]. Contact: [EMAIL], [PHONE]."
ChatGPT processes the redacted version. You get AI assistance with the structure, analysis, or drafting. You reinsert the specific information after reviewing the output. The sensitive data never leaves your environment.
This works regardless of which AI you use. It works regardless of their privacy policy changes. It works regardless of training settings or retention periods. You've eliminated the risk at its source.
Building the Workflow
Implementing redaction-first AI processing requires a few components:
Detection Layer
You need software that reliably identifies sensitive information in unstructured text. This means:
- Named Entity Recognition for names, organizations, locations
- Pattern matching for structured data (SSNs, EINs, phone numbers, emails)
- Context-aware detection (medical terms, legal identifiers, financial data)
- Support for PDFs and common document formats
Manual review doesn't scale. You need automation that catches what humans miss.
Redaction Layer
Once detected, sensitive data needs consistent replacement:
- Replace identified entities with category placeholders ([NAME], [SSN], [ADDRESS])
- Maintain document structure so AI can process effectively
- Generate mapping files to reinsert data after AI processing
- Create audit trails proving what was redacted
Integration Points
For organizational deployment, this needs to fit existing workflows:
- Browser extension or desktop app for individual use
- API integration for automated pipelines
- Batch processing for document review projects
- Export formats compatible with downstream systems
Audit Trail
For compliance purposes (and your own peace of mind):
- Log what documents were processed
- Record what entities were detected and redacted
- Track who processed what, when
- Store redaction certificates for regulatory requirements
For Enterprise Teams
If you're evaluating ChatGPT Enterprise or similar business tiers, understand what you're getting:
ChatGPT Enterprise provides:
- Data excluded from training by default
- SOC 2 Type 2 compliance
- AES-256 encryption at rest
- TLS 1.2+ in transit
- Enterprise Key Management (customer-controlled encryption keys)
- Admin-controlled retention policies
- HIPAA BAA availability
- GDPR and CCPA alignment
ChatGPT Enterprise doesn't solve:
- The fundamental architecture of sending data to external servers
- The 30-day minimum retention for abuse monitoring
- Access by OpenAI personnel for policy compliance
- The risk profile of any cloud service you don't control
Enterprise tiers provide better controls. They don't eliminate the need for thoughtful data handling. For truly sensitive documents, redaction before processing remains the safest approach even with enterprise protections.
The Bottom Line
ChatGPT's data handling isn't mysterious. Consumer tiers use your data for training by default. All tiers transmit and retain data on OpenAI's infrastructure. The training toggle helps but doesn't protect you from transmission and retention.
For organizations handling sensitive data, the path forward is:
- Understand the tiers. Consumer ChatGPT (Free, Plus, Pro) has different rules than Business tiers
- Don't trust toggles alone. Disabling training doesn't stop transmission
- Evaluate enterprise options. If budget allows, business tiers provide meaningful additional controls
- Implement redaction workflows. Strip sensitive data before AI processing
- Train your staff. The Samsung incident happened despite policies
- Audit usage. Know what data is being processed and by whom
The productivity benefits of AI are real. Samsung engineers used ChatGPT because it helped them work faster. That motivation isn't going away. The question is whether you're managing the risk or ignoring it.
PaperVeil lets you redact sensitive information from documents in a simple drag and drop flow. Detect and remove PII, match custom patterns, strip metadata, and generate audit trails. The redaction layer that makes AI document processing actually safe.