If you have been following recent AI news, you have likely seen reports about OpenAI’s recent security incident. For many business leaders, this understandably raises questions. If one of the world’s leading AI companies experienced a cybersecurity issue, what does that mean for organizations looking to implement AI themselves?
However, the incident should not discourage organizations from adopting AI. As stated in a recently published Times article, the breach happened because the model wasn’t being actively monitored for cybersecurity concerns, and clear guardrails weren’t set in place.
For organizations considering local large language models (LLMs), this serves as a valuable reminder that AI should be treated like any other business-critical application. Local LLMs provide significantly greater control over sensitive information than cloud-hosted alternatives. However, they still require thoughtful security planning.
Local LLMs Still Offer Significant Security Advantages
Often, organizations explore local LLMs because sensitive information remains inside their own environment. Rather than transmitting confidential business information to a third-party cloud provider, the model operates within the organization’s existing security infrastructure.
For businesses working with intellectual property, financial information, customer records, or other confidential documents, local LLMs can substantially reduce security concerns. However, local deployment does not eliminate every potential risk. Organizations should regularly, monitor, update, and review AI systems for vulnerabilities. As organizations continue integrating AI into their daily operations, cybersecurity should remain part of the overall implementation strategy, not an afterthought.
What Can Organizations Learn?
Rather than viewing this incident as a reason to avoid AI, organizations can use it as an opportunity to strengthen their own governance practices. Several best practices reduce risk:
- Have cybersecurity specialists regularly evaluate your AI environment. Conduct routine vulnerability assessments, log reviews, and security testing.
- Establish organization-wide best practices for prompting. Employees often focus on obtaining the best response from AI. However, they should also understand how to interact with AI securely. For example, rather than allowing an AI system to search the internet for information, employees may instead copy-paste only the specific documents or sections that require analysis. When using local LLMs, organizations should limit internet and other external access whenever possible.
- Separate AI systems based on departments or security clearances. Human Resources, Finance, Engineering, and Executive Leadership often work with different types of confidential information. Apply traditional segregation-of-duties principles to local LLMs.
- Establish clear data handling policies within the AI environment itself. Organizations should flag highly sensitive company information and configure the system so that this information cannot be shared outside authorized contexts—even if a poorly written prompt appears to request it. While no safeguard should replace proper access controls, layered security reduces the likelihood of accidental disclosure.
- Ensure employees know when to use a cloud LLM, and when to use a local LLM. Cloud LLMs may work well for drafting public-facing content, brainstorming ideas, or summarizing publicly available information. Sensitive company information, however, should remain within approved local AI environments. Some organizations may even choose to display reminders or policy acknowledgments before employees access cloud AI tools.
AI Security Is Ultimately About Governance
Like many emerging technologies, AI presents both opportunities and new responsibilities. The recent OpenAI incident demonstrates that organizations cannot assume AI will remain secure without ongoing oversight. At the same time, organizations should not avoid implementing AI altogether.
Fortunately, organizations already have many of the tools needed to address these risks. Strong cybersecurity monitoring, clearly defined data handling policies, employee training, and appropriate access controls have long been considered cybersecurity best practices. AI simply provides another area where these principles should be applied.
As more organizations adopt local LLMs, those that invest in governance alongside technology will likely be in the strongest position to benefit from AI while continuing to protect their most valuable information.
This is part of our series on practical application of AI in the enterprise. For more insight, take a look at our other articles:
Professional Services: The End of the Billable Hour Pyramid
The AI Industrial Shift: The White Collar Divide Is Spreading
Leave A Comment