A large medical center is considering AI integration for their current processes. They may see potential applications across clinical care, research, administration, health information management, and IT. However, with so many departments, technologies, and possible use cases to consider, leadership may struggle to determine where to begin and how to prepare employees with different levels of technical experience.
A large medical center is considering AI integration for their current processes. They may see potential applications across clinical care, research, administration, health information management, and IT. However, with so many departments, technologies, and possible use cases to consider, leadership may struggle to determine where to begin and how to prepare employees with different levels of technical experience.As organizations move into planning AI architecture, the number of available technologies can quickly become overwhelming. One team may recommend retrieval-augmented generation, or RAG. Another may propose semantic search. Yet another may suggest Model Context Protocol servers, while another brings up reusable AI skills.
At first, these options may appear to compete with one another. However, they generally address different parts of an enterprise AI system:
- Semantic search helps AI find relevant information.
- RAG helps AI use that information to prepare an answer.
- MCP connects AI applications to external data, tools, and workflows.
- Skills guide AI through repeatable processes.
Rather than asking which technology will win, CEOs should take a step back and strategize. This way, they can consider which combination supports the organization’s goals, risk level, data environment, and existing systems.
Begin With the Business Problem
Before selecting an AI architecture, begin with the outcome employees need to achieve. In most cases, enterprise AI use cases fall into three broad levels:
- On the first level, employees may simply need to find information, such as a maintenance procedure, quality document, product specification, or company policy.
- Alternatively, they may need AI to interpret that information by summarizing documents, comparing requirements, or explaining technical material.
- Finally, employees may need AI to interact with SAP or another operational system as part of a controlled workflow.
Each level introduces additional complexity. For instance, a search tool that retrieves approved procedures presents different risks from an AI agent that can update a supplier record or initiate a business transaction. As the system moves closer to taking action, identity management, authorization, monitoring, and human approval become increasingly important.
RAG and Semantic Search Serve Different Purposes
Semantic search attempts to understand the meaning and intent behind a query instead of relying entirely on exact keywords. For example, a manufacturing employee may describe an equipment symptom without knowing the terminology used in the official maintenance manual. Semantic search can help locate related procedures based on meaning.
On the other hand, RAG builds on retrieval by supplying relevant information to a large language model. The model can then use that information to prepare a contextual response. In this example, a RAG system might summarize the retrieved maintenance procedures and provide citations or links to the original documents.
Therefore, organizations do not always need to choose between semantic search and RAG. Semantic search can serve as part of the retrieval process within a larger RAG system.
A similar AI architecture could apply in life sciences. For instance, semantic search may locate relevant controlled documents, while RAG compares or summarizes those documents for an authorized employee. However, the employee should still review the original approved sources before making a regulated or consequential decision.
RAG Does Not Guarantee Accuracy
While RAG can improve the grounding of an AI response, it does not guarantee that the response will be correct. For instance, if the system retrieves irrelevant, incomplete, outdated, or contradictory information, the resulting answer may also be unreliable.
TechTarget identifies the following as primary challenges of RAG:
- Retrieval problems
- Remaining hallucination risk
- Latency
- Debugging difficulty
- Infrastructure complexity
- Performance monitoring
- Data security
Each stage of the pipeline, including embedding, searching, reranking, and preparing context for the model, can introduce additional time, cost, and opportunities for error.
Document permissions require particular attention. If an employee is not authorized to view a document, the retrieval system should not expose its contents through an AI-generated summary. Organizations must also prepare for prompt-injection attempts hidden within user requests or retrieved material.
The technology ecosystem itself can add complexity. The RAG Technology Index organizes more than 100 libraries, frameworks, and tools across functions such as document processing, embedding, storage, retrieval, reranking, orchestration, and evaluation. This range illustrates why organizations should select components around a defined use case, rather than assembling a large technology stack simply because the tools are available.
RAG Retrieves Information While MCP Creates Connections
RAG and MCP are also frequently compared, but they are not direct substitutes. Namely, RAG primarily helps an AI system retrieve and use relevant knowledge. Meanwhile, MCP is an open standard for connecting AI applications to external systems. According to the official MCP documentation, these connections can include data sources, tools, and workflows.
Consider a procurement use case involving SAP. A RAG system might retrieve the organization’s purchase-order approval policy. Meanwhile, an MCP-connected tool could retrieve the current purchase order or supplier record from an authorized system. The AI could compare the transaction with the policy and prepare a recommendation for an employee.
However, reading a purchase order and changing one are not equivalent. Because of this, organizations should clearly distinguish between read-only connections and action-enabled tools. Any connection to an operational system should include appropriate authentication, authorization, logging, data boundaries, and human approval gates.
MCP also expands the attack surface of an AI environment. Recent research into the MCP landscape and its security risks emphasizes the need to evaluate threats “across four major attacker types: malicious developers, external attackers, malicious users, and security flaws…”. Some risks include the following:
- No Central Authority for Security
- No Framework to Authenticate
- Limited Debugging
- Challenging Design to Keep Consistent Workflow
- Difficulty Scaling
An MCP server should be treated as an enterprise integration, not as an informal chatbot extension. For more detailed recommendations on secure creation, deployment, operation, and maintenance, we recommend that you take a look at Section 6.3 of the MCP research.
MCP and Skills Work Together
At an executive level, the distinction is relatively straightforward:
MCP gives the AI access to capabilities. On the other hand, Skills tell it how and when to use those capabilities.
A skill is a reusable package of instructions and supporting resources for completing a particular type of work. Instead of placing every possible procedure into one large prompt, the AI can load the relevant skill when the task requires it.
For example, MCP could provide controlled access to approved quality records. A quality-review skill could instruct the AI to identify the appropriate record, follow the organization’s review procedure, cite its sources, document uncertainties, and request human approval before proceeding.
Google Cloud’s explanation of agent skills describes skills as reusable instructions and context that can be loaded progressively. It also characterizes skills and MCP as complementary, with MCP connecting an agent to live external data and skills providing procedural guidance for using those connections.
Like integrations, skills require governance. Instructions, reference files, and executable scripts should be reviewed, version-controlled, tested, and limited to approved sources.
Build the AI Architecture in Controlled Layers
Organizations can reduce risk by implementing these technologies in layers:
- Search: Confirm that employees can reliably locate approved information.
- Generate: Add RAG when employees need grounded summaries, comparisons, or explanations.
- Connect: Add MCP or another controlled integration when AI must access live systems or tools.
- Standardize: Develop skills for repeatable, organization-specific processes.
- Govern: Apply permissions, validation, monitoring, audit records, and human approval throughout the system.
A bounded pilot might focus on searching maintenance documentation, summarizing validated procedures, reviewing SAP support tickets against approved resolution documents, or preparing procurement recommendations without posting them.
Before investing, leaders should ask which sources are authoritative, who maintains them, whether AI will inherit user permissions, which actions require human approval, and how the organization will measure retrieval and response quality. They should also determine whether the system can show which sources and tools influenced its output.
Let’s return to our medical center analogy. A recent hospital training program demonstrates why employee preparation should accompany this technical progression. In a 2026 JMIR Medical Education study, Asan Medical Center provided an eight-week, 56-hour generative AI program to employees across clinical, research, administrative, health information management, and IT roles. Participants progressed from foundational concepts to RAG, MCP, orchestration, and AI agent design before completing “team-based capstone projects.”
The program dedicated 71% of its on-site instructional time to hands-on practice and provided each team with funded AI tools, an internal cloud platform, and three to six hours of weekly mentoring. “11 of 12 teams presented functional prototypes, and one has since entered active pilot use in clinical departments ahead of planned hospital-wide deployment.” Notably, non-IT professionals generally reported benefiting more from this project than IT specialists, suggesting that advanced AI adoption does not need to remain entirely within the technology department.
However, the study also identified a continuing gap between understanding the technology and applying it within a specific job. This finding reinforces the importance of ongoing support after formal training. A pilot should not only test whether the AI architecture works. It should also help employees determine where the technology fits within their responsibilities, which safeguards apply, and when human judgment must remain in control.
Ultimately, semantic search, RAG, MCP, and skills address different parts of the enterprise AI workflow. However, the strongest AI architecture is not necessarily the one with the most tools. It is the one designed around a measurable business outcome, supported by reliable information, and controlled according to the risk involved. Before committing to a technology stack, leaders should assess their data, use cases, SAP environment, integration requirements, and governance readiness.
Stay tuned! In our next article, we will dive into how to integrate SAP systems and data with your AI workflow. In the meantime, feel free to take a look at our other articles:
Building the Connections for Enterprise AI
Is Your AI Infrastructure Ready to Launch?
Leave A Comment