Like an undergraduate college student drowning in assignments, organizations often feel like they have a lot to juggle. Both may choose an approach of prioritizing productivity — whether that is clearing through an incomprehensible number of projects and essays over a painfully short weekend, or tackling an impossibly high number of project items and reports before a relentlessly approaching deadline. In many cases, employees and students alike gravitate to AI.
However, another concern deserves equal attention. As employees become more productive with AI, are they continuing to develop the skills and judgment required to do their work?
AI can help employees research topics, summarize information, write content, analyze data, and develop potential solutions. Used appropriately, it can give people more time to focus on valuable work.
Used without enough thought or oversight however, it can also encourage employees to accept convenient answers without fully understanding them.
Because of this, leaders need to create an environment where AI supports employee expertise without gradually replacing it.
Looking Deeper into the Siemens Case Study
In our previous article, we explored how Siemens democratized AI by helping employees identify valuable applications, collaborate with technical specialists, and scale successful use cases. (Please view our previous post and the scholarly article.) However, the case study also raises a broader leadership question.
As employees continue to use AI, they can easily fall into AI dependency, which can discourage actual critical thought and even lead to temporary cognitive decline. For instance, in section 3.1 of the article AI-overdependence and human cognitive decline: Hazards, evidence, and mitigation strategies, researchers found that AI overuse can lead to the following:
- Inability to think critically and originally, make decisions, or evaluate information.
- Inability to remember events properly or maintain attention span.
- Inability to visualize creatively or maintain writing skills.
Fortunately, the study found a silver lining: As long as users (in the study’s case, undergraduate students) incorporated AI systematically while avoiding reliance, they could actually experience an increase in critical thought (section 3.1.1).
With these observations in mind, how can organizations expand access to AI while continuing to preserve the knowledge, judgment, and creativity of their employees?
Several of Siemens’ core principles can be applied more broadly to organizational leadership. Siemens did not approach democratization as simply providing more employees with AI tools. Rather, they developed an organizational capability that involved domain experts, data scientists, and IT professionals. Siemens also created learning opportunities, shared best practices, supported experimentation, and kept human expertise involved in higher-risk applications.
By looking more closely at these practices, other organizations can consider how to encourage AI adoption without allowing the technology to replace critical thought. To do this, we recommend a general workflow:
- Define Use Cases
- Review Outcomes Critically
- Ask Employees to Explain Decisions
- Preserve Opportunities for Independent Thinking
- Share Organizational Knowledge
- Monitor Long-Term Effectiveness (which is tied to Reviewing Outcomes Critically)
Define Where Human Judgment Is Required
Before introducing AI across the organization, leaders should determine what role it will play in different activities.
Some work can be assisted with relatively little risk. For example, an employee may use AI to organize meeting notes or develop an initial list of ideas. Other activities may require careful human review because they influence customers, financial decisions, security, compliance, or organizational strategy. Certain decisions may need to remain entirely human-led. (To learn more, read our other article on this series.)
When democratizing AI, Siemens brought together domain experts, data scientists, and IT professionals to identify valuable use cases and define the tasks that AI would perform. These employees also considered what data the systems would use and how their probabilistic outputs would fit into business processes (pg 13).
Ultimately, employees who understand the organization, its customers, and its operations are better positioned to identify where AI can create value and where human judgment should remain central.
Review Outcomes Critically
Managers may try to determine AI success through number of tokens or level of output. Unfortunately, this approach does not necessarily reveal whether the work is accurate or valuable.
A better approach is to review the quality of the outcome.
Managers can evaluate whether the work is accurate, well-reasoned, original, and appropriate for the situation. For higher-risk activities, they can also review compliance, security, testing, and documentation. Random work reviews may help organizations identify whether employees are verifying AI outputs or simply submitting them with minimal consideration.
One place to begin is by establishing quality criteria for common types of work. An organization could provide selected outputs to an approved AI system, then ask it to evaluate them against a defined rubric. Depending on the role and level of risk, below are some useful criteria.
As an illustration, AI could help determine whether a report addresses the original request, supports its conclusions, acknowledges important risks, and meets the organization’s standards. Managers could then review a sample of the results, particularly when the work affects customers, financial decisions, security, compliance, or organizational strategy.
Leaders may also compare results over time. Are employees catching more errors? Are they asking better questions? Can they explain their reasoning more clearly? Are they developing stronger subject knowledge, or are they increasingly unable to proceed without AI?
The Siemens case study reinforces the need to evaluate AI within its operating context. Its AI systems required testing, quality-management measures, data evaluation, and involvement from domain experts. In healthcare, Siemens adjusted workflows, interfaces, and model choices based on clinicians’ explainability needs (pg 14–15).
Ultimately, leaders should focus on whether the work meets the organization’s standards, not merely whether AI appears to have been involved.
Ask Employees to Explain Their Decisions
An employee does not need to complete every task without AI to demonstrate understanding. However, the employee should be able to explain the work.
Depending on the situation, managers could ask the following of more critical outputs:
- Why was this approach selected?
- Which assumptions were made?
- How was the AI output verified?
- What did the employee change?
- Who remains accountable for the result?
These questions help reveal whether AI supported the employee’s reasoning or substituted for it.
Similarly, Siemens involved domain experts in managing the risks associated with AI’s outputs. Quality assurance, regulatory, and product experts adapted quality-management practices to account for data dependency and changing system behavior. Siemens also kept clinicians involved rather than allowing its healthcare AI to make decisions independently (pg 14–15).
Human oversight is most effective when the person reviewing the output understands both the subject and the reasoning behind the decision.
Preserve Opportunities for Independent Thinking
If employees consult AI before forming any observations of their own, the technology may begin shaping the direction of their thinking.
Over time, teams could become faster at producing familiar answers while becoming less likely to propose alternatives.
Organizations can protect independent thinking through practices such as the following:
- Asking employees to record initial observations before using AI
- Holding occasional AI-free brainstorming sessions
- Comparing human-generated and AI-assisted approaches
- Rotating responsibility for proposing alternatives
- Reviewing unsuccessful ideas and lessons learned
- Rewarding thoughtful disagreement rather than speed alone
For example, Siemens created an AI Lab where domain experts, data scientists, and IT professionals could collaborate on prototypes. The company also used an AI Academy to build knowledge and help employees participate in its wider AI community (pg 7 and 13). Later, the authors recommend providing an environment where employees with different expertise can experiment together and develop a shared understanding of AI (pg 18).
For other organizations, an AI lab does not necessarily require a separate facility or large department. Leaders could create a recurring workshop where employees identify problems, test use cases, compare results, and discuss failures. The important part is creating a structured environment for experimentation, rather than simply giving employees access to a tool.
Share What the Organization Learns
Individual departments often begin experimenting with AI independently. While this can encourage innovation, it may also cause teams to repeat the same mistakes or develop overlapping solutions.
Siemens addressed this challenge through best-practice exchanges and AI showcases. These activities helped teams transfer knowledge and scale successful AI applications into similar environments (pg 13).
Organizations can take a similar approach by asking employees to share not only successful outputs, but also how those results were achieved. A useful showcase might explain the problem, assumptions, human expertise involved, verification process, limitations, and lessons learned.
Shared infrastructure can also help teams reuse approved models, applications, data practices, and other AI assets. Siemens used scalable data and cloud-based infrastructures to support data management and expand AI solutions across facilities (pg 13–16). For most organizations, the appropriate infrastructure will depend on their size, risks, data, and business needs.
Monitor Long-Term Effectiveness Alongside Productivity
If they do not carefully monitor their work, an organization can increase its output while quietly weakening its internal capabilities. If employees produce work faster but become less able to perform, evaluate, or explain the underlying work, they may be experiencing unhealthy dependency.
Fortunately, management reviews do not need to become another burdensome performance system. A small number of role-specific criteria, periodic work samples, and short conversations may reveal more than productivity figures alone. Leaders can even employ AI to assist with the initial evaluation. (However, they should validate its assessments rather than allowing one AI system to grade the work of another without human review.)
At the end of the day, education should grow alongside AI adoption. For instance, Siemens’ AI Academy illustrates how organizations can provide scalable learning, rather than assuming that access to technology is enough (pg 13).
Just like how a college student experiences higher success when evaluating and organizing their coursework, an organization can achieve both higher productivity and quality work through a systematic approach.
This is one part of our series. To learn more about this topic, take a look at our other articles:
(Summary) Why Original Thought Will Determine Organizational Success
Think, Assist, Verify — And How It Can Be Your New AI Framework
Leave A Comment