AI for Managing Directors and CEOs: From Hype to ROI - The 2026 Guide for SMEs

Artificial intelligence has arrived in most companies - but the profit from it has not. 88 percent of organizations now use AI in at least one area, yet only around 6 percent generate measurable economic value from it (McKinsey, State of AI 2025). This gap between usage and value creation is the real leadership task for managing directors and CEOs in 2026.

For decision-makers in small and medium-sized enterprises (SMEs), the question shifts fundamentally: no longer "should we use AI?", but "why do only a few manage to create real value - and how do we become one of them?". This guide answers exactly that. It shows where AI really pays off in SMEs, what obligations the EU AI Act 2026 brings, why so many AI projects fail - and provides a concrete 90-day roadmap for adoption.

Key Takeaways

• The value gap is the real problem: 88 percent use AI, but only about 6 percent achieve a measurable value contribution (McKinsey 2025).
• SMEs are catching up: 41 percent of German companies actively use AI - a doubling compared to the previous year (Bitkom 2026).
• Failure is the rule: 95 percent of AI pilots deliver no measurable ROI - almost always for organizational, not technical reasons (MIT 2025).
• AI is a boardroom matter with liability: Since February 2025, the EU AI Act requires AI competence in the company; managing directors can be held personally liable.
• Success is 70 percent a question of people and processes, not technology (BCG 10-20-70 principle).
• Structure beats chance: A clear 90-day roadmap delivers results faster than aimless experimentation.

AI in SMEs 2026 - where do we really stand?

The use of AI is no longer a niche topic in the German Mittelstand, but it is far from universal. According to the Bitkom AI Study 2026, 41 percent of German companies with 20 or more employees actively use AI - a doubling within a year. At the same time, KfW Research shows in February 2026 that across the broader Mittelstand only around 20 percent of companies actually use AI, which is about 780,000 businesses.

These seemingly contradictory figures are explained by different reference groups: the larger the company, the higher the AI usage. The ifo Institute reports 54.4 percent for the overall economy in June 2026 - driven by large enterprises, while smaller businesses lag behind.

One development is decisive: AI has become a boardroom matter. In 75 percent of companies, the leadership level now drives the topic (BCG AI Radar 2026). That is logical, because AI is not a pure IT question but a strategic one.

Why 95 percent of AI pilots deliver no ROI

The sobering truth behind the AI boom: most projects do not pay off. A widely noted study by the MIT project NANDA concluded in August 2025 that 95 percent of enterprise-wide GenAI pilot projects have no measurable effect on the profit and loss statement.

The right interpretation matters: what is measured is the missing business value, not a technical failure. The technology works - but it is too rarely used in a way that ends up putting more money in the bank. This is precisely where the management's task lies.

What "AI as a boardroom matter" concretely means for managing directors

AI as a boardroom matter means that management sets direction, resources and framework - and does not delegate what is strategic. In concrete terms: prioritize use cases, release budget, take responsibility for governance and bring the workforce along.

The figures show why this counts: according to McKinsey, active steering by the CEO correlates most strongly with a measurable EBIT effect from AI. Yet only 28 percent of CEOs take responsibility for their company's AI governance themselves. Anyone who leaves AI to chance or to the IT department alone gives away the very lever that decides between success and failure.

For managing directors in SMEs, this is good news: you do not have to become a data scientist. You have to ask the right questions, set clear priorities and organize responsibility.

Where AI really pays off in SMEs - three use cases with ROI

Not every AI deployment pays off equally well. McKinsey shows that around 75 percent of AI value potential is concentrated in four functions: customer service, marketing and sales, software development, and research and development. For SMEs this means: the best entry point is where processes are recurring, data-intensive and time-consuming.

The following three scenarios are illustrative examples of typical SME situations. The figures given come from industry studies and should be understood as orientation - not as guaranteed results.

Scenario 1: Back office - the incoming invoice as a quick win

A typical entry scenario in mechanical engineering: a business with around 180 employees processes hundreds of incoming invoices manually every day. With AI-supported document recognition, invoice data can be automatically extracted, checked and posted. Industry studies show time savings of 60 to 80 percent here, with recognition rates of over 95 percent. An additional driver is the e-invoicing obligation that has applied since January 2025. Because the results are immediately measurable, the back office is ideally suited as a first quick win.

Scenario 2: Sales - faster to the right quote

A technical wholesaler with around 100 employees loses time and orders because quote generation is manual and slow. AI can prepare quotes from product data and customer history and prioritize incoming inquiries by closing probability. McKinsey puts the potential in B2B sales at 13 to 15 percent additional revenue growth and a ROI improvement of 10 to 20 percent. The lever is not in replacing sales, but in accelerating it.

Scenario 3: Customer service - with a reality check

A service provider with around 60 employees wants to answer recurring customer inquiries automatically. An AI assistant can solve standard cases around the clock and increase service productivity by 30 to 45 percent according to McKinsey. An NBER study from 2023 measured 14 percent more resolved inquiries per hour, and for new employees as much as 34 percent.

But honesty belongs here: the Klarna example shows the limits. The company automated two thirds of its service chats, but in 2025 brought humans back into the team for complex cases. The lesson for SMEs is: AI plus human, not AI instead of human.

From practice: In our projects at ArkeonTech, we repeatedly see that the first use case should not be the most spectacular one, but the most quickly measurable. A well-chosen quick win finances and legitimizes the next steps. There is also a clear pattern: buying beats building - according to MIT, purchased solutions achieve a success rate of 67 percent compared to 33 percent for in-house developments.

Agentic AI - the top trend of 2026 with a reality check

AI agents are the dominant topic of 2026 - and at the same time the one with the greatest hype risk. An AI agent is a system that does not just respond, but independently plans and executes multi-step tasks: for example checking an order, creating it in the system and notifying the customer. Anyone who wants to dive deeper will find more in our article on AI agent strategy.

The market potential is enormous. Gartner forecasts that by the end of 2026 around 40 percent of enterprise applications will contain task-specific AI agents - compared to under 5 percent in 2025.

But the reality check belongs here too: Gartner also expects that over 40 percent of agentic AI projects will be discontinued by the end of 2027 - due to excessive costs, unclear benefits or insufficient risk control. Added to this is the phenomenon of "agent washing": of the thousands of providers advertising agents, only around 130 offer genuine agentic capabilities.

For CEOs this means: take the trend seriously, but do not follow the hype. An AI agent only pays off along a clearly defined business problem - not because it happens to be in fashion.

Why AI projects fail - and how you avoid it

The majority of AI projects fail. The RAND Corporation puts the rate at over 80 percent, more than twice as high as for classic IT projects. The reasons are rarely technical. They repeat themselves:

1. Missing data foundation: The data is incomplete, scattered or of poor quality.
2. Unclear use case: The project starts without a defined business problem and without a measurable goal.
3. Missing integration: The pilot remains an island and is never embedded into everyday work.
4. Neglected change management: The workforce is not brought along, acceptance is lacking.
5. Competence gap: The knowledge to use and steer AI sensibly is missing.
6. Wrong expectations: Impatience leads to premature cancellation.

Behind this is a pattern that the Boston Consulting Group summarizes in the 10-20-70 principle: success with AI consists of 10 percent algorithms, 20 percent technology and data - and 70 percent people and processes. The biggest lever is therefore not in the model, but in the organization.

McKinsey confirms this: the strongest difference between successful and unsuccessful companies is whether they redesign their workflows around the AI. Only 21 percent do this at all. Anyone who simply layers AI over existing processes rarely reaps more than an expensive gadget.

EU AI Act 2026 - what managing directors need to know now

The EU AI Act is not an abstract Brussels topic, but concerns the obligations - and the liability - of every management. The legal situation is in motion in 2026: with the so-called Digital Omnibus, the EU Parliament confirmed the first amendment to the AI Act on 16 June 2026. Formal approval by the Council was still pending at the end of June 2026, which is why the details should be read with caution. This will likely postpone many obligations for high-risk AI to the end of 2027. A detailed assessment for SMEs is provided in our article on the EU AI Act 2026.

What is decisive, however, is what has not been postponed and already applies:

Two points should be taken particularly seriously by managing directors. First, the AI competence obligation under Article 4: since February 2025, companies must ensure that their employees have sufficient AI competence; enforcement begins in August 2026. Certification is not required, but training must be documented. Second, the fines: violations can be penalized with up to 35 million euros or 7 percent of global annual revenue.

Personal liability is especially important. If AI policies, risk monitoring or employee training are missing, management can be held personally responsible. AI is therefore not only a strategic boardroom matter, but also a liability one.

The 90-day roadmap for AI in SMEs

The most common mistake is aimless experimentation. A structured entry over 90 days delivers results faster than years of trial and error. The following roadmap is divided into three phases.

Day 1-30: Stocktaking and quick win

1. AI inventory: where is AI already being used in the company - even unofficially?
2. Collect use cases: where do processes cost the most time and money?
3. Prioritize with the impact-effort matrix: high benefit, low effort first.
4. Select a quick win and define a clear metric by which success can be measured.

Day 31-60: Pilot and governance

1. Implement the quick win as a pilot - preferably with a proven purchased solution rather than in-house development.
2. Set up a lean AI policy: what is allowed, which data may be used?
3. Start AI competence training - this also fulfills Article 4 of the EU AI Act.
4. Clarify data protection: order processing, data residency and documented approvals.

Day 61-90: Scaling and anchoring

1. Measure the ROI of the pilot and assess it honestly.
2. Redesign the workflow - do not just set up the tool, but adapt the process.
3. Define responsibilities: who maintains, monitors and improves the solution?
4. Continue the roadmap and tackle the next use case.

After 90 days, the finished AI transformation is not in place - but a proven success, a functioning governance and an organization that has learned how AI works for it.

AI governance and risks - the CEO's duty

Governance is not a bureaucratic accessory, but the prerequisite for ensuring that AI does not become a risk. The reality is sobering: according to the IBM report Cost of a Data Breach 2025, 63 percent of organizations have no AI governance policies.

Three risks should be kept particularly in view by managing directors:

• Shadow AI: Employees use AI tools on their own initiative and share sensitive data in the process. Data breaches involving shadow AI cost an average of 4.63 million US dollars (IBM 2025).
• False information and liability: In the Moffatt v. Air Canada case, the company was held liable in 2024 for false information from its chatbot - the argument that the chatbot was independently responsible was rejected.
• Manipulation and fraud: At the engineering group Arup, employees transferred 25 million US dollars in 2024 after a deepfake-faked video conference.

The NIST AI Risk Management Framework and the ISO/IEC 42001 standard have established themselves as a framework for serious AI governance. They help to cleanly regulate responsibilities, risks and controls.

For SMEs in particular, one aspect is decisive: data sovereignty. Anyone who wants to use AI in a GDPR-compliant way and without mandatory dependence on US cloud providers needs a deliberate architecture. This is exactly where we at ArkeonTech come in - with AI solutions that consider data protection and control from the very beginning.

Conclusion: AI is a boardroom matter - but not a self-runner

The gap between usage and value creation is the central challenge of 2026: almost everyone uses AI, but only a few make money with it. The difference lies not in the technology, but in leadership - in clearly chosen use cases, a structured approach, lived governance and the willingness to truly rethink processes.

For managing directors and CEOs in SMEs, this is an opportunity: anyone who starts in a structured way now gains a lead that aimless competitors will not catch up on quickly. The 90-day roadmap is the pragmatic entry point.

Would you like to know where AI creates value fastest in your company? In a free potential analysis, we identify your best first use case together. Arrange a no-obligation consultation.

Frequently Asked Questions (FAQ)

What does AI concretely bring to my small or medium-sized company? AI reduces costs and creates time - fastest in the back office, sales and customer service. Typical quick wins are automated invoice processing (60 to 80 percent less effort) or AI-supported quote generation. The decisive thing is to start with a measurable use case.

Where should I start with AI as a managing director? With the use case that promises high benefit at low effort. Collect time-consuming processes, assess them in an impact-effort matrix and choose a quick win with a clear metric. The 90-day roadmap in this article provides the structure for it.

What does introducing AI in the company cost? That depends on the use case. For getting started: a proven purchased solution is usually cheaper and more successful than building in-house. It is important to budget for the hidden costs - according to BCG, 70 percent of the effort goes into people and processes, not the technology.

How do I calculate the ROI of an AI project? Define a metric in advance and measure before and after: time saved times personnel costs, higher conversion or shorter lead times. Be honest about the time horizon - according to Deloitte, many companies only achieve a significant ROI after two to four years.

What do I need to consider as a managing director with the EU AI Act 2026? Three things already apply: the ban on unacceptable AI, the AI competence obligation under Article 4 (train and document employees) and, from August 2026, the transparency obligations. Obligations for high-risk AI are likely postponed to the end of 2027. Fines reach up to 7 percent of global revenue.

Am I personally liable if the AI makes a mistake? That is possible. If AI policies, risk monitoring or employee training are missing, management can be held personally liable. The Moffatt v. Air Canada case also shows: companies are liable for false information from their chatbots. Clear governance is therefore mandatory.

Can I use ChatGPT and other AI in a GDPR-compliant way? Yes, with the right framework: data processing agreements, clear rules on data use, ideally data residency in the EU and avoiding shadow AI. For sensitive data, sovereign solutions without mandatory dependence on US cloud providers are recommended.

Why do so many AI projects fail - and how do I avoid it? According to RAND, over 80 percent of AI projects fail, almost always for organizational reasons: unclear use cases, poor data, missing integration and inadequate change management. The most important lever is to redesign workflows instead of just layering AI over old processes.

---

Sources

• The State of AI 2025 - McKinsey & Company
• AI Study 2026 - Bitkom e.V.
• AI in the Mittelstand - Focus on Economics No. 533 - KfW Research
• MIT study: 95 percent of GenAI pilots without ROI - Fortune / MIT Project NANDA
• Gartner: 40 percent of enterprise apps with AI agents by 2026 - Gartner
• Gartner: Over 40 percent of agentic AI projects canceled by 2027 - Gartner
• The Leader's Guide to Transforming with AI (10-20-70) - Boston Consulting Group
• Generative AI and the future of work - NBER
• EU AI Act - Article 4 (AI literacy) - EU AI Act
• Cost of a Data Breach Report 2025 - IBM