Questioning the Hype

Before you flip the switch on AI-Powered learning tools, here’s what I would want to know first.

I've been in Learning & Development for over two decades, most of that time leading teams in financial services. I was an early adopter of learning management systems when they were still clunky and oversold. I was in the room when MOOCs were going to "democratize learning" and make corporate training obsolete. I watched the LXP wave promise to kill the LMS entirely, only to see most regulated organizations quietly keep both running because neither could do the full job alone.

I say this not to be cynical. I say it because I've lived through enough hype cycles to know that the distance between a compelling product demo and a functional enterprise deployment is often enormous. And right now, I'm watching a new wave build with an energy that feels very familiar.

Modern, AI-powered tools and platforms designed to empower HR teams are here. Nearly every major platform is either shipping one or about to. The promise is genuinely exciting: an intelligent layer that sits on top of your existing learning content, coaches your people in real time, personalizes the experience, answers questions, identifies gaps, and accelerates development. Plus, provides insights to leaders about their employees and how they are working, learning and growing. If it works as described, it's transformational.

I want it to work. I think it can work, eventually. But as someone who's managed massive content libraries, navigated regulatory audits, and inherited the kind of tech debt that accumulates over a decade of rapid tech acquisition, I have questions. Not objections. Questions. The kind I'd want answered before I invested budget, time, and organizational trust in a tool that my people, and my regulators, would need to rely on.

Here are the three areas that give me the most pause.

The Content Accessibility Problem: Can the AI Actually Read What You've Built?

Let's start with the most foundational question: can the AI actually access and understand the learning content it's supposed to be coaching on?

If you've spent the last decade in a regulated industry (financial services, healthcare, insurance), your content library probably looks a lot like the ones I've managed. Hundreds or thousands of custom courses, built to address your specific products, your proprietary systems, and the very particular requirements of your governing bodies. These aren't generic leadership modules pulled from a marketplace. They're bespoke, detailed, and built to serve a precise purpose.

And most of them are packaged as SCORM files.

SCORM has been the backbone of eLearning interoperability for years. It allowed us to build in one authoring tool and deploy across different platforms. But SCORM was never designed with AI in mind. It was designed to communicate a narrow set of data back to your LMS (think completion status, pass/fail, time spent) and to package course content into a zipped, self-contained file that could run independently.

Here's the challenge: that zipped package is not uniformly structured in a way that AI can easily read. Different authoring tools, and there are many, organize internal components in their own proprietary ways. There's no universal standard for where the actual instructional text lives inside a SCORM file, where the metadata is stored, or how media and interactions are formatted. Even the creators of the SCORM standard have acknowledged this reality. Course data is not universally formatted when published, which makes automated AI extraction (the kind you'd need for an intelligent assistant to understand and teach from your content) highly complex.

This doesn't mean it's impossible. Tools and middleware solutions exist to parse SCORM files and extract readable content. Some platforms are bypassing SCORM entirely, choosing to ingest raw source files like PDFs, PowerPoints, or video transcripts as the AI's reference material rather than attempting to crack open the published packages. Others are using behavioral signals and metadata tagging to infer what a course is about without actually reading its contents.

These are clever engineering solutions. But they introduce a question that buyers should be asking: what is the AI actually reading when it coaches my learner? Is it reading the course itself? A proxy document? A metadata description someone wrote ten years ago? An external source?

And here's where it gets real for anyone who's inherited a mature content library: do you still have the source files? In many organizations I've worked with, the original Articulate or Captivate project files lived on a departed instructional designer's laptop. All that remains in the system is the published SCORM package and a course title like "BSA Refresher 2021 v2 FINAL." There's no supplementary PDF to upload. There's no transcript sitting in a knowledge management system. The SCORM file is the institutional knowledge, and if the AI can't reliably read it, you have a gap between what the tool promises and what it can deliver.

I'm not saying this problem is insurmountable. I'm saying it requires real work (time, money, and operational effort) to solve. And I'd want to understand exactly how much of that work has been done, or needs to be done, before I trust an AI to coach someone through content it may not fully understand.

The Compliance Verification Problem: Can You Prove What the AI Taught?

My second concern is one that will resonate immediately with anyone operating in a regulated environment: if an AI-powered assistant is now part of how your people learn, can you demonstrate to a regulatory body what was taught, how it was assessed, and that it meets the standard?

I've watched this movie before. When learning experience platforms emerged, they brought a beautiful, Netflix-style interface and a promise of self-directed, personalized learning. What they didn't bring, at least not initially, was the kind of airtight reporting and documentation that regulators require. In financial services, when an auditor asks you to prove that every person in a specific role completed specific training on a specific regulation by a specific date, you need more than engagement analytics. You need definitive records. The LMS survived the LXP era in regulated industries precisely because it could still tell that evidentiary story: here's the content, here's the assessment, here's the completion record, here's the timestamp, here's the mapping to the regulation.

Now AI-powered learning assistants introduce a fundamentally new dynamic. The value proposition is personalization: the AI adapts to you, answers your specific questions, meets you where you are. But personalization and regulatory consistency exist in tension.

Consider two bank tellers going through compliance training on the same topic. In a traditional course, they both see the same content, complete the same scenarios, and pass the same assessment. You can stand behind the consistency of that experience. Now put an intelligent assistant in the middle. Each teller asks different questions. The AI provides different explanations, possibly emphasizing different aspects of the material. Maybe both explanations are accurate. But maybe one is subtly incomplete. Maybe one omits a nuance that a regulator would consider material.

Who's auditing every AI interaction for accuracy and completeness? Who's validating that the coaching experience for Person A met the same standard as Person B? When the regulator asks what someone was trained on, are you handing them a chatbot transcript and hoping it holds up? And how are you certain the AI didn't hallucinate a detail, present outdated guidance, or contradict what the actual course material says?

There's also a version control question that concerns me. When a regulation changes, in the traditional model you update the course, re-assign it, and can pinpoint exactly when the new content went live. With an AI assistant pulling from various sources or generated responses, how do you confirm it stopped teaching the old guidance immediately? How do you prove it did?

These aren't hypothetical risks. Regulatory bodies have, if anything, gotten more stringent about training documentation over the past several years. The standard of proof is going up, not down. And I'd want to understand, very clearly, how an AI-powered learning tool accounts for this before I deploy it in any environment where a regulator might come knocking.

The Identity Problem: Does the AI Know Who It's Talking To?

The third area I keep coming back to is one that sounds simple on the surface but is deeply complex in practice: does the AI actually know who it's talking to, and what that person needs?

Most AI tools in the HR Space rely on some form of learner profile to personalize the experience. They pull from your job code, your role, your department, your level, and use that information to calibrate what content to serve, at what depth, and toward what goals. This sounds logical. It also makes a massive assumption: that your job architecture is clean.

In my experience, particularly in large enterprises, it isn't.

Job architecture in most organizations is a sprawling, organic mess that's accumulated over years of mergers, reorganizations, and one-off accommodations. There are thousands of job descriptions, often stored as individual Word documents with no standardized format. The job description for a Project Manager in IT reads nothing like the one for a Project Manager in Product. Different skills, different expectations, possibly different pay scales, yet they share the same title. Or the reverse: two roles with completely different titles that functionally require the same competencies.

So when an AI-powered HR Tool pulls your job code and says "based on your role, here's what you should learn next," what is it actually referencing? Is it reading your specific, internal job description? A generic one from a third-party taxonomy? An open-source skills framework that may not match how your organization actually defines that role? If you titled someone a "Project Manager" but their actual responsibilities map more closely to a "Program Manager," the AI is personalizing against the wrong profile.

And the depth problem compounds this. Even within legitimately similar roles, the knowledge and skill depth required can vary dramatically depending on the business unit, the client base, the product set, or the regulatory environment that team operates in. A one-size-fits-all skills inference based on a job title or code simply isn't granular enough for the promise these tools are making.

Some platforms address this by allowing learners or their managers to self-identify skills and development goals. But anyone who's attempted to build a skills inventory at the enterprise level knows how this plays out. People don't know what they don't know. They're often reluctant to admit gaps. Managers may not have enough proximity to each team member's daily work to make accurate assessments. Self-reported skills data is better than nothing, but it's a shaky foundation for an AI that's supposed to intelligently guide someone's growth path.

If the job architecture feeding the AI is inaccurate, the content it recommends will be misaligned. If the content it's recommending is a SCORM file it can't fully read, the coaching will be incomplete. And if the reporting can't prove what was actually taught, no one catches the error. Each broken foundational layer makes the next one worse.

This Isn't a Technology Problem. It's a Readiness Problem.

I want to be clear: I'm not arguing against AI in HR. I think these tools represent a genuine inflection point in how we develop people, and the organizations that get this right will have a meaningful advantage. But "getting it right" requires more than turning on a feature or purchasing a new platform. It requires honest assessment of whether your foundational layers (your content, your data, your architecture) are in a state where an AI can work with them reliably.

The demos are impressive. The vision is compelling. But demos are typically run on clean content, durable skills topics like leadership and communication, and open-source material that's widely available and low-risk if the AI gets it slightly wrong. That's a very different environment than a library of thousands of proprietary SCORM packages built for a highly regulated business, mapped to specific policies that change quarterly, and served to roles that are inconsistently defined across the organization.

I've seen too many technologies arrive with enormous promise and then underdeliver, not because the technology itself was bad, but because the organizational infrastructure wasn't ready to support it. The tool works beautifully in controlled conditions. Then it meets the reality of legacy content, messy data, and operational complexity, and the gap between what was sold and what's functional becomes painfully apparent.

Four Questions I'd Ask Before Buying (or Turning On) an AI-Powered HR Tool or Platform

If I were evaluating one of these tools today, whether as a new purchase or a feature that's now available within a platform I already use, here's what I'd want to know:

1. How does your AI access and interpret the content in my existing library, specifically my SCORM-packaged courses? I'd want to understand exactly what the AI is reading, whether it requires supplemental source files, and what happens with content it can't parse. I'd want to see it work on my content, not a demo library. I would want to know how large of a library its been tested on, and if I could speak to a current customer who has navigated bringing the tool into their HR Tech Ecosystem that contains thousands and thousands of proprietary SCORM files.

2. What does the evidentiary trail look like for a regulated audit? Show me what documentation this system produces when a regulator asks how a specific employee was trained on a specific topic. What does the record of an AI-coached interaction look like, and has any regulatory body reviewed or accepted that format?

3. How does the tool ensure consistency of instruction across learners in the same role with the same compliance obligations? If personalization means two people get different learning experiences, how do I confirm both met the minimum standard? What guardrails exist?

4. What does the AI use to determine who a learner is and what they need? I'd want to understand exactly what data sources it pulls from (job codes, descriptions, skills profiles) and how it handles inconsistency or gaps in that data. What happens when the input is wrong or incomplete? And similar to the questioning around SCORM files, can they connect me with a customer who has already implemented the tool using thousands of inconsistent job profiles saved in formats like Word Documents (and across multiple libraries and sources)?

A Final Thought

I've spent my career building learning functions in environments where the stakes are high, where "good enough" isn't good enough, and where the systems that support it need to be defensible. I believe AI will make all of HR better. I also believe that rushing to adopt it without doing the foundational work is a recipe for the same disillusionment we've seen with every previous wave of HR technology that arrived faster than organizations could absorb it.

Ask the hard questions. Pressure-test the promises against your actual reality. And if you need a partner to help you assess your readiness or navigate this landscape, that's exactly the kind of work I can help you with.