Day in the Life of Shadow AI: The Performance Review That Left the Building
- Read time: 5 min
Friday, March 22 – 2:21pm
Friday afternoon, three performance reviews left to finish before my PTO week starts, and my kids’ pickup line begins in 40 minutes.
I love this part of the job most days, watching someone’s growth show up in black and white. But end of quarter always turns thoughtful feedback into an assembly line, and today the clock is not exactly on my side.
I open an AI chat window in my browser, the free account I set up months ago to help draft emails and brainstorm interview questions, and paste in my rough notes for the first review, asking it to tighten the language and make the tone more … constructive. It works better than I expected, so I do the same for the second review, and then the third.
Three reviews done. Got the tone dialed in and the kids picked up on time. A good day.
It doesn’t even cross my mind to think about where those notes went. Why would I? It felt no different than using spellcheck.
What I did not know that Friday was the account I used had no contract with my employer behind it, no data processing agreement and no guarantee about how long that information might sit somewhere, who might see it or whether it would end up training a model down the road.
The reviews I pasted contained names, compensation information, and a few candid notes about performance issues that weren’t exactly flattering. All of that left our network the moment I hit enter.
That Friday wasn’t the only time either.
Review season after review season, for more than a year, I kept using that same account to help me—without a second thought.
I would not learn what any of this meant until 14 months later, and by then, it was too late.
Tuesday, May 11 – 9:03am
Our security team got the call on a Tuesday morning, from a vendor that monitors the dark web for stolen credentials tied to our company domain. The free account I had been using for over a year turned up in a batch of stolen logins pulled from an unrelated malware infection. Apparently, this malware scraped saved passwords off an infected laptop and sold them in bulk on a criminal marketplace. Nobody was sure how long my account had been exposed, how many times it had been accessed or who was accessing it.
What they could say was what was sitting inside it. Over a year of chat history. Dozens of performance reviews, complete with names, salary info, and disciplinary notes, all of it sitting in my personal account with no audit trail and no way to prove what had or had not been seen.
Legal counsel got involved within the hour. So did compliance. So did our CEO.
Our state law is clear about what counts as personal information and clear about what happens once that information may have been exposed. Names paired with compensation details and disciplinary records checked enough of those boxes that we had an obligation to notify every employee whose review had gone through that account, whether or not we could prove anyone had actually seen their data.
I helped draft those letters (I, uhhhh … did not use any AI tools). Some of the people receiving the letters were coworkers whose growth areas and compensation conversations I had once tried to soften with better wording, delivered instead as a notice that their information may have sat exposed for over a year. A few asked me directly whether their review had been one of the ones affected. I didn’t have an answer. The account had no logs of what I had pasted or when, and neither the company nor I could tell them with any certainty what had happened to their information.
Thursday, June 17 – 1:05pm
The forensics investigation took weeks and cost more than anyone wanted to say out loud. Outside counsel got involved. We offered credit monitoring to everyone named in the exposed data, and the town hall that followed was the hardest one I can remember sitting through. Leadership never tried to hide what happened. There was little anyone could tell people with confidence, and that uncertainty was its own kind of damage.
A few people forgave it faster than I expected. A few did not, and how could I blame them? Trust, once it becomes a question mark, does not snap back into place immediately someone says ‘Sorry.’
Monday, June 21 – 8:00am
Leadership decided not to ban AI tools. Their reasoning was that a ban would have addressed this one mistake and left the employees’ instinct behind it—wanting to do good work faster—free to find its next workaround. That instinct was never going anywhere on its own.
Instead, the company stood up a single sanctioned AI tool with an actual contract behind it, one that doesn’t train on our data, and logs every interaction. Personal accounts were blocked at the network level, and a reporting channel went up so people could ask questions about AI use before a mistake instead of after one.
Friday, September 24 – 2:21pm
I still use AI to help draft reviews. I trust the tool now, because I finally understand what it does with the information I give it, and because if something goes wrong again, someone will be able to tell our people exactly what happened and when. A year ago, I could not have told a single coworker what became of the words I wrote about them. Now I could. Unfortunately, I—and our company—had to learn that the hard way.
Every version of this story starts in a very similar way: someone capable, trusted and busy reaching for a tool that makes their job faster. The same instinct that makes them good at the job in the first place, also makes them the one most likely to find a shortcut when nobody has given them a sanctioned way to move that fast.
Most of the time, nothing ‘bad’ happens. The account isn’t compromised, the vendor never has its own breach, and the risk stays invisible. Nothing has to go wrong for a long time, right up until it does.
The best fix is giving people like me a version of the tool we already want to use, one that comes with a contract, a log and someone accountable for both, long before a breach forces the issue.
Every organization needs an answer for what happens when someone pastes a performance review, a client email or a contract into an AI tool.
This is a fictionalized, composite scenario built from patterns common to real-world shadow AI and data breach incidents. It does not depict any actual person, client, partner, employee or breach.
Talk to HBS about AI Governance and Risk Management before your shadow AI moment.
Related Content
Developing an AI Risk Management Framework
Learn to develop an AI Risk Management Framework to ensure safe and effective AI deployment. Discover key elements and tips, and explore the NIST AI RMF.
AI Liability Insurance: Who Pays When AI Gets It Wrong?
AI liability insurance is evolving fast. Learn what coverage gaps exist, how insurers are responding, and what your organization should do now to manage AI risk.
AI Governance for Trustworthy AI Deployment
Unleash AI’s potential responsibly. Learn what AI Governance is, why it’s crucial, and how you can implement it.
