Decoding Business-Speak with AI: The Power and Paradox of Large Language Models
ChatGPT can streamline professional communication for us but maybe we should just drop it entirely
I've been exploring the potential of Large Language Models (LLMs) such as ChatGPT (both GPT-3.5 and GPT-4 versions). One area of interest for me is the ability of ChatGPT to generate business speak. Business-type communication is both highly formulaic and highly flowery. In 200 years, we’re going to look back at the style of communication and think the same as how we think about 18th century letters in their flowery speak:
Dearest Lady Arabella Stuart,
I beg your indulgence as I humbly inquire about my previous missive of April 23rd, which detailed the matter of our forthcoming assembly and the necessary arrangements. Time presses upon us, and I find it necessary to entreat your response.
Kindly favour me with your esteemed thoughts at your earliest convenience. I shall be ever obliged for the honour of your correspondence.
Yours in gratitude,
Lord Percival Harrington
A follow up email in the styleof an 18th century letter
LLMs are quite good at generating this type of text, and 18th century epistles as well apparently. In this blog post, I’m going to talk a bit about my experience in generating the sort of formulaic business-speak, some of the reasons why there is a difference between how junior and senior staff write emails, and whether any of this is worth it at all.
Interview Feedback
One of my responsibilities at work is to administer a logic and reasoning test as part of our hiring process. The test is designed to evaluate a candidate’s critical thinking and problem-solving abilities. After the tests, I need to write a short memo giving my feedback on the candidate, how they did on the test, what I thought, etc. The point is to provide insight into the candidate for those who are making the hiring decisions: how did they do on the test? do I think they will be successful at the firm based on what I saw? Were there any red flags?
These feedback memos take me a long time to write. I take notes during the interviews, but I need to sort, filter and collate them. I thought it was a natural place for ChatGPT to step in. I have been working on a generic prompt describing the test in some detail, along with an example of a well written feedback summary, along with a series of bullet points about the candidate's test.
ChatGPT does a fantastic job of generating the sort of feedback memo I look for. It does a good enough job that I don’t think I ever need to write one again. I probably save myself about half an hour an interview now, which works out to be maybe 5 hours a month, a 3% performance improvement. This might not seem like much, but what would you do with 5 extra hours a month?
ChatGPT has two versions available now: GPT-3.5 and GPT-4. I observe notable differences between the outputs of the two models. GPT-3.5 generated more holistic summaries of a candidate's performance, highlighting their strengths and weaknesses in a general manner, such as stating that a candidate was good because of factors X, Y, and Z. I had a hard time getting specific examples to be included though. For example, I might have received something along the lines of "the candidate struggled with addition", where I would be expecting "the candidate struggled with addition because of this one question they flunked".
In contrast, GPT-4 produced a chronological feedback memo that closely mirrored my personal writing style. This approach provided a more detailed and step-by-step account of the candidate's performance, including how they started, improved or struggled during specific sections, and their overall performance.
What is interesting is that GPT-4 is able to make sense of my assorted thoughts (by far the hardest part of the feedback process). From that, it is able to write a good memo that described the candidate well. Most interestingly, it has been able to pick up on the fact that I like chronological memos. I did not ask it to write that way, only a generic “here is a well written example”. GPT-4 informs itself based on my hidden preferences, which I didn’t even know existed.
Business Speak Assymmetries
This begs the question though: could a human do what ChatGPT does, and take my assorted thoughts and not worry about all the business speak? Why can I not just throw the bullet points up as feedback?
I could obviously do this, and I doubt I would get any negative feedback. Something doesn't feel right about doing it though.
One thing I've noticed in my career is an asymmetry between bosses and subordinates when it comes to communication. The two most driven leaders I know have the worst email etiquette, which is often jarring when communicating with them.
For example, let's say I want to forward an email to them because the client didn't cc them. My email would say something like "Dear [name], see the attached email below, the client didn't cc you and I wanted to make sure you got it. Please let me know if you have any questions. Thanks, Nick". If the situation was reversed, the email I would get would be "fyi" at best, probably just an empty email. Why can my boss get away with this while I can't?"
One possible answer is an asymmetry of power. Senior leaders have a lot more power in an organization, and thus have (potentially) more leeway in their communication style. Junior employees, on the other hand, often feel like they need to be more polite and diplomatic in their emails to make a good impression and build relationships with colleagues and clients. This asymmetry of power can create a situation where employees feel the need to write longer, more formal emails than their bosses.
But really, I think the answer is truly that I could get away with it. Why would my boss care? He's busy and doesn't read the boilerplate anyways. It's all in my head. Part of gaining experience in your career must be realizing that all that is bullshit anyways. I don't think at any point in my career has anyone told me to change how I write emails. No one ever told me to be polite, no one has ever told me to be less polite. I'm not sure if anyone actually cares!
My ability to write professional emails though is a sign of privilege. I had the chance to learn how to do it properly, while many people do not. Many people only realize there is a problem when they don't get the job they are qualified for, or never got promoted, etc. However, the rise of LLMs can help to level the playing field. By using LLMs to generate professional business text, employees can now spend less time on writing emails and feedback memos, and still produce the same level of quality as their bosses. They can also use them to parse the formulaic business speak into natural language to get better understanding.
But what happens when both parties in a conversation are using LLMs to communicate? Doesn't it seem silly to wrap all our words in sentences that only serve to make our thoughts more vague and less direct?
Why Business-Speak At All?
It is similar to how some companies try to communicate only by API, through code itself. Amazon structures itself that information should be available by API to remove the overhead associated with asking people for information. Instead of talking with individuals, you can talk to code that people have written. Maybe it is possible to cut out the "code" in between and just talk to each other without the LLMs involved at all.
If so, it means we've found a technology that is essentially useless. LLMs can provide value by translating to and from business speak It provides value by doing something that we've just decided is all bullshit anyways. It's like the bureaucrat who is extremely efficient at pushing the meaningless paper around. He's really good at what he does, but he shouldn't be doing it in the first place.
Still, there is another assymmetry involved here: those who use LLMs to structure their communication vs those who don’t. There’s a benefit for me to using it to structure communication, especially if those reading the communication rely on it being structured as business-speak. If my boss is expecting it, I get a benefit from giving it to him without the time and cognitive energy in creating it. Only my boss suffers then from needing to process it. I have a benefit over my peers then as well. If those reviewing my cover letter and CV are expecting my communication in a certain way, I need to give it to them even if its a waste of their time because otherwise I’m wasting my time.
It’s a Moloch problem: its a trap where multiple people have incentives to not coordinate. Individual actions all make sense within their context, but viewed from above it is a waste of everyone’s time. We would all need to agree that this is a waste of time, and to leave the boilerplate of our emails aside but as long as not everyone has agreed there is incentive to keep doing it. This only gets worse with ChatGPT because I can gain the benefits without any of the downsides - I no longer have skin in the game for improving this.
So, LLMs have a useless ability. They can generate business-speak to benefit myself to the detriment of others. It’s useless because it should not be necessary, but because we all agree that it is necessary it has some benefits. If I’m getting a benefit without skin in the game, I might as well make full use of it.