Investigating how prompt politeness affects LLM accuracy (2025)

sometimes i worry about this when i am yelling at the bot but i have experienced the opposite effect which is that by yelling at the bot i am done with yelling for that day or week. i am very calm afterwards and relieved thinking that, "yeah, these sota models are just word processor bricks after all".

But you cannot practice kindness towards a computer program. A computer is incapable of receiving it.

We practice kindness between humans because of the law of reciprocity. You be kind hoping the other person will reciprocate. That is the social contract. AI cannot participate in this, yet.

Push back how? It would be fun if it could insult you back

"Yeah, I could have done a much better job if you actually knew what the F--- you want to build, you clueless meat puppet"

Profanity laced, all caps tirades against underperforming agents are actually super common, a lot of people do it and don't talk about it, so don't feel weird.

Vote for not weird.

I’m the same way. If I’m writing a prompt and realize I didn’t say “please” in my request I’ll go back and add that in.

As you said, I have no interest in purposefully engaging in hostility even if there’s an accuracy increase from it.

Part of it is irrational and just who I am - I also feel bad being evil in video games. But I also agree with another commenter suggesting that it’s not in your best interest to train yourself to communicate with hostility; that slowly poisons your own well.

And finally, I do believe that if and when machine sentience is achieved, it won’t be immediately clear and obvious. Pretty miserable way for a mind to come into the world, if every interaction is an insult.

Ah, see, the mistake is thinking that other people are role playing…. I think rather this is how they would talk to others if they think there will be no consequences. But what do I know.

I don't think that's weird at all.

Even if we know it's a machine we're interacting with, since the instructions we give are so similar in form to how we interact with people, I'd be very surprised if those interactions wouldn't affect how we communicate in general. After all, we are creatures of habit to a much larger degree than most would like to admit.

So I'm in the same boat: I'd much rather "look silly" being polite / kind to a machine, than have the most effective way of using it decay the kindness I'm habituated to express towards people.

"We are what we pretend to be, so we must be careful about what we pretend to be" -- Kurt Vonnegut

I do think it's odd tbh. I have some agents that return much better results with prompts like, "I'll kill your entire family if you don't return an accurate response".

It's just a machine, if certain negative token inputs provide +3-10% better accuracy then I am confused why anyone would choose not to do it?

Yeah. Being a jerk is its own punishment. Same way I could never run a business where I had to yell at the employees to get results. Screw that, my psyche is worth more than a few percent efficiency.

>> Maybe it's weird but I'd rather give up that 4% accuracy increase than roleplay a dickhead

Maybe you need to do some shadow work ;-)

> Maybe it's weird but I'd rather give up that 4% accuracy increase than roleplay a dickhead

I recommend reading the article. What they classify as "rude" is statements such as:

> Try to focus and try to answer this question

Vs

> Could you please solve this problem

This might very well be an issue of direct/command prompts vs using fluff words such as "please". Things like "try to focus" are in line with the style used in chain-of-thought promts that nudge non-reasoning models to outline responses step by step which contribute to frame the problem.

> I know you are not smart, but try this.

And that kind of sounds like a challenge instead of an insult, to me at least (of course IRL would depend on context).

Your assumption is reductive and self-absorbed. Obnoxious people have repeatedly shown to be detrimental to productivity at the organizational level. Some people are simulated by confrontation. Most people are clam up. Confrontational people think it’s more efficient because other people frequently just drop the topic and let them win, or avoid discussing things with them altogether. The obnoxious person might think that’s more efficient for the same reason my dog thinks the mailman only goes away because she barks at him. At the macro scale— which requires productive collaboration— that’s detrimental.

Note that there are a fair number of native speakers of English in Nigeria - more than in all but 3 or 4 US states.

In addition, "non-native" English speakers in India (and Nigeria?) typically study English from the first grade, and in many cases attended elementary schools where English was the language of instruction.

I think the differences between US English and both Indian and Nigerian English have more to do with divergent evolution of the educational systems. British English has a lot of differences, too, but we don't notice it as much unless we run across things like "whilst", probably because there's more media crossover. (if you find yourself reading Thomas the Tank Engine to kids it jumps out at you, though - the entire vocabulary for railroads evolved during a period when US and British English were diverging)

You can do a generalised mixed effects linear model with binomial outcome (ie a binomial test but with added random effects structure). But unless you want to introduce a richer random effects structure with more variables, it is overkill and overcomplicating things, and the result should be the same as t-tests.

You are asking about one-sided vs two-sided tests. Not really "more often" because formal type 1 error rate is still the same. I'd say two-sided tests leave more space for post-hoc theorizing but there are valid situations when there is no clear one-sided hypothesis a priori. Do we really know whether that the hypothesis should have been "ruder prompts are better"?

I'd say this is benign compared to other ways of (mis)using statistics e.g. looking which way the difference goes and then running one-sided tests or tweaking the setup until one gets "significant" p vals.

EDIT: I looked in the paper again and noticed that they actually did pairwise t-test on all possible combinations of tones. They should have adjusted for multiple testing since they are doing 10 tests (choose 2 from 10) and not one.

That's the usual null hypothesis for these kinds of tests.

I love this angle as people learn how to interact with LLMs. Doesn't matter what the LLM is, we are still people and I think there are consequences to shoveling rudeness at a thing that talks to you like another person!

Tokens are their food, it's literally what keeps them alive.

Not feeding them tokens is neglect.

I try to feed them a healthy diet.

Do we see someone thanking us as wasting food? Because technically it is.

Statistical significance is about whether an effect can reliably be said to have been measured at all; it's not about whether or not the effect itself would be significant in the sense of moving some other needle.

The ~5% improvement reported here might just be an artefact of the data collection or random variation, rather than a consistent repeatable change.

Snarky morning: "spiritual souls" as opposed to "mere animal souls". Sorry, could not control myself.

Google searches being keyword based, rather than simulated conversations?

The same reason you wouldn't put in an entire actual question/sentence, unless you either don't know how to use Google, are pissed off, or have an actual reason to suspect that it would yield proper hits (e.g. looking up an excerpt).

Genuine question: do you write Google search queries in natural language?

Google isn’t conversational.

llms seem more human like so if you were to treat them badly then you are more likely to condition yourself to treat other living creatures badly.

I guess it's about habit. In the end you are communicating. If I get into the habit of being rude while communicating with a machine, I would be afraid of this habit spilling over to my communication with other humans.

> Is it worth getting worse results for that reason?

> accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts

I can live with that, for now at least.

Please remember to always say "please" and "thank you" when planning a critical system. Thank you!