So, I got up in the middle of the night to pee, and when I got back, I looked around for a pillow, and found it, and picked it up. Under it was a cat. He gave me… a look. He was not happy.
In fact, he looked put upon.
… What’s scary is that at 4AM, this seemed likely to me to be the actual etymology of the phrase.