So says Theodore Dalrymple, whose regular readers know was a prison physician. Excerpt:

The problem with low self-esteem is not self-dislike, as is often claimed, but self-absorption. However, it does not follow from this that high self-esteem is not a genuine problem. One has only to go into a prison, or at least a prison of the kind in which I used to work, to see the most revoltingly high self-esteem among a group of people (the young thugs) who had brought nothing but misery to those around them, largely because they conceived of themselves as so important that they could do no wrong. For them, their whim was law, which was precisely as it should be considering who they were in their own estimate. It need hardly be said that this degree of self-esteem is certainly not confined to young thugs. Most of us probably suffer from it episodically, as any waiter in any restaurant would be able to tell us.
In short, self-esteem is but a division of self-importance, which is seldom an attractive quality. That person is best who never thinks of his own importance: to think about it, even, is to be lost to morality.
Self-respect is another quality entirely. Where self-esteem is entirely egotistical, requiring that the world should pay court to oneself whatever oneself happens to be like or do, and demands nothing of the person who wants it, self-respect is a social virtue, a discipline, that requires an awareness of and sensitivity to the feelings of others. It requires an ability and willingness to put oneself in someone else’s place; it requires dignity and fortitude, and not always taking the line of least resistance.

An interesting paradox: to respect others is to respect yourself.