How could you politely end a conversation with a person who is so annoying?

I am very blunt; I just tell people I don’t want to talk to them and walk away or hang up if that’s the case. I don’t think you should put up with stuff if you are not comfortable for the sake of someone else.