"Pardon me", like "excuse me" is an unnecessary apology to be avoided because it is too polite, over-used and weak sounding.