While install the puppet dashboard, I noticed puppetlabs yum repo changed from using "http://yum.puppetlabs.com/RPM-GPG-KEY-reductive" to "http://yum.puppetlabs.com/RPM-GPG-KEY-puppet" causing yum to fail on the key import. I had run the key import manually couple times before I realized, yum was complaining about a different key.
Below is how I got around to removing all of the imported gpgkeys:
Find GPG keys and owners:
rpm -qa gpg-pubkey\* --qf "%{version}-%{release} %{summary}\n"Remove corresponding GPG Key.
rpm -e gpg-pubkey-8347a27f-4afe0f12 --allmatches