plugs and muffs are good ideas.If you handload some lighter loads may help,or try some .45 colt loads.They are pleasent to shoot in the raging bull.A glove on the shooting hand is a good idea too.The 454 seems to want to tear the web out of my shooting hand so a glove is necessary for me with this cal.
I have other large bore pistols 12 in. 45-70 358jdj 308 .45 colts that dont bother me like the .454 does.as for shooting off sand bags or rolled up towels cover them with a piece of soft leather, as the cylinder gap blast will wreck them in a hurry.