Welcome to the forum Krankie!
If you practice, I think you will find your limitation. I have NO problem shooting with iron sights out to 100 yards. But I have shot with iron sights a lot.
What is the handgun you intend on using? Have you thought about putting a scope on it if you are worried about the iron sight?
Duane