The .357 is fine for deer if used properly. Most does around here are 75-100lbs tops. Bucks may go to 160lbs or so.
Either way, the .357 does it within 40 or so yards, with a proper bullet. I never used anything but regular 158gr JSP ammo, picked my shots behind the shoulder. No problems.
Be all of this as it may, I also chose a larger caliber.
Mainly because, when handgun hunting, you have limited shots due to range. I chose to up the ante to my favor, with a caliber/bullet, that can punch deeper through all angles.
Some things that I have noticed over the years:
When talking home defense, a 357 is often highly regarded as
a manstopper.
If you mention a 22lr, you get scoffed at and ridiculed at
how underpowered it is.
When talking deer hunting, a 357 is often scoffed at and ridiculed as being too small.
If you mention a 22lr, you get stories of all the large game
effectively killed with it.
Weird.