If you can barely afford one gun then try to make it a good one. If you can barely afford one gun, then most likely you cannot afford to get into handloading. So I understand the choice between 357 and the 44mag. I would go unabashedly with the 44 mag since the ammo choices are probably a little better. But do look at your budget!!!! Make a little clearer decision upon based upon that, to determine if you can afford which ammo, and how much? Not much good to have a handgun and not be proficient with it. 357 ammo will be cheaper.
Example: the 38sp for the 357 are quite cheap. But to get a reduced load in the 44mag, usually requires shooting the 44 SP and it gets rather expensive in comparison. I am speaking of Store bought ammo here. Or store bought options.
If you are retired, or have had some tragedy happen which forced you into this limited budget,, then ask yourself how well do I handle recoil ??? My suggestion is to first off go somewhere to a range. The outdoor ranges usually have friendly folks at them who will often times let you shoot a cylinder full of their favorite wheel gun. I do know of a few Indoor ranges that have rentals available. This would help you in choosing which caliber also.
Last consideration is weight. Here again you have to decide just how much weight you are will to attempt to carry. As much as I like a longer barreled pistol the shorty can be handier when getting in and out of a vehicle. Older folks with hip problems fair better with a shoulder holster. Chest holsters are great when at all possible. Here again you do not want to go with a cheapy. Get a holster that will be comfortable in carry and if you go with a Chest holster consider one with the options of being able to adjust the pistol to your liking.
Please excuse the length of this post but I always look ahead and try to weigh out all the options.
Last thing is the pure pleasure from the use of any tool. grips, weight, balance, feel. If you don't enjoy each and every shooting session then you will not be inclined to carry at all times.
357 has more blast and I think more noise. Not all 357s can handle the heavier loads or handle the longer cartridges of the heavier loads. Such as a 173 or 180 gr bullet.
Pro ammo is cheaper and the choices close to the same as the 44mag.

44mag more recoil in full house loads. plenty of ammo choices but higher in price.
If you can afford the 44Sp Its Much better to shoot than 357 or 44 mag. no blast and less noise. not much recoil either.
Heavier gun for carry but the weight is a trade off for when the recoil comes into play.