JD, first off, I highly recommend you research California's gun laws at packing.org. Depending on where you are going to be, it may be nearly impossible to get a carry permit. In the more rural counties, you have a much better chance.
As far as caliber choice, for any of the centerfire pistol cartridges, 9mm and up, the weight of the gun is going to be the most important factor in how you perceive recoil. 9mm recoil in a 2 pound gun is easily tolerated by most shooters. 9mm recoil in a 1 pound gun...many people want to put it down after the first shot.
And, of course, if you plan to carry it, weight represents exactly the opposite trade off. A 1 pound gun is not much of a burden. A 2 pound gun can feel like an anchor at times, if it's on your belt with certain types of clothing.
I know you said you want a semi-auto, but if you have to be a "one gun" family and that means comfortable shooting for your wife, as well as ability to carry an effective self-defense weapon, I highly recommend a moderate weight .357 Magnum 5 shot revolver with a 3" barrel, such as a S&W model 60 (link below). You and your wife can shoot 38 special wadcutters for practice and load it with 38 special +P or 357 magnum hollow points for self-defense. Yeah, I know it's not as cool looking as the popular semi-auto, but it's effective, very easy to shoot, accurate, and versatile in ways that a 9mm (or 40S&W) isn't.