Seems like you might need to slow down and think about your camera settings as you go, and keep adjusting as needed, especially in challenging or less than ideal conditions. I guess being a longtime film photographer I compare it to film - it would be like shooting 50 rolls of film on a trip (and that's half of the total since there were two of you), which would be a lot of film! LOL even for a few days' of taking pictures.
When I learned photography seems like I learned to not shoot toward the sun if at all possible; and as mentioned the time of day wasn't the best and a lens hood and/or polarizer might have helped. If I'm trying to get a proper exposure in challenging outdoor lighting sometimes I'll lower the camera a little downward to meter then reframe the shot. As John (tirediron) said, the camera is recording light and sometimes you're pushing the limits of what a camera can do. If I'm in tricky lighting too I'll often take more than one shot changing the settings each time to try to get at least one that's exposed properly.