The "reflex" in the term "Single Lens Reflex" means that the viewfinder sees exactly what the actual lens on the camera sees. That's different from a camera like a rangefinder, or even a cheap disposable, where the viewfinder is a completely separate bit of optics. The mirror is what makes that possible, but shooting the light from the lens into the top of the camera.
Before electronic sensors in digital cameras, that was the only way to have the viewfinder exactly match the image the lens sees, especially since you could mount any of dozens of different lenses (other than the larger Twin Lens Reflex, which uses a pair of identical lenses, one for the viewfinder and one for the film.) Yes, mirrors add a mechanical complexity to the camera, and shooting with them created a certain amount of noise, with the mirror movement and the shutter curtain movement.
Mirrorless cameras replace all that with an electronic display that projects the sensor image to your eye. Mirrorless cameras are mechanically simpler, and generally smaller and lighter.
