You could get an umbrella box, one that has "double diffusion", meaning the light hits the inside of the umbrella, and is diffused, and that reflected, scrambled light then passes through the front diffusing fabric. The diffusing fabric is closed either by three shirt zippers, or by a sleeve that has a cinch drawstring in it, and the majority of the light unit (SMALL, Alien Bee) goes mostly INSIDE the diffuser panel, and the sleeve is cinched closed, or the zippers are closed; with bigger light units (ancient monolights, for example), only the reflector goes inside the diffuse panel, and the sleeve cinches closes or the zippers zip closed, leaving the rear of the light OUTSIDE the diffusing fabric.
In this type of umbrella box--again,you get DOUBLE diffusing...the light hits the interior of the umbrella, and then you get soft, diffused light that goes thru the front panel.
The Lastolite Umbrella Box is a good example of this...it has eight ribs/eight sided catchlight; there are also 10-ribbed, almost round umbrella boxes (Like the ones Annie Liebovitz uses so,so often). Liebovitz uses the Photek Softlighter; it has been iterated to version II, and has more features now, like changeable inset fabrics in silver or gold.
These umbrella boxes do not cost a lot of money...Made in China ones in 43 inch size are $34.95 each....easy to pack, easy to carry, inexpensive,instant set-up, no speed ring needed, work on standard umbrella shaft mounts. The Lastolite Umbrella Box costs more than the non-name Made in China ones; the Photek Soflighter II is more costly; many octaboxes cost "more", per size. Lastolite uses a soft, dull-finished interior that gives a slightly softer light quality (a little bit less specular light) than the cheaper, Made in China types I also own.
The light quality of the Lastolite Umbrella Box is very good; it has a SOFT, DULL white interior, not silvered, not satiny silvery-white, but a DULL, FLAT, matte white vinyl, so the light is very,very soft and low in specularity.
The ONE BIG advantage with these: you can EASILY install an 11-inch reflector inside the umbrella, and get a neat doughnut-with-hole black-dot catchlight at close range...the standard 5" or 7" reflector will not tend to leave this black dot in the center of the catchlight, but the "big" reflectors will tend toward the black-dot-inside-the-white catchlight.