Liquid nails?
If you don't want to use some sort of adhesive, which will be pretty much permanent, I think you're going to need to frame it, or something close to framing it.
You could mount two mouldings horizontally on the wall, the right distance apart, and of suitable shapes to make a little "shelf" on the bottom and an opposing upside-down shelf above, and then slip the thing in from the side. Essentially building the top and the bottom of a frame, and affixing that to the wall. I am dubious that this would result in anything good long term, though, I think the foam board would eventually sag out. You'll want to support the sides as well, so now you're building a frame anyways.
So, just fabricate or have fabricated a frame, put it into the frame, and hang the frame on the wall!