Not all mounts are adaptable to provide infinity focus. I'm not familiar with the Vivitar mount, but Nikon has quite a long flange distance to begin, so very few other mounts adapt well to it. Adapters are available with optics to compensate for incompatible flange distances, but unfortunately, the market has yet to produce any of these with a reputation for quality.
At the very least, you should be able to get some pretty crazy macro shots with the adapter you've got!