There's several easy ways.
#1: Select, select all (ctrl+a), edit, stroke, pick your color, choose inside, and since you are only doing 2 pixels, it's not much. If you want to do a bigger border, then should try #2.
#2: Ctrl + J to duplicate your background layer. Make your background color the color you want for your border. Image, canvas size, check relative, and add the amount you want to each side. I use anywhere from 20 pixels to 250. If you want to change the color later, fill the bottom layer with any color you want.
You can also add a small stroke around the image that is contrasting color to your background. After you've done #2, ctrl+click the layer with your photo, and do #1, only select outside instead of inside.