I think that you're doing great so far mate - beardies are pretty hardy and it takes a lot to get em down.
Just keep an eye on him and see how he goes. Get them all sexed if you can and if you DO have 4 males, (if it wouldn't break your heart to do so) maybe look at just keeping the ones you like the best

That or, in my experience (and according to just about all the info you'll find on beardies anywhere), you might have to look at building a few extra tanks and keeping one male in each one...
Good luck - and with beardies, your best indication of how they're feeling is whether or not they're eating, so as long as he doesn't go off his food, I reckon you should be right

Bex