I think this is what happened in Infinity War.

They were having some fun while standing guard outside.

And Steve knew nothing about it.

#sambucky