net: lazy listen on handler

This allow the server to be shared without the need to handle connection
from master