Remove this Socket from the collection. Does nothing if the socket is not in the collection already.