So if I want to get a VPS for trading the ES and NT is my broker, Dorman is my clearing firm. Do I want to be close to the Dorman's server or Ninja Traders server and not necessarily the Globex server?

You can check yourself: to which servers is NinjaTrader process connected to?

It's connected to Continuum/CQG servers (and to few other things not interesting here, NT license server, NT historical data servers, ...), not to the FCM and not to the CME or any other exchanges servers.