I don't think it matters. It should just be communicating with your X server. Some graphics effects you have enabled with Gnome or KDE could cause problems or slow down in performance.