IIRC OSX has slow symbol lookup, I wonder if that's what's going on. Wouldn't make X slow, though. Don't take it for granted. File a bug, perhaps.