Try setting up ssh client with compression and use arcfour/blowfish encryption instead. Also avoid ipv6 lookup and reuse connections using
socket:
Add below to ~/.ssh/config
Host *
Ciphers arcfour,blowfish-cbc
Compression yes
AddressFamily inet
ControlMaster auto
ControlPath ~/.ssh/socket-%r@%h:%p