It is possible to speed up ssh connections from the same host by using ControlMaster. What we need is establish a ControlMaster connection to the host and all the subsequent connection from the same host will be allowed with out prompting for passwords.
The configuration required is simple. Just create a file ~/.ssh/config and add the contents provided below.
vi ~/.ssh/config
Host *
ControlPath ~/.ssh/master-%r@%h:%p
ControlMaster auto
For more information refer man page
man 5 ssh_config