Can I limit the ip(s) that sshd listens on?

Yes, this can be set withing the sshd_config file.

Backup the sshd_config file,edit it with your favorite editor, and change the ListenAddress configuration line:
root@root [~]# cd /etc/ssh/
root@root [/etc/ssh]# cp sshd_config sshd_config.save
root@root [/etc/ssh]# vi sshd_config
root@root [/etc/ssh]# diff sshd_config sshd_config.save
15c15
< ListenAddress 10.0.0.1
—
> #ListenAddress 0.0.0.0
Restart sshd and verify that it is listening on the specified address:
root@root [/etc/ssh]# /etc/init.d/sshd restart
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
root@root [/etc/ssh]# netstat -anp | grep sshd
tcp        0      0 10.0.0.1:22                 0.0.0.0:*                   LISTEN      1718/sshd
If you want sshd to listen on more than one address then add separate ListenAddress lines:
root@root [/etc/ssh]# diff sshd_config sshd_config.save
15,16c15
< ListenAddress 10.0.0.1
< ListenAddress 10.0.0.2
—
> #ListenAddress 0.0.0.0
root@root [/etc/ssh]# /etc/init.d/sshd restart
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
root@root [/etc/ssh]# netstat -anp | grep sshd
tcp        0      0 10.0.0.1:22                 0.0.0.0:*                   LISTEN      1884/sshd
tcp        0      0 10.0.0.2:22                 0.0.0.0:*                   LISTEN      1884/sshd

Yes, this can be set withing the sshd_config file.

Backup the sshd_config file,edit it with your favorite editor, and change the ListenAddress configuration line:
root@root [~]# cd /etc/ssh/
root@root [/etc/ssh]# cp sshd_config sshd_config.save
root@root [/etc/ssh]# vi sshd_config
root@root [/etc/ssh]# diff sshd_config sshd_config.save
15c15
< ListenAddress 10.0.0.1
—
> #ListenAddress 0.0.0.0
Restart sshd and verify that it is listening on the specified address:
root@root [/etc/ssh]# /etc/init.d/sshd restart
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
root@root [/etc/ssh]# netstat -anp | grep sshd
tcp        0      0 10.0.0.1:22                 0.0.0.0:*                   LISTEN      1718/sshd
If you want sshd to listen on more than one address then add separate ListenAddress lines:
root@root [/etc/ssh]# diff sshd_config sshd_config.save
15,16c15
< ListenAddress 10.0.0.1
< ListenAddress 10.0.0.2
—
> #ListenAddress 0.0.0.0
root@root [/etc/ssh]# /etc/init.d/sshd restart
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
root@root [/etc/ssh]# netstat -anp | grep sshd
tcp        0      0 10.0.0.1:22                 0.0.0.0:*                   LISTEN      1884/sshd
tcp        0      0 10.0.0.2:22                 0.0.0.0:*                   LISTEN      1884/sshd