In this teshuva, Rav Ovadiah Yosef begins by explaining the source for the universally accepted practice of tashlich. Citing the Maharil, he notes that it is based on the Midrash which explains that when Avraham and Yitzchak went to do the akeidah, the Satan tried to stop them by turning into a river and blocking their way. Avraham reacted by calling out to Hashem and saying that if they were to drown in trying to fulfill God's command, then who would be left to publicize the name of Hashem in the world!? Thus, as a way of recalling the merit of this most incredible of religious acts (which Chazal believe took place on Rosh Hashana), we say this tefilla by the banks of a river or stream.
A second reason given is that since kings were anointed by the banks of a river, we as it were "anoint" Hashem on this day that proclaims His dominion over the world.
In terms of whether or not tashlich should be said on Shabbat, Rav Yosef acknowledges that there may be some esoteric reasons for not doing so, but he sides with the view that the main reason to not do so is that people will carry their machzorim to tashlich and thus will violate the prohibition of carrying on Shabbat. As such, if the body of water being used is within the eruv, it would seem permissible to say tashlich even on Shabbat.