from
http://www.askmoses.com/en/article/287,2470885/Why-do-we-say-the-words-Baruch-Shem-kevod-malchuto-leolam-vaed-Blessed-be-the-Name-of-the-glory-of-His-kingdom-forever-and-ever-quietly-after-the-Shema-prayer.html"The Sages taught that the forefather Jacob wanted to prophetically disclose the date of the final redemption to his children, but the Divine Spirit departed from him, and he could not reveal this date to them. Jacob became concerned that the reason for the Divine Spirit's departure was due to a spiritual failing in one of his children. His children all answered with the verse "Shema…" in order to declare to their father, "Listen to us 'Israel' (Jacob's alternate name) our father. Just as you only have the One G-d in your heart, so too, we all have faith in the One in our hearts." Jacob then replied, "Baruch Shem…" thanking G-d for his children's unwavering faith. The Sages decided that since this verse was not one written in the Torah by Moses, but rather recited by Jacob, it should be said quietly in our prayers.
A second opinion is that Moses heard this prayer recited by the angels and then 'borrowed' is for us to use in prayer. Since this verse was not originally our 'own' we say it in a modest undertone during the year. However, on Yom Kippur, when we are likened to the angels, we can recite "Baruch Shem" out loud just as the angels do."
Seems like according to the first opinion it's only recited quietly by shema. And according to the second opinion it should be always.