Rabbi Sorotzkin in "Oznayim LaTorah" discusses the question of why the Kohen Gadol [high priest] should wear gold in the first place, since the gold is a "prosecutor" (reminding Hashem of the golden calf). He brings up two points in Acharei Mos when Rashi uses that as a reason why the Kohen Gadol enters the Kodesh Kadashim [Holy of Holies] with only the four linen garments of the regular priest.
1. Why is he not given four "additional" garments made of linen? For example, the tzitz, the gold plate on his forehead with the words "Kadosh Lashem" on it could have been a linen band with the words embroidered on it. Another example are the gold bells around the Meil [coat or tunic] which could have been made out of a different metal.
2. When he comes back out, he puts the golden garments back on. Shouldn't he leave them off since he is still attempting to atone for Bnei Yisroel? The gold is still a reminder of the Chait HaEgel [Sin of the calf].
He connects this with the actual purpose of the Kohen Gadol. When a person sees someone committing (or who has committed) a sin, he has a mitzvah to rebuke him (without causing emabarrasment. Thus, the purpose of the gold is to actually remind those who see it that they must atone for their sins. When the Kohen Gadol goes into the Kodesh Kodoshim on the other hand, he is there to defend the Bnei Yisroel. He gives the analogy of Rabbi Levi of Berditchev who would defend Bnei Yisroel in his prayers. However, he points out that someone who attempts to tell a rav that he should not rebuke someone for committing a sin is himself acting improperly.
When the Kohen Gadol goes in to the Kodesh Kodoshim, noone else is allowed to be present (not even the malachim [angels] who normally attend him). He is alone with Hashem as the representative of the people and is there to defend them. Not only is the gold improper there, but he is like a "regular" Kohen wearing only the four linen garments.
When he comes out, he is also the representative of Hashem as well as the defender of the people. He is attempting to arouse the people who see him to do teshuvah and to atone and resolve to behave properly in the future. For example, the words on the Tzitz are actually a reminder to the people on how they must behave as well as a why of keeping the Kohen Gadol reminded as to how he must control his thoughts. Similarly, the purpose of the other "extra" garments is to have an effect on those who see the Kohen Gadol.
Rabbi Sorotzkin also points out that the purpose of each individual part of the additional four garments of the Kohen Gadol are not required when he is alone with Hashem. Another example is that the sound of the bells is not required because the censor actually had a piece that would bang on it and make a sound the entire time that the kohen Gadol was in the Kadosh Kodoshim.
May we merit to actually be able to have the Avodah of Yom Kippur be carried out in full bimheirah beyameinu [speedily and in our day].