The brown proves that once they are yellow from photolytic breakdown of the fire retardants, they subsequently oxidize, with the final color being a medium brown. It makes sense. Yellow chromophores almost universally become brown as a result of subsequent oxidation. That would make heat your worst enemy once they have turned yellow, since that would accelerate the oxidative process. The good thing is that we can be sure the final color is brown, oxidation should reduce the chromophore to its base energy state, where it doesn't respond to any further processes.