Mill finish should be alright, as long as you thoroughly clean it first.
We just did a dumpster with new steel on it and the paint shop forgot to clean the sheets and everything was done from welding to priming and painting, but when we put our letters on..... it completely took A L L of the paint right off.