Using the engineering calculator I rely on, a 6x6 treated Yellow Pine post is only 45% utilized for that sign, when set in a 24" diameter hole 38" deep. That's based on our maximum wind of 90 mph. So what you're planning would be good. Personally, for that height of sign I'd take the hole to 48"...