Will you sell it direct to them at retail or wholesale?
The biggest question here is, are they selling just the inputs of production(blank shirts for example) at wholesale to joe blow or are they selling the input at wholesale plus the extra value added into the product(embroidery for example).
I wouldn't call them direct competitors unless you are also selling the blank product to customers. If you are putting extra value into the shirt, they aren't in direct competition. You might have more people coming in with their own shirt and you might be doing embroidery that way and not making anything on the shirt as well, but you are still bringing in a value to the shirt that they don't do(for whatever reason).
Now, if you are just taking what you buy from your supplier and then selling it as is with a markup, then I could understand.