I looked up the horsecollar tackle rule:

Horse Collar Tackle 
ARTICLE 15. All players are prohibited from grabbing the inside back collar 
of the shoulder pads or jersey, or the inside collar of the side of the 
shoulder pads or jersey, and immediately pulling the ball carrier down. 

This does not apply to a ball carrier, including a potential passer, who is 
inside the tackle box (Rule 2-34). Note that the tackle box disintegrates 
when the ball leaves it.

This indicates that you can get flagged for a horsecollar by pulling but you have to have the inside of the jersey.

Does anyone interpret this differently?