Based on the realistic background of the increasing floating population, this study takes the agricultural regional public brand as an example to explore whether and how the regional cultural distinctiveness perceived by sojourners affect the brand of the residence negatively, and what is the possible mechanism to reduce this negative impact. The experimental data from Zhijiang and Chongqing show that when participants perceive a great regional cultural distinctiveness, their self-brand connection with the agricultural regional public brand in the residence will be reduced, comparing with those who perceive little regional cultural distinctiveness. This effect is achieved through group identity. The greater the distinctiveness of regional culture perceived by the sojourners, the lower their senses of group identity to the local people will be, and the weaker their self-brand connection with the brand of the residence will be. However, the social acceptance of the sojourners can adjust the influence of regional cultural distinctiveness on sojourners’ self-brand connection of agricultural regional public brand. Under the circumstance that social acceptance exists, the negative effect of regional cultural distinctiveness on self-brand connection with the agricultural regional public brand is weakened.