I guess, here's an example I had. I was terrified of talking to my T about his upcoming (and now past) vacation of 2 weeks. I told him I was so afraid that while talking about my fear, that I would be asking him, in a roundabout way, to not go on vacation. So he asks me "what do you think I'd say if you asked me not to go on vacation?" He said that if I asked him not to go on vacation, he would tell me, basically, that he enjoys his vacations and that he will be going on vacation. He never said anything about me not being allowed to ask him not to go on vacation. I guess that's why what you describe bothers me. You should be able to ask anything and he should be able to reply.
I don't think I have any good advice. The way you write, it sounds like you two are not connecting the way you need to in order to work on your feelings. Maybe you can negotiate and find a way to work on these feelings together without the constant "threat" of being transferred to another therapist.