I have 2 silver Apple TV remotes as I have 2 Apple TV's. The device upstairs gets no use, so I have bought the 2nd remote downstairs so I can use it with my Mac Mini which I want to run Plex on, as I can't get the Samsung Plex App to work at all and the PS4 App wants me to sign up for something I don't want!! 
So in simple terms I want:
1 x Apple Remote to work with 1 x Apple TV
1 x Apple Remote to work with 1 x Mac Mini
A simple enough request I believe....
So I un-pair 1 remote from the Apple TV and pair it to the Mac Mini. That seems to work....for about 2 minutes when all of a sudden, no matter how many times I un-pair said remote, it wants to keep controlling the Apple TV.
Now I am aware that you must have at least 1 Apple Remote paired, which I have, this is the one that I bought down from upstairs. That works fine with the Apple TV, as does the TV remote which I have also programmed.
The challenge, should you chose to accept it....
How the **** can I un-pair the Apple Remote I wish to use ONLY with the Mac Mini from the Apple TV because I have tried for 1 hour tonight an no matter what after a period of a few minutes on comes the Apple TV and it starts to control it again.
What am I missing?

So in simple terms I want:
1 x Apple Remote to work with 1 x Apple TV
1 x Apple Remote to work with 1 x Mac Mini
A simple enough request I believe....
So I un-pair 1 remote from the Apple TV and pair it to the Mac Mini. That seems to work....for about 2 minutes when all of a sudden, no matter how many times I un-pair said remote, it wants to keep controlling the Apple TV.
Now I am aware that you must have at least 1 Apple Remote paired, which I have, this is the one that I bought down from upstairs. That works fine with the Apple TV, as does the TV remote which I have also programmed.
The challenge, should you chose to accept it....
How the **** can I un-pair the Apple Remote I wish to use ONLY with the Mac Mini from the Apple TV because I have tried for 1 hour tonight an no matter what after a period of a few minutes on comes the Apple TV and it starts to control it again.
What am I missing?