What happens if a patient refuses to see a visitor? No violence, no arguing, just a simple and calm 'nope, not going to happen?'

They can't really carry you in there. If they did use force, you could just respond in kind, and if you didn't back down you'd end up in some kind of solitary confinement...right?