Suppose I have two proteins A and B. I know they bind but I don't know the conformation of the complex. There is a region of interest in the surface of protein A, and I have reasons to believe that A and B DO NOT bind through this region. How can I use Rosetta to DISCARD a docking interface?
A side effect of Monte Carlo style calculations is that you really can't do this sort of thing. You are never guaranteed to see all the good conformations, so it's always possible there are good A-patch/B interfaces Rosetta didn't think of. (I'll use patch for the known interesting regions). Rosetta is optimized to find good things, not note the bad ones it is not finding. Another consideration is that the energy function is parameterized on real structures, so considering the energies of a structure you expect to be non-physical is kind of suspect. It depends on why you expect the interface not to bind, of course. For more subtle reasons, structures of bad interfaces are usable, but if it's a reason like "the A patch is actually unstructured and highly dynamic", Rosetta's energy function isn't very useful.
You can make weak arguments by doing global docking (you'll want to read the documentation, but I think the flag is -spin1 -spin2 or similar) - this will attempt to dock random orientations. If you produce a large number of models, [assuming your hypothesis is correct, and that Rosetta's energy function agrees], you will have HIGH-energy, BAD models with the A-patch and B bound, and LOW-energy, GOOD models with B bound elsewhere to A. Physics dictates the high-energy conformations are wrong.
You can try constraining docking to the unwanted patch and demonstrating that all the models that result are poor - but Rosetta does a good job of polishing garbage, so you really need "actually good" conformations to compare to, because Rosetta will desperately try to make constrained docking to the wrong location look good anyway.
A good modeling practice for problems like this is to accept the limitations of your hypothesis. Sometimes you do modeling like this where the results HAVE to confirm your hypothesis, because of the way the code is set up - in those cases, you have to either learn to read a "degree of agreement" (which is basically impossible), or use your "confirming model" as only a pretty picture to help you explain the hypothesis in a structurally specific way, as opposed to a real pillar of support.
To clarify - you already know that the interaction doesn't occur through those regions and you want to tell Rosetta about that information, right? (That is, you're not looking to use Rosetta to confirm the hypothesis that the interaction doesn't occur in this region.)
The best course of action is probably to do a test run first without any sort of biasing. Rosetta might already realize that it's not a good interface an will ignore it without any extra prodding.
If you do want/need to bias your simulations away from that conformation, one possibility is to (ab)use the constraint system to penalize things that are close to the docked conformation that you don't want. There's a number of functional forms (https://www.rosettacommons.org/docs/latest/constraint-file.html#Function...) which can be used with constraints. While most of these are built with the intent that they're going to be used to promote a specific conformation (rather than penalize it), some should be flexible enough to allow you to do what you want.
For example, if you have an atom pair distance that you want to keep greater than 10 Ang, you can use something like
AtomPair CA 54 CA 246 FLAT_HARMONIC 1010.0 1.0 1000.0
Every model with a distance between 10 and 2010 Ang will have a zero penalty, and anything that falls below 10 Ang (or above 2010 Ang) will be penalized with a quadratic penalty forcing it towards 10 Ang (or 2010 Ang). This will fail if you are considering docking conformations where the atom pair distances can be greater than 2010 Ang, but you can fiddle with the number until you get something that will work for your system.
I would recommend testing this before production run - that is, rescore the results from your original non-biased test docking run, and check that the constraint values you see appropriately penalize the conformations you don't want, and don't penalize the conformations you want.