Is there a different understanding of “rubber” in British and American English?

Anyway, when talking to British people do I really have to expect that
they don’t know what I’m talking about when using the word rubber
instead of condom?

Yes. That definition of rubber is not really used in the UK. Some people would know it from personal experiences of Americans. Like me. I know it because I have met some Americans in the UK and this word came up in the context of meeting one of them: in primary school, there was an American pupil in the last year. This definition would not generally, be reliable in the UK.

And when talking to American people, should I avoid rubber and use
eraser instead whenever I expect it might sound ambiguous? To be more
concrete: the OALD designates rubber (as eraser) as British English,
thus do I have to suspect that an American may not be aware of that
meaning?

That would be the most reliable approach. As you can see from this discussion, there are some Americans who are aware of the British meaning of rubber but it is not used in the USA. Because of this, it is unreliable in the USA and many Americans might not know this meaning. When talking to Americans, it would be best to use the American word, eraser.

Finally, when talking to both an American and a British at the same
time, which words do I use for condom and eraser so that both will
understand me correctly?

For a condom, just use the word condom. This is the proper word used in the UK and the USA. As the following links show, using the word rubber to mean a condom, is American slang:

US slang for a condom (Cambridge Dictionaries Online)

[countable] American English informal a condom (Longman Dictionary)

And the USA has this slang for the word, as well as the word itself.

For the word eraser, there will have to be a different kind of answer. This is an example of, one of the many differences in American English. Because the word eraser is not generally, reliable in the UK, it is not certain that any, particular British person, will know its American meaning. Whether or not it will be understood, will depend on which, particular British people you talk to.

If you talk to an American person and a British person at the same time, just use the word condom. It is understood in both countries.

Regarding eraser, you have a few possibilities. If you find out first, that one of the two knows the word used in the other’s country; you can use a sentence that has just that word. For example, if you knew that the American person knows the British meaning of rubber, you can use just that. It could be that you are writing with a pencil and want to change some of the writing. You could ask, “have you got a rubber?”. If you don’t know in advance whether one of the two knows the word used in the other’s country; you would need to use a sentence that covers both words. For example, you could ask “have you got a rubber/eraser?”. You could also find out by asking, when you talk to them.