Hello and thanks a lot!

yes, sometimes for very short sentences that do not have enough contextual information the model might give the wrong answer.

On average, performance is coherent with what expected (we have also tested with a bunch of other datasets).