For example:
> it is obvious that XLNet always learns more dependency pairs given the same target and contains “denser” effective training signals
BERT is only masking 15% of the tokens, so isn't the amount of dependency pairs like 18% higher at most?
That's a small but significant difference, allowing them to improve performance by a small but significant amount.
For example:
> it is obvious that XLNet always learns more dependency pairs given the same target and contains “denser” effective training signals
BERT is only masking 15% of the tokens, so isn't the amount of dependency pairs like 18% higher at most?