Yes, as documented ...
And even if CAST were allowed behind ON (which it is not), it wouldn't work because you cannot cast CHAR to RAW and vice versa.
As Sabarna Chatterjee said, in cases that are not supported in the ABAP CDS DDL, you can fall back to Native SQL, e.g. using a CDS Table Function implemented natively in AMDP.