Relative Addressing With XOR - Removing an Adder

Could be an interesting compromise: use XOR instead of ADD for relative addressing to remove an adder. Specifically,

Address = AddressRegister XOR Immediate

Forces the programmer to keep address register on some power of two alignment. With caches and/or parallel access to banked memory, this would be a bad. But likely fine for a core with a private memory, and code written in assembly.