Are you sure your .ashrc is actually sourced before Xfce is started? I would put the assignment instead in ~/.pam_environment. System wide would be /etc/environment, also read by pam_env. In systemd systems you have other possibilities with environment.d, too. Setting variables on Linux in a consequent way is hard.