Variable defined in remoteEnv via devcontainer.json are not present

I have a container in which I define an env variable GATSBY_TELEMETRY_DISABLED in prop remoteEnv in devcontainer.json. When I create a container from the config, the env variable is not always present. I cannot reproduce this issue, i.e. when I notice that my instance does not have the env variable, I delete the container and create a new one and then I am usually lucky (not always) and the variable is present. Screenshots show 2 different instances.

I just noticed that when the env variable is present, all I need to do is close the tab and reopen it and then the env variable is gone. I double checked and this setting is set to true

"terminal.integrated.inheritEnv": true

so I am really not sure what is happening.