Self hosted runner with a proxy

I registered a self hosted runner with a proxy. The runner is able to connect to github. When i run the action workflow, i get an error saying that Failed to download action, connection attempt failed because the connected party did not respond after a period of time. The self hosted runner doesnt have internet access but it is able to connect to github when i was registering the runner. For the action workflow to run should i whitelist any github ips or urls as well