Our airflow implementation pulls code from Github every time is runs a dag (program). We can run 64,000 programs a day, which means we can do 64k clones a day. I know that github will throttle too many clones a day, but when is this going to be an issue for our implementation?
This doesn’t answer my question. Since airflow is cloning thousands of times an hour from the same production branch is this an issue? This is a typical way to use airflow with k8, with a clone occurring for each of the 1000s of pods that airflow spins up (and down) in an hour.