2 builds in pipeline - against PR and branch

There are 2 builds in pipeline - against PR and branch. This then sets the PR status after a result of 2 checks. This causes the PR build too much time to get green/red and it seems to be duplication of builds, which can take up to ~40mins-1hr per one build, resulting the queue of builds to build.
Can branch build be removed from pipeline to reduce the time?

Also, please confirm if the below is true : ??
When we have a branch and PR for that branch in the ‘same repo’ -
GitHub fires the webhook twice :
– 1st call when you push to branch and
– 2nd call when PR for that branch is synchronized.

If the above is true – Is my issue of 2 builds a result of this behavior ??