Docker build action gives 403 error on Push request

When I do a push request on a repo running a github build action, The build always fails with the following error : buildx failed with: error: failed to solve: failed commit on ref "manifest-sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14": unexpected status: 403 Forbidden
However, once the change is merged, a new build is triggered and works properly…
What can the issue be ?

I’m using docker/build-push-action@v2.

full log :

#42 exporting to image
#42 exporting layers done
#42 exporting manifest sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14 done
#42 exporting config sha256:9b0e58bfdd0e4e7ba1eefa34c16d83091e6f16dc05d1e072b927e075cc63f7b6 done
#42 pushing layers
#42 ...

#44 [auth] xxxxxxxx/yyyyyyy:pull,push token for ghcr.io
#44 DONE 0.0s

#45 [auth] xxxxxxxx/yyyyyyy:pull,push token for ghcr.io
#45 DONE 0.0s

#42 exporting to image
#42 pushing layers 0.8s done
#42 pushing manifest for ghcr.io/xxxxxxxx/yyyyyyy:1a51fb2fd1c8e5d6a484ff3d51fba88fe630553f@sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14
#42 pushing manifest for ghcr.io/xxxxxxxx/yyyyyyy:1a51fb2fd1c8e5d6a484ff3d51fba88fe630553f@sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14 0.2s done
#42 ERROR: failed commit on ref "manifest-sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14": unexpected status: 403 Forbidden
------
 > exporting to image:
------
error: failed to solve: failed commit on ref "manifest-sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14": unexpected status: 403 Forbidden
Error: buildx failed with: error: failed to solve: failed commit on ref "manifest-sha256:3cdbf65dc9301e91376c81b09ed39532bc0c2a671c478686b25ad7fa6d15ed14": unexpected status: 403 Forbidden