Webhooks and SSL verification #23434
-
We’re running Jenkins with HTTPS acces using a certificate from Let’s Encrypt. The following procedure was used to generate and install the certificate: https://github.com/hughperkins/howto-jenkins-ssl/blob/master/letsencrypt.md Connecting from my browser works as it should, the browser indicates the connection is secured and verified by Let’s Encrypt. After updating the webhook in our GitHub repository from connecting over HTTP to HTTPS, GitHub fails to deliver the webhook payload with the following message: “We couldn’t deliver this payload: Peer certificate cannot be authenticated with given CA certificates” How come the certificate works in my browser, but doesn’t work for webhooks? |
Beta Was this translation helpful? Give feedback.
Replies: 8 comments
-
All HTTPS certificates are issued by some entity. These certificates are often signed by a Certificate Authority or CA. That signature’s authenticity is verified against the CA’s certificate which could also be signed by another CA. This can continue in a chain until you either reach the end of the chain or reach certificate signed by a CA that the system implicitly trusts (also known as a “root CA”). For example, you can see the list of root CAs recognized by your system on macOS by using the Keychain Access app under the System Roots keychain: Different systems can have different lists of trusted root CAs. This is why your browser and our webhook system can have different results when attempting to create an HTTPS connection to the same system. On the other hand, we haven’t had any reports of Let’s Encrypt certificates not being honored by our webhook system. You may want to ensure that your system is serving the entire certificate chain when establishing HTTPS connections. If you continue to run into problems, you may want to write into private support at https://github.com/contact so they can take a deeper look at your configuration. I hope that helps! |
Beta Was this translation helpful? Give feedback.
-
Were you able to determine if this was this issue? |
Beta Was this translation helpful? Give feedback.
-
I still haven’t been able to determine what is the issue. https://whatsmychaincert.com/ indicates my server is misconfigured and provides a chain it should be using. The suggested chain only contains Let’s Encrypt Authority X3 , whereas the If this persists I will write into private support. |
Beta Was this translation helpful? Give feedback.
-
Ah, interesting, I’m having the exact same issue! We’re using fullchain.pem and if we use the suggested chain neither the browser nor GitHub successfully connect over HTTPS. Not sure how to proceed at this point. If you do make progress through private support, please update here if possible. |
Beta Was this translation helpful? Give feedback.
-
We solved this issue by starting Jenkins using the two flags: --httpsKeyStore and --httpsKeyStorePassword instead of --httpsCertificate and --httpsPrivateKey. In the Jenkins startup logs it mentions that httpsKeyStore is the preferred method. It seems that using the old flags Jenkins was not serving the correct certificate chain. You can generate the keystore using: openssl pkcs12 -inkey privkey.pem -in fullchain.pem -export -out keys.pkcs12 After starting jenkins using this generated keystore we no longer had issues with github webhooks (also https://whatsmychaincert.com showed that the correct chain was being served) |
Beta Was this translation helpful? Give feedback.
-
This solution works great. Thank you! Note that Jenkins won’t start when generating the keys.pkcs12 and keystore files using two different passwords. Using the same password for both files solves this problem. |
Beta Was this translation helpful? Give feedback.
-
Someone posted a solution to this problem, which worked for me. You may want to give it a try. Thanks for your help :slight_smile: |
Beta Was this translation helpful? Give feedback.
-
Thanks to give solustion but how when using jenkins in kubernetes any clue running customer jenkins with option --httpsKeyStore and --httpsKeyStorePassword |
Beta Was this translation helpful? Give feedback.
We solved this issue by starting Jenkins using the two flags: --httpsKeyStore and --httpsKeyStorePassword instead of --httpsCertificate and --httpsPrivateKey.
In the Jenkins startup logs it mentions that httpsKeyStore is the preferred method. It seems that using the old flags Jenkins was not serving the correct certificate chain.
You can generate the keystore using:
openssl pkcs12 -inkey privkey.pem -in fullchain.pem -export -out keys.pkcs12
keytool -importkeystore -srckeystore keys.pkcs12 -srcstoretype pkcs12 -destkeystore keystore
After starting jenkins using this generated keystore we no longer had issues with github webhooks (also https://whatsmychaincert.com showed that the correct c…