We have our Github set up to trigger automated testing using Jenkins and deliver back the test outcome; this works fine.
However, our Github testing is not a simple build, it is a build and test on real hardware with real third-party entities (servers, radio connection, etc.). We are now at the point where the most common cause of failure is the third party, hence the test failures are always curated by someone who knows what they’re looking at to determine whether the cause of failure is “us” or “them”.
My problem is that all one ever sees from a Github perspective in the failure case is the big red X. Is there an easy way of controlling this, making it turn into a nice green tick under moderation? FYI, the test infrastructure is all internal, so clicking on the X would always result in “404 not found” for our public users; we can’t put a “don’t worry” indication behind there.