We use Behaviour-Driven Development (BDD) to ensure both:
Our tool of choice for BDD is Behat, which has extensions specifically for Drupal, MailHog, and so forth.
The entire test suite ought to always pass locally.
Running tests locally should be as simple as:
make tests # Run app-specific tests
make tests-wip # Run tests for "Work-In-Progress" feature(s)
Dues to vagaries in the CI testing environment, we exclude certain classes of tests from our Continuous Integration suite. For example, it is notoriously difficult to set up JS-enabled browser testing in container-based (Docker) testing environments. So long as such tests are consistently tagged (e.e., @javascript) and pass local testing, this should be acceptable.
Still, we should strive to run our entire test suite in CI.