The AbstractConsumerTest.consumeMessages method can sometimes fail when a test server is under load because the request to the REST Proxy returns fewer messages than expected. Even if messages have been produced with acks=all, this code assumes that the consumer will fetch them all promptly enough to return them. This causes transient errors in the tests which, unsurprisingly, occur more frequently on more heavily loaded machines, e.g. Jenkins servers running multiple builds concurrently.

For example, ConsumerBinaryTest.testConsumeTimeout produces some messages, consumes them, then validates that a subsequent consume call will reach the timeout. However, you'll sometimes see the first step to consume messages fail because it returns fewer messages than expected.

java.lang.AssertionError: expected:<4> but was:<3>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:834)
    at org.junit.Assert.assertEquals(Assert.java:645)
    at org.junit.Assert.assertEquals(Assert.java:631)
    at io.confluent.kafkarest.integration.AbstractConsumerTest.consumeMessages(AbstractConsumerTest.java:271)
    at io.confluent.kafkarest.integration.ConsumerBinaryTest.testConsumeTimeout(ConsumerBinaryTest.java:132)