Testing Spring Boot application in post-JUnit 4 world

Testing Spring Boot application in post-JUnit
4 world

About myself
● Application Architect
@Thomas Cook/Ciklum
● Tech Lead - Platformers
team
● +9 years Java

Agenda
1. What problems we are facing with legacy testing frameworks;
2. Alternative frameworks - Spock and JUnit 5;
3. Spring Boot testing support under the hood;
4. Demo

Mule -> Spring Boot
TestNG + JUnit 4

TestNG issues
Test class lifecycle:
@BeforeMethod
private void beforeMethod() {
objectUnderTest = null;
MockitoAnnotations.initMocks(this);
….
}

TestNG issues
Test class lifecycle:
@BeforeMethod
private void beforeMethod() {
objectUnderTest = null;
MockitoAnnotations.initMocks(this);
….
}
*Funnily enough you can use the same lifecycle for JUnit 5 with
@TestInstance: enum Lifecycle PER_CLASS, PER_METHOD;

Issues with JUnit 4
Not possible to have several @RunWith(imagine you want
MockitoJUnitRunner and SpringRunner)

Spock: Overview
Modules
● spock-core
● spock-specs
● spock-spring
● spock-tapestry
● spock-guice
● …..

Spock: based on JUnit runner
No hassle with the runner: it even extends Junit runner so it can run by
the tools you used for your tests before.
@RunWith(Sputnik.class)
public abstract class Specification extends MockingApi
….
public class Sputnik extends Runner implements Filterable, Sortable

Spock: formal semantics
JUnit tests lack formal semantics
The number one reason for using Spock is to make your tests more
readable.

Spock: test blocks
● setup:
● when:
● then:
● expect:
● cleanup:
● where:

Spock: test example
def "should fetch Bob and Alice without errors"() {
given:
def response =
mockMvc.perform(MockMvcRequestBuilders.get("/bookings/$id")).andReturn().response
def content = new JsonSlurper().parseText(response.contentAsString)
expect:
response.status == OK.value()
and:
content.passengerName == result
where:
id || result
'5' || "Bob"
'15' || "Alice"
}

Spock: Data Pipes and Tables
where:
a | b || c
3 | 5 || 5
7 | 0 || 7
where:
a << [3, 7, 0]
b << [5, 0, 0]
c << [5, 7, 0]

Spock: @Shared fields
For expensive resources:
@Shared res = new VeryExpensiveResource()

Spock: error reporting
Nice and layered: Condition not satisfied:
content.passengerName == result
| | | |
| Bob | Bob1
| false
| 1 difference (75% similarity)
| Bob(-)
| Bob(1)

Spock: Interactions
A way to express which method invocations are expected to occur:
(1..3) * subscriber.receive("hello") // between one and three calls (inclusive)
(1.._) * subscriber.receive("hello") // at least one call
(_..3) * subscriber.receive("hello") // at most three calls

Spock: Interactions - powerful but should be
used at your own risk
1 * subscriber./r.*e/("hello") //method name starts with 'r' and ends in 'e')
1 * subscriber.receive(_) // any single argument (including null)
1 * subscriber.receive(*_) // any argument list (incl. the empty argument
list)
1 * _._ // any method call on any mock object
1 * _ // shortcut for and preferred over the above

Spock Drawbacks: A bit scary versioning

Spock Drawbacks: new language
If you start writing tests for more than just basic CRUD APIs, you’ll
have to learn it.

Groovy “gems”
Everything associated with dynamic typing/duck typing should be treated
with extra care:
https://stackoverflow.com/questions/9072307/copy-groovy-class-properties/9072974#9072974

Groovy “gems”
https://stackoverflow.com/questions/46952475/copy-object-properties-to-another-object-in-groovy

Groovy “gems”
Comments/Answers:

JUnit 5: module composition
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

JUnit 5: native support in Spring 5.0
Spring-boot version 1.5.9-RELEASE is based on Spring 4 and the
SpringExtension is only available since Spring 5.

JUnit 5: reason for re-architecture
● JUnit 4 test discovery and execution were tightly coupled.
● JUnit 4 - java 5 compatible - that’s ridiculous!

JUnit 5: powerful @ExtendWith
Only one @RunWith but many @ExtendWith:
@ExtendWith(SpringExtension.class)

JUnit 5: asserts are never skipped
Assert are not skipped if one failed:
void getBooking() throws Exception {
String bob = mockMvc.perform(MockMvcRequestBuilders.get("/bookings/5"))
.andReturn().getResponse().getContentAsString();
ObjectMapper objectMapper = new ObjectMapper();
assertAll("Analyzing Bob's record",
() -> assertEquals("NotBob", objectMapper.readValue(bob,
Booking.class).getPassengerName()),
() -> assertEquals("USA", objectMapper.readValue(bob, Booking.class).getDestination()),
() -> assertEquals(Double.valueOf(35), objectMapper.readValue(bob,
Booking.class).getAge()));
}

Assert are not skipped if one failed:
void getBooking() throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
assertAll("Analyzing Bob's record",
() -> assertEquals("NotBob", objectMapper.readValue(bob,
Booking.class).getPassengerName()),
() -> assertEquals("USA", objectMapper.readValue(bob, Booking.class).getDestination()),
() -> assertEquals(Double.valueOf(35), objectMapper.readValue(bob,
Booking.class).getAge()));
}
Fail
Fail
Pass

Will require fewer iterations to stabilize your code

JUnit 5: out-of-the-box support for
Exceptions
@org.junit.jupiter.api.Test
void testException() {
Executable closureContainingCodeToTest = () -> {throw
new ConcurrentModificationException();};
assertThrows(ConcurrentModificationException.class,
closureContainingCodeToTest);

JUnit 5: Autowiring into a method
@org.junit.jupiter.api.Test
void getBooking(@Autowired MockMvc mockMvc) throws Exception {

JUnit 5: Autowiring into a method
No need to create loads of fields in your test class. Inject
what is needed to a specific test method.

JUnit 5: Missing fixed method execution order

Spring Boot Testing: Overview
Spring Boot BOM specifies following test-related
dependencies:
● spring-boot-test
● spring-boot-test-autoconfigure
● spring-boot-starter-test
● solr-test-framework
● groovy-test
● groovy-testng
● hibernate-testing
● spring-batch-test
● spring-kafka-test
● spring-ldap-test
● ...

Spring Boot Testing: Overview
Only one test-specific starter:

Spring Test: Integration Testing
Spring integration testing support has the following primary goals:
● To manage Spring IoC container caching between test execution.
● To provide Dependency Injection of test fixture instances.
● To provide transaction management appropriate to integration
testing.
● To supply Spring-specific base classes that assist developers in
writing integration tests.

Spring Test: how are transactions handled?
By default, the framework will create and roll back a transaction for each
test.
If a test method deletes the contents of selected tables while running
within the transaction managed for the test, the transaction will rollback by
default, and the database will return to its state prior to execution of
the test.

test.
the test.
*More flexibility is available via @Commit and @Rollback

test.
the test.
*Not always that straightforward. Probably depends on implicit
commits and DB engines. Will also only work for MOCK transport.

Spring Test: main actors
TestContextManager TestContext
TestExecutionListeners

Spring Test: TestContextManager
Main entry point into Spring test related functionality
Spring
Test
Context

Spring Test: factories for ExecutionListener
org.springframework.test.context.TestExecutionListener =
org.springframework.test.context.web.ServletTestExecutionListener,
org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener,
org.springframework.test.context.support.DependencyInjectionTestExecutionListener,
org.springframework.test.context.support.DirtiesContextTestExecutionListener,
org.springframework.test.context.transaction.TransactionalTestExecutionListener,
org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener

Spring Test: factories for ExecutionListener
Will scan your test class, look at its annotations and add its logic
accordingly.
org.springframework.test.context.TestExecutionListener =
org.springframework.test.context.web.ServletTestExecutionListener,
org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener,
org.springframework.test.context.support.DependencyInjectionTestExecutionListener,
org.springframework.test.context.support.DirtiesContextTestExecutionListener,
org.springframework.test.context.transaction.TransactionalTestExecutionListener,
org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener

Spring Test: static cache
The Spring TestContext framework stores application contexts in a static
cache. This means that the context is literally stored in a static variable. In
other words, if tests execute in separate processes the static cache
will be cleared between each test execution, and this will effectively
disable the caching mechanism.

Spring Boot + Spock
Detached mocks via the DetachedMockFactory and
SpockMockFactoryBean classes.
class TestConfigurationForSpock {
private final detachedMockFactory = new DetachedMockFactory()
@Bean
BookingService bookingService() {
detachedMockFactory.Mock(BookingService);
}
}
Then just use: @Import(TestConfigurationForSpock)

Spring Boot + Spock: Compilation/Build
caveat
GMavenPlus plugin is far from being perfect.
Check https://github.com/groovy/GMavenPlus/issues/ for issues reports.

Spring Boot + JUnit 5
Will need to include an additional library to use JUnit 5 with Spring
Framework 4.3

There’s nothing at all Spock specific in spring-test, but you can find
junit.jupiter is there:

Be careful with your surefire version:

Spring Boot + JUnit 5: Compilation/Build
caveat
Not a single jar any more. If you can’t migrate all tests at once you will
need to have both junit 4 and junit 5 on your test classpath.

Spock vs JUnit 5: integration with Spring
Both frameworks provide a class with exact same name:
SpringExtension.
However, their name is the only thing they have in common.

Spock vs JUnit 5: integration with Spring
SpringExtension
Spock JUnit 5
Tries to find any Spring-specific annotation on a test
class(spec): ContentHierarchy, BootstrapWith,
ContextConfiguration.
Using Spring-provided MetaAnnotaionUtils, so that it
can traverse class hierarchy.
Spring Test SpringExtention implements all possible
JUnit 5 callbacks(.e.g.: BeforeAllCallback,
AfterAllCallback)
If found, creates a TestContextManager and
delegate to it.
Will either get TestContextManager from a store of
create one.
Attaches Spock-specific listener for managing
Mocks:
testContext.setAttribute(MOCKED_BEANS_LIST,
mockedBeans);
Will wrap TestContextManager calls and delegate to
it. E.g.:
public void beforeAll(ExtensionContext context) {
getTestContextManager(context).beforeTestClass();

Spring Boot: autoconfigure slicing
The spring-boot-test-autoconfigure module includes a number of
annotations that can be used to automatically configure different ‘slices’ of
your app for testing purposes.
Examples:
● @WebMvcTest
● @JsonTest
● @DataJpaTest
● @JdbcTest
● @DataMongoTest
● ...

@*Test would normally contain several @AutoConfigure* annotations:
For example:
@BootstrapWith(SpringBootTestContextBootstrapper.class)
@OverrideAutoConfiguration(enabled = false)
@TypeExcludeFilters(DataJpaTypeExcludeFilter.class)
@Transactional
@AutoConfigureCache
@AutoConfigureDataJpa
@AutoConfigureTestDatabase
@AutoConfigureTestEntityManager
@ImportAutoConfiguration
public @interface DataJpaTest

To tweak @*Test mechanism, you can use a corresponding
@AutoConfigure* annotation.
For example:
@RunWith(SpringRunner.class)
@DataJpaTest
@AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
public class ExampleRepositoryTests {
// ...
}

Have you noticed this relatively unremarkable meta annotation:
@OverrideAutoConfiguration(enabled = false)
It will effectively set:
spring.boot.enableautoconfiguration=false
By applying ContextCustomizer to ContextLoader

String Boot: Bootstrapping
@BootstrapWith is used on @SpringBootTest, for example.

@BootstrapWith is used on @SpringBootTest, for example.
● Aware of Spring Boot
application structure;
● Helps SpringBootTest;

As TestContextManager is the main entry point for test frameworks; all they
need is to create one:
public TestContextManager(TestContextBootstrapper testContextBootstrapper) {
this.testContext = testContextBootstrapper.buildTestContext();
registerTestExecutionListeners(testContextBootstrapper.getTestExecutionList
eners());
}

Spring Boot: Slicing
Do not litter the application’s main class with configuration settings that
are specific to a particular area of its functionality.
Extract them into specific @Configuration instead.
@SpringBootApplication
@EnableBatchProcessing - DO NOT DO IT THIS WAY
public class SampleApplication { ... }

Spring Boot: Custom component scanning
If your application may resemble the following code:
@SpringBootApplication
@ComponentScan({ "com.example.app", "org.acme.another" }) - ALSO BAD
public class SampleApplication { ... }
This effectively overrides the default component scan directive with the
side effect of scanning those two packages regardless of the slice that
you’ve chosen.

Spring Boot: TypeExcludeFilter
AutoConfigurationExcludeFilter - tells Spring Boot to exclude scanning
autoconfigurations. To use SpringFactoriesLoader instead.
TypeExcludeFilter - is an interesting case. While it’s in spring-boot jar, the doc
actually says: primarily used internally to support spring-boot-test.
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@SpringBootConfiguration
@EnableAutoConfiguration
@ComponentScan(excludeFilters = {
@Filter(type = FilterType.CUSTOM, classes = TypeExcludeFilter.class),
@Filter(type = FilterType.CUSTOM, classes = AutoConfigurationExcludeFilter.class) })

Pivotal guys care about your application’s tests so much, they put a
test-specific logic into their main Spring Boot module!
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@SpringBootConfiguration
@EnableAutoConfiguration
@ComponentScan(excludeFilters = {
@Filter(type = FilterType.CUSTOM, classes = TypeExcludeFilter.class),
@Filter(type = FilterType.CUSTOM, classes = AutoConfigurationExcludeFilter.class) })

Indeed, all subclasses seem to be coming from spring-boot-test-
autoconfigure and spring-boot-test:

@TestConfiguration :
● Unlike a nested @Configuration class which would be used instead of
your application’s primary configuration, a nested @TestConfiguration
class will be used in addition to your application’s primary
configuration;
● When placed on a top-level class, @TestConfiguration indicates that
classes in src/test/java should not be picked up by scanning. Use
explicit @Import to use them.
Spring Boot: overriding configurations

Spring Boot: Mocking and spying beans
Spring Boot includes a @MockBean annotation that can be used to define
a Mockito mock for a bean inside your ApplicationContext.
@RunWith(SpringRunner.class)
@SpringBootTest
public class MyTests {
@MockBean
private RemoteService remoteService;

Spring Boot: AutoConfigureMockMvc
Enables you to use MockMvc for any WebEnvironment(MOCK,
RANDOM_PORT, etc.)

Spring Boot: WebMvcTest
@WebMvcTest
Only web layer is instantiated, not the whole context.
Often @WebMvcTest will be limited to a single controller and used in
combination with @MockBean to provide mock implementations for
required collaborators.

Spring Boot: @ImportAutoConfiguration
@EnableAutoConfiguration - will scan spring.factories for itself.
@ImportAutoConfiguration - will scan spring.factories for the annotated
class entry.

Spring Boot: WebMvcTest vs
AutoConfigureMockMvc
You’ll notice that both WebMvcTest and AutoConfigureMockMvc have a
@ImportAutoConfiguration, but there’s no entry in spring.factories for
WebMvcTest. By design it allows for extensibility. You can add your
own WebMvcTest entry.
In general, WebMvcTest has a lot more configurations but also contains
exclude filter to narrow down it’s search to only web-related components.

Spring Test: Reactive support
The WebTestClient builds on the mock request and response to provide
support for testing WebFlux applications without an HTTP server.

Testing reactive apps: same concepts with
different names
Non-reactive Reactive
@SpringBootTest @SpringBootTest
@WebMvcTest @WebFluxTest
@AutoConfigureMockMvc @AutoConfigureWebTestClient

If you want to understand your reactive code
inside out:
@Test
public void testSteppingThroughFlux() {
Flux<Long> flux = Flux.just(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L);
StepVerifier.create(flux)
.expectNext(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L)
.expectComplete()
.verify();

Spring Boot: SpringBootTest
RANDOM_PORT WebEnvironment
@SpringBootTest(classes = RamlBasedProducerApplication, webEnvironment =
SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureMockMvc
class BookingServiceImplSpockTest extends Specification {
@LocalServerPort
private int port;
@Autowired
MockMvc mockMvc;

Spring Boot: Test utilities
● EnvironmentTestUtils(addEnvironment(env, "org=Spring",
"name=Boot"))
● TestRestTemplate(behave in a test-friendly way by not throwing
exceptions on server-side errors)
● MockRestServiceServer(part of Spring Test)

Demo
Let’s take a look at a set of CRUD tests and re-write them
using JUnit 5, Spock and different Spring Boot/Spring
Framework test features.
https://github.com/yuranos/raml-based-producer-application
https://github.com/yuranos/raml-based-consumer-application
(unit_testing branch)

Thank you!
Mail: yura.v.nosenko@gmail.com
Linkedin: https://www.linkedin.com/in/yuranos/

Testing Spring Boot application in post-JUnit 4 world

In this document