Testcontainers

Not sure how I hadn't encountered this before, I LOVE this pattern.

I find integration tests that exercise actual databases/Elasticsearch/Redis/Varnish etc to be massively more valuable than traditional unit tests. In the past I've gone to pretty deep lengths to do things like spin up a new Elasticsearch index for the duration of a test suite and spin it down again at the end.

It looks like Testcontainers does all of that work for me.

My testing strategy is to have as much of my application's functionality covered by proper end-to-end integration-style tests as possible - think tests that simulate an incoming HTTP request and then run assertions against the response (and increasingly Playwright-powered browser automation tests for anything with heavy JavaScript).

I'll use unit tests sparingly, just for the bits of my code that have very clear input/output pairs that afford unit testing.

I only use mocks for things that I don't have any chance of controlling - calls to external APIs for example, where I can't control if the API provider will be flaky or not.

Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are fully initialized and ready to accept connections.

Pretty much every project I create now has testcontainers for integration testing :)

I setup CI so it lints, builds, unit tests then integration tests (using testcontainers)

https://github.com/turbolytics/latte/blob/main/.github/workf...

Their language bindings provide nice helper functions for common database operations (like generating a connection uri from a container user)

https://github.com/turbolytics/latte/blob/main/internal/sour...

I use them in $day job use them in side projects use them everywhere :)

Pulling up infra to run unit tests is an anti-pattern. This is a great tool for integration tests, though.

I did not come in here expecting to read such effusive praise for testcontainers. If you’re coming from a place where docker wasn’t really a thing I can see how it looks beautiful. And in a fair amount of use cases it can be really nice. But if you want it to play well with any other containerized workflow, good freaking luck.

Testcontainers is the library that convinced me that shelling out to docker as an abstraction via bash calls embedded in a library is a bad idea. Not because containerization as an abstraction is a bad idea. Rather it’s that having a library that custom shell calls to the docker CLI as part of its core functionality creates problems and complexity as soon as one introduces other containerized workflows. The library has the nasty habit of assuming it’s running on a host machine and nothing else docker related is running, and footguns itself with limitations accordingly. This makes it not much better than some non dockerized library in most cases and oftentimes much much worse.

I didn't quite understand why this was made. We create our local test environments using docker-compose, and so I read:

Creating reliable and fully-initialized service dependencies using raw Docker commands or using Docker Compose requires good knowledge of Docker internals and how to best run specific technologies in a container

This sounds like a <your programming language> abstraction over docker-compose, which lets you define your docker environment without learning the syntax of docker-compose itself. But then

port conflicts, containers not being fully initialized or ready for interactions when the tests start, etc.

means you'd still need a good understanding of docker networking, dependencies, healthchecks to know if your test environment is ready to be used.

Am I missing something? Is this basically change what's starting your docker test containers?

No more need for mocks or complicated environment configurations. Define your test dependencies as code, then simply run your tests and containers will be created and then deleted.

Wait what? They think you don't need unit tests because you can run integration tests with containers?

It's trivial to set up a docker container with one of your dependencies, but starting containers is painful and slow.

Testcontainers aren't even compatible with kubernetes, that's a tool from the past.

I dont understand how this is better than a docker-compose.yml with your dependencies, which plays nicer with all other tooling.

Especially if there are complex dependencies between required containers it seems to be pretty weak in comparison. But i also only used it like 5 years ago, so maybe things are significantly better now.

Unit test suppose to be fast. Especially during coding. I wonder how this is necessary & not affecting the test feedback speed

I looked at testcontainers and ended up rolling my own version. One issue I had is that Docker is a very leaky abstraction. I needed to write one test and have it run in all these scenarios:

- on a Mac

- on a Linux VM

- in a Docker container on a Linux VM, with a Docker socket mounted

The networking for each of these is completely different. I had to make some opinionated choices to get code that could run in all cases. And running inside Docker prevented the test from being able to mount arbitrary files into the test containers, which turns out to be a requirement often. I ended up writing code to build a new image for each container, using ADD to inject files.

I also wanted all the tests to run in parallel and spit out readable logs from every container (properly associated with the correct test).

Not sure if any of these things have changed in testcontainers since I last looked, but these are the things I ran into. It took maybe a month of off and on tweaking, contrary to some people here claiming it can be done in an hour. As always, the devil is in the details.

edit: I did end up stealing ryuk. That thing can’t really be improved upon.

Does this work with podman or is it docker only?

Not everything can be placed in a docker which seem as a requirement.

This looks pretty useful! Question on the nginx container. For my tests, this container is only useful when files are mounted into the container. For example, it needs an nginx.conf passed in. How do I do this with NginxContainer?

I was intrigued to see that they have PyPI packages for a ton of different things - like https://pypi.org/project/testcontainers-postgres

That wheel file is only 2.9KB, so I grabbed a copy to see how it works. I've put the contents in a Gist here: https://gist.github.com/simonw/c53f80a525d573533a730f5f28858...

It's pretty neat - it depends on testcontainers-core, sqlalchemy and psycopg2-binary and then defines a PostgresContainer class which fires up a "postgres:latest" container and provides a helper function for getting the right connection URL.

Its great but I find it harder to debug. And I have to say, I usually dont need it. Typically i just have some command which spins everything up from docker-compose files. I prefer this over putting configuration in code like you often do with test containers. You can also load from docker compose files but at that point the test container API isn’t really doing much.

Its pretty much required when you want to setup/teardown in between tests though. This just usually isnt the case for me.

Forgive the question... but... why can't 'test' containers be 'prod' containers?

You can do something similar with docker compose, driving the system from the outside. Create dockerized versions of dependencies like the database, build and run tests, and then run tests against the production app container. It's particularly useful for testing a set of microservices.

See https://github.com/cogini/phoenix_container_example for a full example. This blog post describes it in detail: https://www.cogini.com/blog/breaking-up-the-monolith-buildin...

Unit tests with real dependencies

That's an integration test. These are integration tests. You're literally testing multiple units (e.g., Redis, and the thing using Redis) to see if they're integrating.

Why do we even have words.

These are valuable in their own right. They're just complicated & often incredibly slow compared to a unit test. Which is why I prefer mocks, too: they're speedy. You just have to get the mock right … and that can be tricky, particularly since some APIs are just woefully underdocumented, or the documentation is just full of lies. But the mocks I've written in the past steadily improve over time. Learn to stop worrying, and love each for what they are.

(Our CI system actually used to pretty much directly support this pattern. Then we moved to Github Actions. GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.)

Somewhat related: anyone here using AWS Neptune graphql database? How do you develop locally against Neptune? Apart from Localstack, is there a way to mock Neptune for local testing and development?

Can I run this in GitHub Actions?

I think it’s a step in the right direction. There’s probably a few uses cases where the dependent system is configured much differently in your “test container” vs production instance. But if the aim is to programmatically spin up dependencies and provide 99% guarantee that your app/workflows will work, then this seems it can do the job.

One small note: test run time will probably increase. If a person has an outdated computer, I suspect they will have a hard time running the IT suite. Especially if it’s a complicated system with more than one dependency.

Integrated this in an afternoon. It was surprisingly simple and works great locally and also inside GitHub actions to do psql integration tests of our core app.

been doing this for years, I would not say this gets rid of testing though.

Running integration tests are significantly more complicated to write and take longer to run.

There is also race conditions present that you need to account for programmatically.. Such as waiting for a db to come up and schema to be applied. Or waiting for a specific event to occur in the daemon.

That being said, this looks like a decent start. One thing that seems to be missing is the ability to tail logs and assert specific marks in the logs. Often you need to do an operation and wait until you see an event.

I found test containers to be slow to startup last year. It wasn’t worth the effort considering how long it took to run compared to traditional spring IT h2 hibernate.

I read through the docs and am still confused about what this actually does beyond running a single docker run command in the background and returning control to your code when the container is up.

About 7 years ago I wrote conex, which is basically testcontainers for Go, with first class integration with the Go's official testing framework:

https://github.com/omeid/conex

Each test gets a fresh, clean instance of the browser, without having to worry about variations in plugins or required updates.

Except where everyone is saying that's too slow and instead they have a long-lived instance which they manually teardown each time. That's even what the examples do (some, at least, I didn't check them all).

If you've already bought into the container world then why not embrace a few more. For everyone else, not sure there's much point in extra complexity (they call it simplicity) or bloat.

Wow ... the syntax reminds me so much of aspire (microsoft new "composer"-syntax). Makes a lot of sense.

Why not keep this information in code .. often the developers are ending up doing those task anyway. (not recommended .. but seen it so many times)

Link: Microsoft aspire (https://learn.microsoft.com/en-us/dotnet/aspire/get-started/...)

This is a very nice project, but it's awful that they're blurring the line of unit test and integration test. They are very different and both very important.

Things in the software world are very trendy. If this starts a trend of making people think that they're writing unit tests when they are writing integrations tests, we are fucked.

If I need to change code that you wrote I need a lightning fast way to figure out that I haven't broken your code according to the tests that you wrote. That's unit tests.

My changes might break the whole system. That's integration tests. I just to run that once and then I can go back to unit tests while I fix the mess I've made.

My team maintain a lot of flink connectors, We've changed external test resources to testcontainer as much as possible, it makes things simple and saves money as well.

I never really liked testcontainers. Too complicated. And I don't want to have my tests make too many assumptions about what level of control there is over their environment. IMHO it's just the wrong place to be messing with docker.

I don't like layering abstractions on top of abstractions that were fine to begin with. Docker-compose is pretty much perfect for the job. An added complexity is that the before/after semantics of the test suite in things like JUnit are a bit handwavy and hard to control. Unlike testng, there's no @BeforeSuite (which is really what you want). The @BeforeAll that junit has is actually too late in the process to be messing around with docker. And more importantly, if I'm developing, I don't want my docker containers to be wasting time restarting in between tests. That's 20-30 seconds I don't want to add on top of the already lengthy runtime of compiling/building, firing up Spring and letting it do it's thing before my test runs in about 1-2 seconds.

All this is trivially solved by doing docker stuff at the right time: before your test process starts.

So, I do that using good old docker compose and a simple gradle plugin that calls it before our tests run and then again to shut it down right after. If it's already running (it simply probes the port) it skips the startup and shut down sequence and just leaves it running. It's not perfect but it's very simple. I have docker-compose up most of my working day. Sometimes for days on end. My tests don't have to wait for it to come up because it's already up. On CI (github actions), gradle starts docker compose, waits for it to come up, runs the tests, and then shuts it down.

This has another big advantage that the process of running a standalone development server for manual testing, running our integration tests, and running our production server are very similar. Exactly the same actually; the only difference configuration and some light bootstrapping logic (schema creation). Configuration basically involves telling our server the hosts and ports of all the stuff it needs to run. Which in our case is postgres, redis, and elasticsearch.

Editing the setup is easy; just edit the docker compose and modify some properties. Works with jvm based stuff and it's equally easy to replicate with other stuff.

There are a few more tricks I use to keep things fast. I have ~300 integration tests that use db, redis, and elasticsearch. They run concurrently in under 1 minute on my mac. I cannot emphesize how important fast integration tests are as a key enabler for developer productivity. Enabling this sort of thing requires some planning but it pays off hugely.

I wrote up a detailed article on how to do this some years ago. https://www.jillesvangurp.com/blog/2016-05-25-functional-tes...

That's still what I do a few projects and companies later.

Yes [1].

1. https://www.atomicjar.com/2023/06/running-testcontainers-tes...

I see a lot of people commenting to use docker-compose instead.

Testcontainers does have a docker compose integration [1].

1. https://java.testcontainers.org/modules/docker_compose/

If you build inside docker, running tests that use docker is a pain.

Go has a lot of in-memory versions of things for tests, which run so much quicker than leaning on docker. Similarly, I found C# has in-memory versions of deps you can lean on.

I really feel that test containers, although solving a problem, often introduces others for no great benefit

I'm surprised this is getting so much attention. I thought this just standard practice at this point? If you use things like Gitlab CI then you get this via the `services` in your pipeline. The CI job itself runs in a container too.

I use a very similar thing via pytest-docker: https://github.com/avast/pytest-docker The only difference seems to be you declare your containers via a docker-compose file which I prefer because it's a standard thing you can use elsewhere.

works on my container

  Testcontainers is awesome and all the hate it gets here is undeserved.

  Custom shell scripts definitely can't compete.

  For example one feature those don't have is "Ryuk": A container that testcontainers starts which monitors the lifetime of the parent application and stops all containers when the parent process exits.

  It allows the application to define dependencies for development, testing, CI itself without needing to run some command to bring up docker compose beforehand manually.
  
  One cool usecase for us is also having a ephemeral database container that is started in a Gradle build to generate jOOQ code from tables defined in a Liquibase schema.

I tried to use Testcontainers just last week but ended up using simple docker commands instead. I didn’t find an easy way to connect an already running set of containers started via docker compose. Was straightforward to do with a set of scripts that just call docker exec.

No thanks, I will continue to roll my own to control it full, at home and at work.

I see testcontainers being used in tests making the test code style feel more like typical unit tests with fake implementations for system components. Which is misleading as these are more on the integration testing side typically. In essence this is another DSL (per language) for managing containers locally. And this DSL comes in addition to whatever system is actually used for managing containers in production for the project.

Reading through the comments, I'm quite shocked to see how many deterrent conversations are happening without any understanding of the underlying tech stacks being tested. Testcontainers can be fantastic, especially when you are facing test environment overhead challenges, assuming you have the appropriate architectures / service boundaries to support it. I believe there is more code out there in existence with architectures that make using Testcontainers more challenging than it is worth.

Never heard of this, also the label in footer for careers said "We're hiring" but then lists no open positions :/

I've been using Docker containers for integration testing for last few years. I usually roll my own custom solution in Go using the https://github.com/ory/dockertest package though that adds necessary functionality around running migrations, creating kafka topics, or similar. Will definitely need to check out this next time I'm writing a test package

Duke as the logo for java?