An inefficient way to wait for Meilisearch to complete all async tasks

I have been working on integrating Meilisearch into Hyvor Talk as a secondary database to make comments search faster. Meilisearch is a fine solution for our use case, however, when it comes to automated testing, Meilisearch’s asynchronous document handling poses a problem.

Our search endpoint tests look like this (we use PHP and Laravel):

1it('searches for comments', function() {
2	// seed some comments
3	Seeder::comments(5);
4	
5	// call the API
6	ApiCaller::get('/comments/search', ['search' => 'Test'])
7		->assertOk();
8});

The problem is Meilisearch’s document-adding process is asynchronous. I could not find a way to disable this behavior per instance, per index, or per API call basis.

So, the above test fails sometimes, because even if the documents are sent from the app to Meilisearch to be added, that process is not completed by the time we call the /comments/search endpoint.

One easy solution is to use sleep() or usleep() before calling the API endpoint. But, we will still have a flaky test. Not good.

Here’s an (inefficient) solution I came up with.

 1use Meilisearch\Client;
 2
 3class MeilisearchInefficient
 4{
 5
 6    /**
 7     * This is an inefficient way to wait for all tasks to complete
 8     * Only use in testing
 9     */
10    public static function waitForAllTasks() : void
11    {
12
13        $client = new Client(
14            strval(config('scout.meilisearch.host')),
15            strval(config('scout.meilisearch.key'))
16        );
17
18        while (
19            collect($client->getTasks()->toArray()['results'])
20                ->filter(fn($task) => $task['status'] === 'enqueued' || $task['status'] === 'processing')
21                ->count()
22            > 0
23        ) {
24            // 150ms
25            usleep(150000);
26        }
27
28    }
29
30}

It calls Meilisearch’s /tasks endpoint until all tasks are completed. It sleeps for 150ms between two requests. Note that it uses the Meilisearch\Client, which comes with the meilisearch/meilisearch-php package.

I have noticed (on my M1 8GB Macbook) in most cases, only one API call is made to the /tasks endpoint. The time the HTTP request takes is enough for Meilisearch to complete up to 10 inserts. So, it is not hugely inefficient. We’ll use this solution for now for testing.

Now, our test would look like this:

 1it('searches for comments', function() {
 2	// seed some comments
 3	Seeder::comments(5);
 4	
 5	MeilisearchInefficient::waitForAllTasks();
 6	
 7	// call the API
 8	ApiCaller::get('/comments/search', ['search' => 'Test'])
 9		->assertOk();
10});

Hopefully, Meilisearch will add a way to force synchronous processing in the future.