Testing Mago as an alternative to PHPStan

I discovered Mago, a PHP toolchain written in Rust, on Reddit a couple of days back. It recently published version 1.0. I am testing it primarily as a candidate to replace PHPStan.

PHPStan has been incredible in terms of the features it provide. But, it’s slow. Having experienced a near-instant DX with typescript, PHPStan's speed has always been a problem. In Hyvor Relay repo, which has about 3000 PHP source and test files, it takes about 5 seconds to re-analyze after a few changes. This makes development feedback loop slow that in most cases, we just write code, push it to CI, and finally fix the PHPStan issues later. Also, the IDE intergations are not great without a LSP.

I'm looking for two things in Mago:

• It must support all the PHPStan features (PHPDoc types, generics, custom types, etc.)

• It must be fast.

Note that Mago also has a linter and a formatter, which I will not be testing today.

Setup

• I'll be testing Mago on Hyvor Relay (~3000 PHP files). It has a good amount of dependencies, uses generics and custom types, and is already configured with PHPStan.

• My machine specs:

  • OS: Ubuntu 24.04.3 LTS x86_64

  • AMD Ryzen 7 7800X3D

• Hyvor Relay runs inside Docker

  • Image: dunglas/frankenphp:1.10.0-php8.4.15

  • Composer: 2.8.8

• PHPStan

  • Level: max

  • Paths: src, tests

PHPStan Performance

Before getting started, I'm using the time command to test PHPStan performance:

1time vendor/bin/phpstan --memory-limit=1G

Running a couple of times:

16.376s -- first run has no cache
20.261s -- cached results
30.277s
40.264s

Running again with one file updated each time:

11.088s
21.769s
32.210s

Installing Mago

It would be ideal if there was a Docker image where I could copy the mago binary from. For now, I simply did what the documentation says. I added the following to the Dockerfile.

1RUN curl --proto '=https' --tlsv1.2 -sSf https://carthage.software/mago.sh | bash

Checking that it works:

1> mago --version
2mago 1.0.2

Initialization

Mago needs a mago.toml config to work with. I generated it using mago init (See Initialization in Mago docs)

This is the generated mago.toml file:

 1# Welcome to Mago!  
 2# For full documentation, see https://mago.carthage.software/tools/overview  
 3php-version = "8.4.0"  
 4  
 5[source]  
 6workspace = "."  
 7paths = ["src/", "tests/"]  
 8includes = ["vendor"]  
 9excludes = []  
10  
11[formatter]  
12print-width = 120  
13tab-width = 4  
14use-tabs = false  
15  
16[linter]  
17integrations = ["symfony", "phpunit"]  
18  
19[linter.rules]  
20ambiguous-function-call = { enabled = false }  
21literal-named-argument = { enabled = false }  
22halstead = { effort-threshold = 7000 }  
23  
24[analyzer]  
25find-unused-definitions = true  
26find-unused-expressions = false  
27analyze-dead-code = false  
28memoize-properties = true  
29allow-possibly-undefined-array-keys = true  
30check-throws = true  
31check-missing-override = false  
32find-unused-parameters = false  
33strict-list-index-checks = false  
34no-boolean-literal-comparison = false  
35check-missing-type-hints = false  
36register-super-globals = true

Analyzing

1time mago analyze

1.732s for the first run! That's a x3 improvement. I ran the same 5 more times:

11.731s
21.685s
31.714s

Mago does not seem to support incremental analysis yet.

Static Analysis Results

Mago detected 1000+ errors, where PHPStan at max level showed 0.

• ~500 of those errors were from check-throws = true setting. This checks that all thrown exceptions are handled. There were many unhandled exceptions in the tests/ directory since many PHPUnit assertions throw them. For now, I disabled this check until I find a way to disable it only in the tests directory.

• Most of the other errors came from Zenstruck Foundry proxy objects. Hyvor Relay still uses PersistentProxyObjectFactory , which is now deprecated.

• There were some errors that were better detected that PHPStan. For example:

1$dnsIp = count($ips) > 0 ? $ips[0]->ip : "";
2                           ^^^^^^^ This expression can be `null` here

Cargo-like Error Reporting

My favorite so far is Cargo-like error reporting in Mago. They are so beautiful :)

 1warning[possibly-null-operand]: Left operand in `>` comparison might be `null` (type `bool|float|int|null|string`).  
 2   ┌─ src/Service/Queue/QueueService.php:51:16  
 3   │    
 451 │           return $this->em->createQueryBuilder()  
 5   │ ╭────────────────^  
 652 │ │                 ->select('COUNT(q.id)')  
 753 │ │                 ->from(Queue::class, 'q')  
 854 │ │                 ->where('q.type = :type')  
 955 │ │                 ->setParameter('type', QueueType::DEFAULT)  
1056 │ │                 ->getQuery()  
1157 │ │                 ->getSingleScalarResult() > 0;  
12   │ ╰─────────────────────────────────────────^ This might be `null`  
13   │    
14   = If this operand is `null` at runtime, PHP's specific comparison rules for `null` with `>` will apply.  
15   = Help: Ensure this operand is non-null or that comparison with `null` is intended and handled safely.

My Thoughts

First, I am genuinely impressed by the results. PHP ecosystem took many years to get to the stage where we have tools like PHPStan. Mago seems to just do it after a couple of years of development.

What I am most excited for is the upcoming LSP. This, along with the PHP extension installer, would dramatically change the PHP ecosystem. A nice LSP integration would make heavy IDEs less attactive.

We will be integrating Mago into our workflows and see how it performs in real-world scenarios.