The long road to a fix for CVE-2021-20316 (2023)

Forjonathan corbet
February 10, 2022

Well-maintained open source projects often struggle to fix known security issues quickly, and thesamba project, which provides interoperability between Windows and Unix systems, is no exception. So it is natural to wonder why the solution forCVE-2021-20316, a symbolic link vulnerability, took more than two years to arrive. Sometimes a security bug can be fixed with a simple code tweak. Other times, the solution requires a massive rewrite of much of a project's internal code. This particular vulnerability fell firmly into the latter category, requiring a public rewrite of Samba's Virtual File System (VFS) layer to address an undisclosed vulnerability.

The story begins witha bug reportBy Michael Hanselmann May 2019 When an SMB client tells the server to create a new directory, the server must perform a series of checks to make sure the client has the right to do so. Among other things, the server ensures that the requested directory actually resides within the exported SMB share, rather than in an arbitrary location elsewhere on the server's file system. Unfortunately, there is inevitably a window between when the server performs the check and when it actually creates the directory. If a malicious user is able to replace a component in the path to the new directory with a symlink during that window, Samba will happily follow the link and misplace the directory, with results that are generally viewed as unsavory by anyone but an attacker. . . 🇧🇷

This is a classic scantime/usetime (TOCTOU) vulnerability, the kind for which symlinks have become notorious. It's also hard to fix, especially for a system like Samba where portability is a major concern. There is no easy, cross-platform way to query a path's attributes on the filesystem and safely act on the result, knowing that a malicious actor can't change things in between. Still, something clearly had to be done, so Samba developer Jeremy Allisonjumped into write a correction. The CVE number CVE-2019-10151 has been successfully assigned to this issue.

the real problem

The hope was to find a quick solution, but early on, Allison identified the real problem: the use of paths to interact with the server-side file system. Each time a path is passed to the kernel, the traversal process must be performed again; any user who can make this process arrive at different places at different times (through carefully timed use of symbolic links, for example) can use this ability to confuse the server. The good news is that there is another path that does not depend on immutable paths.

Over the years, the kernel has gained a set of system calls that operate on file handles (open file descriptors) instead of path names. A carefully written server could, for example, useopen en2()to create a file descriptor for a directory of interest, do your checks to make sure the directory is what you expect and usemkdirat()to create a subdirectory that cannot be redirected to the wrong place. Used correctly, these system calls remove the TOCTOU race from this type of operation, but they only work if they are being used, and the use of Samba in 2019 was limited. At the time, Allison commented: "Ultimately, we need to modify the VFS to use the syscallAT() variants of all system calls, but that's a VFS rewrite that we'll have to schedule for another day.".

About a month after trying to close the vulnerability (and other symlink issues that surfaced when people started looking for them), it became increasingly clear that "another day" was coming sooner than previously thought. . In mid-July 2019, Allisonseemed resignedfor the big rewrite: "This is going to be a long job, rewriting pathname processing on the serverHowever, there was a complication: while this hard work was ongoing, the vulnerability would remain unpatched and unrevealed. So how would all this work be explained to anyone else who sees the Samba project?according to allison:

We need to rewrite the fileserver to [make] the arbitrary symlink change execute safely on all pathname operations. This is too big to do privately, so I'm doing it publicly under the guise of "modernizing the VFS to use identifier-based operations" (without explicitly saying *why* I'm doing this).

There were a few other aspects of this project, aside from the need to hide its true purpose, that made it difficult. One is that the SMB version 1 (SMB1) protocol relies on the path name at its core, making it nearly impossible for the server to use anything else. The abrupt suspension of SMB1 inRelease Samba 4.11in September 2019 it was partially driven by this issue.

The SMB2 protocol, in contrast, relies heavily on file handlers, which should make it easy for a server implementation to work the same way. But Samba is an old program with a lot of history, many of the internal interfaces still used path names even when a file handle was available. This includes the VFS interface which is used to talk to modules for specific host file systems, add functionality like virus scanning, and much more. Changing all these internal APIs was a big job that would affect a lot of code on the Samba server.

thousands of changes

Over the next two years, Allison would contribute 1,638 commits to the Samba repository, 17% of the total during that period. Not all of them were geared towards a VFS rewrite, but most were. And Allison wasn't alone; Ralph Böhme (1,261 confirmations), Noel Power (438) and Samuel Cabrero (251) also contributed greatly to this project. "Modernizing" the Samba VFS took up much of the project's attention, staying mostly under the radar of anyone not aware of the actual problem.

Böhme presented this work at the SambaXP 2021 event (video,slideshow) without ever mentioning the (still pressing) security issue that was causing it. The talk goes into great detail about what needed to be done and how various problems were solved in Linux; it is a recommended viewing for anyone who wants to dig deeper. There is also some information aboutla Sambawiki.

In July 2021, Allisonvictory declared:

With the master commit e168a95c1bb1928cf206baf6d2db851c85f65fa9, I believe all the race conditions in the metadata are now set to the default routes. DOS async attribute reading still uses path-based getxattr, and some of the VFS modules are not symlink-safe, but I think Samba out of the box will no longer be vulnerable to this in 4.15.0.

Since then, the remaining route-based extended attribute calls have also been fixed. Of course, there were some details to deal with still, including the fact thatthe original CVE number has expiredbecause it has not been updated for a long time. This required assigning a new number, so this vulnerability is known as CVE-2021-43566. The job appeared as expected in theRelease Samba 4.15.0in September 2021, more than two years after the initial vulnerability report.

Disclosing the vulnerability, the Samba project described the situation this way:

A two and a half year effort was undertaken to completely rewrite the Samba VFS layer to stop using path name based calls in all cases related to reading and writing metadata returned to the client. This work was finally completed in Samba 4.15.0. 🇧🇷

Since all operations are now performed on an open handle, we believe that any other symlink race conditions have been completely removed in Samba 4.15.0 and all future versions of Samba.

The disclosure also notes that due to the bulk nature of the rewrite, it will not be possible to patch this vulnerability in earlier versions of Samba.

In the end, the Samba project was able to patch this vulnerability before word of the problem got out and before any known vulnerabilities occurred. But it was a bit risky; Attackers tend to keep an eye on cool project repositories in hopes of spotting fixes that address undisclosed vulnerabilities. It's hard not to draw comparisons to the events that led tothe reveal of Meltdown and Specter, which also required major changes to resolve an undisclosed vulnerability. But unlike the developers working on Spectre, the Samba developers found a way to do their job in the public eye, making sure all patches were properly reviewed and minimizing issues that needed to be addressed after the patch was revealed. issue.

The gamble seems to have paid off, although things could have turned out differently. Since then, Allison hasmaking the pointthat symlinks are dangerous in general, and that other projects almost certainly have similar problems. He has a speech planned forSambaXPPresumably later this year it will be closer than Böhme's 2021 release. Samba users (at least those who have upgraded) are expected to be immune to symlink attacks, but that's probably not true for many other systems we depend on.

(Thanks to Jeremy Allison for answering questions and providing a technical review of a draft of this article.)

(get connectedto post comments)