Server-side Directory Filtering

Lee Painter

The Maverick Legacy Server and Maverick Synergy servers support the ability to filter directory listings server-side. When directories are large and contain many files, the SFTP protocol provides no means to filter them, requiring that a client read all the files that are present in the directory and filter them itself. This can result in slow responses and frustrate users.

To address this we have implemented our own SFTP extension that allows the client to open and read a directory with a filter. The server will return only the results that match the filter. In order to provide a full reference implementation, we have implemented this in our server and client APIs. 

To support this extension on the server requires enabling it by adding an SFTPExtensionFactory.

sshContext.getSFTPExtensionFactories().add(
new DefaultSftpExtensionFactory(SupportedSftpExtensions.OPEN_DIRECTORY_WITH_FILTER));

 

Please note, that to support this SFTP extension requires the use of our AbstractFileSystem and the AbstractFile framework. Vendors implementing the FileSystem interface directly will not be able to use this extension. 

This can then be used from the SftpClient object using the new ls methods available.

sftp.ls("*.txt", false, 9999);

 

Note that here, the second "false" parameter is indicating that the filter does not contain a regular expression and that glob syntax is required. We can of course implement this in the same way but using a regular expression

sftp.ls("^.+\\.txt$", true, 9999);

 

The third parameter is the maximum number of files to return. This should be used to avoid memory problems with the server an unlimited number of responses. 

 

Directory Filtering Extension Specification 

This extension utilizes an SSH_FXP_EXTENDED message to the server to open a directory. As part of the request, a filter is provided with an indication as to whether the filter is a regular expression (true) or glob syntax (false).

byte   SSH_FXP_EXTENDED 
uint32 request-id
string "open-directory-with-filtering@openssh.com"
string path
string filter
bool regex

 

The server then responds as normal as if it were an SSH_FXP_OPENDIR request, sending an SSH_FXP_HANDLE response. 

The client then requests reads from the directory using the standard SSH_FXP_READ request and all responses and requests from then on are handled in the same way as if the original request was an SSH_FXP_OPENDIR request; however, the server returning only those files that match the filter provided in the original request.