lwio – the SMB redirector: Building the integrated Linux SMB client experience

One of the more painful things about using Linux as a client in a Windows corporate intranet is the clunkiness of integration. The most basic requirement is that you be able to bring up the Nautilus or your favorite graphical file browser and browse through files that live on a Windows (Samba) file server. I’ll reiterate: this is one of the most basic requirements.

Now most affficianados will be up in arms when I say this. Of course, Linux does this and Ubuntu does this out of the box. But here is the rub: Ubuntu’s Nautilus file browser will does this fine. But now I want to go to my bash command shell and I want it to also be able to browse to my Windows file shares. This does not happen out of the box.

Now even if you get all of the pieces to work, the credentials are messed up. The single-sign-on experience needs to immediately transfer itself to all client applications. Even though you can sign in with your AD credentials, if I have to re-enter my credentials, it adds to the clunkiness of the experience.

Introducing the SMB redirector in lwio.

Our SMB redirector driver provides all of the semantics necessary for a file client API. The obvious next step was to pick up FUSE and begin work on a FUSE driver that would call our lwio APIs – NtCreateFile, NtReadFile, NtWriteFile, NtClose. Once you provide a FUSE driver to our SMB client, automatically, a client can mount smb shares without reentering his/her credentials. Once the client mounts a share, he can browse the directory space, copy files, move files, create directories, delete files and perform every UNIX file system operation against the SMB volume. Even better, Nautilus and the Gnome shell work without any additional work.

We’ve started construction of this FUSE driver and its is amazing how fast we’re putting together the pieces. Our Posix Virtual File System (which is the most complete lwio driver) is proxying for the SMB Redirector File System (we’ve implemented enough for the named pipe interface and its missing some of the QueryInfo and SetInfo semantics)

Now here is the even more interesting part: FUSE driver run as a daemon themselves, so a client (like the shell) has to communicate to the FUSE daemon which in turn communicates to the lwio daemon. That’s a transition of 2 processes which means at least 4 context switches to service a call. But lwio can implement a FUSE driver as another lwio driver. Just like the SMB server driver calls the IO Manager to route requests to the Posix Virtual File System, the FUSE lwio driver calls the IO Manager to route requests to the SMB Redirector File System. This implies we reduce the number of context switches between processes to half what we would have if we ran FUSE independently as a separate daemon. This was Brian Koropoff’s epiphany over the weekend. We’re still going to do this first as a separate daemon, iron out the end-to-end scenario, but then placing it into the lwio kernel is a natural optimization.

Well, I thought we’d be doing three key scenarios by April 15th. It looks like we’re going to go for a fourth scenario.

Thanks for reading!

lwio watch: Week 7 ending February 28, 2009

In the course of building a new software system, there are these inflection points when everything comes together. Last week was one such inflection point. Here are the highlights
- Windows explorer, dir, Acrobat reader, thumbnails views on explorer all work
- copying files, deleting files, drag and drop, xcopies are all smooth and seamless
- Word now works – we can click on a Word document on a share and we can open the document and edit it
- Multiple connections and multiple large file downloads work smoothly
- We’ve set up an internal file server that hosts over half a terrabyte of data (with full iso OS images and VMs) and are using this as our internal dog food server.
- The MMC share management snapin and wizard works smoothly; we can point MMC at our Linux server create, delete and manage shares on the server.
- The installation is very simple. There is next to nothing to configure and setup. Install the bits and you have a file server available.

The month of March promises to be very interesting for us. I’ll write another post on what we plan on accomplishing by end of March.

Thanks for reading

lwio watch: Week 6 continued (ends February 20th, 2009 today)

It has been a super productive week.

First the highlights. The lwio SMB File Server has hugely advanced. Most operations from the Windows XP command line work. Copying files, xcopying, making directories, net use, net use /del, single sign-on are all fully operational. In addition, the Windows explorer is almost completely functional. We’ve yet to support server-side file change notifications (i.e changes to the remote directory are not immediately visible in the Windows Explorer, the simple workaround is to hit the refresh button). You can view thumbnails of pictures and photographs, you can bring up notepad on a file directly. In all, we have a pretty useful file server.

Today we plan on starting internal dog-fooding. We will be hosting our own server and placing large size OS iso images which we use, so that everyone in the engineering team can upload/download files to the server. We’re putting all our VMs as well on the server and will subject the server to a significant amount of stress.

The day is not yet done. It is 8:38 hrs PST out here in Bellevue WA. We plan to have the file share management work done by end of day today. This will allow us to create file shares on the Likewise SMB File Server via MMC’s Add Share Wizard. Also, we plan on add a File Management plugin on the Likewise Administrators Console that will allow Linux Administrators to graphically manage their own Linux Likewise File Servers from Linux desktops.

Its been a good week!

Windows Explorer works!

Yesterday, we got Windows Explorer to work against the Likewise SMB File Server

Windows Explorer copying a file from the Likewise SMB File Server