Krishna Ganugapati’s Weblog

1) The smb server has had the most code go in. Sriram has finished NEGOTIATE, SESSION_SETUP and TCONs
2) We have manually populated the share database so that you can actually connect to shares configured in the database
3) The redirector has been fixed to support Mac OS X x86 support and by end of next week we should have full big endian suppport
4) The named pipe file system driver is almost code complete
5) Rafal has begun checking in the beginnings of the Lsa RPC server

I’ve gotten a lot of requests from people asking that they be updated on the status of lwio. We’re working on weekly milestones and I’ll update you on our progress on a week by week basis.

This week – ending tomorrow Friday January 16th, 2009

1) The Nt APIs have been wired to the IO Manager. This means that you can do a NtCreateFile, NtReadFile, NtWriteFile, etc from a user mode client to the lwiod – we’re routing requests to the smb redirector and the posix subsystem.
2) We’ve just enabled the posix subsystem, the redirector and the server driver – the IoMgr will load these drivers in.
3) We’ve nuked the old iomgr which used the old smb client model.
4) The srvsvcd daemon is up and running and will provider server side support for NetShareEnum, NetShareAdd, NetShareDelete, NetShareGetInfo and NetShareSetInfo. The srvsvcd daemon is critical because it represents the end to end server-side RPC pipe making device i/o calls into the server driver.
5) Sriram has pretty much gotten SMB session setup working and will start adding support for tree connects, followed by opens, reads, writes and closes. These five calls are basically what we’ll need for the named pipe server to work.
6) Last and the best, we’ve wired DCE/RPC to call the actual NtCreateFile calls to the redirector for client side named pipes to work.

The plan will be to self-host the lsass subsystem against lwio by the end of the month. That is also our timeline to push this branch of code into trunk where it will start undergoing our rigorous battery of daily bvt, stress and performance tests.

Thanks for reading!

Update Jan 16, 2009: Brian just informed me that lwio is now self-hosting the lsassd daemon over the new smb redirector driver. That means our end-to-end iomgr is fully operational!

Happy New Year everyone!

I’ve some really great news to report. We’re full steam ahead on our new architecture for an SMB framework and hopefully we’ll show case this work at Samba XP 2009.

If you’ve looked at our trunk source tree, you will see that we have the new lsmb server. lsmb hosts an smb client and is itself a service that other clients can connect to. The client interface for lsmb manifests itself as the quasi Win32 API. Think CreateFile, ReadFile, WriteFile and CloseHandle. Remember that these functions are used to access named pipes on a Windows server. We retrofitted DCE/RPC to talk to this client library which in turn talked to the lsmb server which hosted an smb client. We now have a client named pipe API and we can do RPC over named pipes, a key requirement if you want to do any kind of MSRPC protocol.

We’ve checked this into trunk and it’s close to being declared stable. I suspect by the end of the month, we’ll declare this stable.

Remember that the goal for lsmb is that we could support server side named pipes as well. This is a much trickier problem. The problem here is that you need an SMB server – at least a minimal SMB server and you need to support the notion of fifo queues. The “server” client also needs to use the named pipe API, but this case, the server uses a special API – CreateNamedPipe which returns a handle and then subsequently uses that handle to do ReadFile, WriteFile and CloseHandle operations. We’d then need to retrofit the server side interface into the DCE/RPC library so that we could function as an RPC server over named pipes.

Our first cut was to divide lsmb into two halves. The first half was the existing SMB client piece and the second half would contain the server piece. I thought that we would implement a server side virtual file system layer where there would be an SMB server piece which would fork requests off to the named pipe file system and to test this we would put together a posix virtual file system. Let’s face it – I’ve spent the last three years with a lot of Linux and Samba developers and this architecture is very familiar to them. Let’s say I was inspired by them. But this architecture has a problem. The named pipe file system now has two interfaces – the server end of it routes to the VFS layer, but remember we still need to indicate data back to the calling client – the named pipe server. So the upper edge Win32 style API has to route down to the named pipe VFS provider so that we can indicate data. Its a kludgy design – doesn’t work!

When you rethink this – you will see that we really have three “file system” drivers – the smb redirector file system (which was the client side silo in lsmb), the named pipe file system (which actually only indicates data to calling “server” clients), the posix virtual file system (which receives data for writes and indicates data back for reads) and an SMB server driver that serves as a client (both for reading and writing data to the 2 of the file systems – the posix virtual file system and the named pipe file system. Now if you look a little more carefully, we now have four different “drivers”. All of these drivers really should have a common interface and this is the beginning of the Likewise lwio daemon. ln future versions of the product, the lsmb daemon will morph into lwio daemon.

lwio consist of a client side piece that IPCs file system requests to the protected server. We needed an client side API and here our design was inspired. Recall that our goal is to implement a minimal SMB server which is necessary to do named pipe server io which is necessary to do server side DCE/RPC over named pipes. The SMB architecture really is the NT NtCreateFile, NtReadFile, NtWriteFile, NtClose, NtQueryInformation and a NtSetInformation extended to the network. Thus the interface that the I/O Mgr exposes is an analogue to these system calls. The client side exposes these APIs as interfaces and they are ipced over to the server using lwmsg. On the server side the client API is represented similar to what it would be like in the NT kernel. We have IoCreateFile, IoReadFile, IoWriteFile, IoClose etc. This is where the “io” in lwiod resides. The lwiod daemon is the “kernel” and the client to lwiod represents the “user mode” side of the lwiod kernel. We called it io because we really don’t have a kernel and the common runtime routines that the drivers use are all io routines.

The IoMgr is responsible for routing requests to shared object libraries that represent “drivers”. Every driver implements a DriverEntry function and returns back a set of dispatch functions (not a dispatch table). Drivers create device objects which allow the IoMgr to route requests to the a specific driver. Thus when a client calls NtCreateFile, the Create dispatch function in the driver is called. Prior to calling the Create dispatch, the IoMgr creates a file object (represents the file handle) and passes the file object to the Create function. The driver then does the appropriate “create” stuff – in the case of the posix file system, it will call the Posix open function store the handle in a context and attach the context to the file object and return back. You get the rest of it. Similarly when the smb redirector file system gets a Create it does the SMB NEGOTIATE, SMB SESSION_SETUP, SMB_TREE_CONNECT, SMB_NTCREATE and stores the corresponding tree object with the Tid and the file object with the Fid . Understanding the driver architecture is pretty straightforward when you’re mapping this to a file system.

The SMB server driver is a different beast. It doesn’t implement any of the Create, Read, Write, Close dispatch functions. It does implement the DeviceIo dispatch function. Its a pretty empty driver from the file system call perspective. What you can do is send the driver control codes to do stuff. Its the quissential Toaster driver. But when it starts up, it starts listening on port 445. And it processes SMB requests and then translates them into file system requests and hands them to you guessed it. the IoCreateFile, IoReadFile, IoWriteFile, IoClose file functions which the IoMgr then routes to the posix virtual file system or the named pipe file system. The server driver keeps track of the TIDs (a list of tree objects) and the FIDs (the file ids).

Hope you find this as exciting as I do. Stay tuned. I plan to describe end user scenarios as they unfold over the next few months.

Once again, happy new year!