I’m always looking for extraordinary systems programmers

If you worked on distributed systems, operating systems, networking as a software design engineer,  I’m always on the lookout for great and extraordinary engineers

This is how you would define yourself:
- You cut high calibre C code and a ton of it (really a lot of code, no kidding!)
- You are the kind of person who is super technical and is a force multiplier.  You’re generally delighted to work on hard technical problems.
- you are an outstanding systems programmer.
- You have superb analytical skills
- You have an high GPA at University. A high GPA in Computer Science tells me you are very smart and have had to work very hard. The course load and project load in most good CS schools is always crushing.
- You have strong formal Computer Science skills – Operating Systems, Compilers, Computer Architecture, Theory of Algorithms, Computability, Theory of Programming Languages, Advanced Operating Systems, Distributed Systems.
- You have a BS or MS in Computer Science
- You have written or contributed to the writing of one or more C based file servers, dns servers, web servers, ldap servers, certificate servers, radius servers etc etc
- You have a PhD in operating systems/distributed systems, but don’t want to be an academic or work for a research think tank
- You wanted to do a PhD, but finished a Masters and looked for a job
- You’re probably on an IETF or IEEE mailing list – I most definitely would like to talk to you
- You understand networking protocols intimately well – you’re at a place where you read IETF RFCs to do your job.

I value technical people at every level. I think the best product marketing people are those who have a rigorous background in the discipline they are trying to market. I think the best SEs have a solid grounding in the products and technologies they sell. I think the best test engineers are those who actually understand the internals of the technology they are testing. I need specialists, not generalists. I thinks specialists can work as generalists if they have to, but generalists can’t become specialists unless they are willing to roll up their sleeves, get in the muck and figure it out. A little anecdote below..

Recently, I was flying back from Boston and happened to sit next to a guy who was watching movies on his iPhone. We got talking and he told me he used to be the Northwest Account Manager for Cisco (i.e the sales rep). What he told me was astonishing. John Chambers is a extremely hard core driven sales CEO for Cisco. Yet the SEs (the Sales Engineers) are largely drawn from within the ranks of Cisco engineering. He mentioned that his SE was a developer for 6 years before he transitioned to the sales side of the business. This one SE managed the pre-sales process, the post sales process, the deployment and roll out for one of the largest cellular networks in the country – think of how many IOS routers this guy is deploying and setting up. That is impressive!

Anyway I hope to hear from you

Screenshots for Likewise 5.4

Likewise and Moblin

Arlene Berry got Likewise Open running on Moblin. We have a few hoops to jump through. The most
important being that Moblin does not provide a graphical login console – but there are packages available
that provide a gnome window manager login shell. Once you joined the Moblin netbook to Active Directory
you could enter your AD credentials and login with them.

Logging out is also a challenge. There is no application that lets you log out. But as far as Likewise is
concerned, you can install the Likewise Open bits (rpm ones for Fedora) and you’re good to go.

Getting AD authentication on netbooks is a huge deal. Given the fact that over 32% of netbooks are shipped with Linux,that means a Linux netbook with AD integration is a viable alternative to a netbook with Windows 7. Your Linux netbook can be used in corporate environments and can take advantage of the corporate network’s resources.

We’ve gotten Moblin up and running with very little work – hopefully we can do the same with Chrome OS and Android.

Thanks for reading.

Lots of cleanup in LAC and the Likewise Class Libraries

Wei Fu and Suneetha Gunasetti have been working steadily to clean up our LAC sources and make them ready for release. We’re almost there. Here are some of the highlights

1) First, we’ve separated our plugins into two directories – Open and Enterprise.

2) Build systems – we will make the open source LAC code base buildable on Visual Studio and on Monodevelop. This way if you have Visual Studio, you can build the source plugins, but if not, Monodevelop should work just fine.

3)We’re fixing up the licensing – LAC and all of its plugins will ship under the LGPL license. What this means is that you can write proprietary plugins if you want to. LAC then becomes a ready to release administration tool system for any product you’re shipping.

4) But the best thing of all is that we’ve refactored out the core class libraries. The Likewise Class Libraries will give you APIs for the NetAPIs, the Kerberos libraries, the Registry, the Eventlog, the Service Control Manager and a host of other classes that allow you the developer or ISV to build your own management applications. You’re not bound to LAC/LMC rather you have a powerful set of class libraries that let you exploit the power of the Likewise Distributed Systems Platform and build great applications in your favorite managed languages.

Thanks for reading and wait for the press release early November.

LAC now LMC coming soon in open source

One of the singularly important pieces of technology at Likewise has been the Likewise Administrators Console. Now renamed the Likewise Management Console, it is a multi-plugin graphical management console for administration of server infrastructure.

Very similar to those of you who have used the Microsoft Management Console, LMC is a significantly superior platform. It is written in C# and uses the .NET platform and is completely portable on Linux, Mac and Windows – The Linux and Mac ports are possible thanks to the magic of Miguel de Icaza and his Mono team. We love Mono out at Likewise and have been early adopters of the Mono platform for several years now.

LMC will ship with several plugins
- a local users and groups plugin -LUG allows you to point to a Linux machine and create and manage local users and groups
- an event viewer plugin – the EventViewer plugin allows you to examine the eventlog on a remote Linux machine
- a registry viewer plugin
- a file share management plugin
- and a active directory users and computers plugin

LMC is released under the LGPL license. This allows third parties and ISVs to write proprietary plugins if they want. So if you are an ISV or clustered NAS vendor, you would potentially ship LMC with your product line and write proprietary plugins that allow you to provide value-added management functionality to your products

We’re busy tidying up LMC. But expect to see it out in early-mid November.

Thanks for reading.

A NFS protocol head for LWIO

The number of requests we’re getting on an NFS4 protocol head for LWIO is steadily increasing to the point where we can’t ignore it any longer.
Most storage and clustered NAS players have challenges getting CIFS and NFS to work collaboratively without stepping on each other toes. The CIFS server is usually in user space and the NFS stack is usually in the kernel. The challenge is synchronizing file locks and generally making CIFS and NFS aware of each other.
lwio is a natural solution to this problem. The lwio architecture cleanly separates the file server protocol from the file system layer. Thus it is super easy for us to build an NFS protocol head driver that calls into our pvfs driver for file system service. Since PVFS manages oplocks and general file access, all file access is centralized in the PVFS driver.
Anybody out there have any thoughts on this. Feel free to drop me a line.
Thanks for reading

Service Control Manager, Registry, Spooler and Sample RPC Client-Server

October will be the month for finishing up 5.4.
5.4 will have every Likewise service use the Likewise registry. Current list of services that we will be converting over to use the registry are the following
a) lsassd
b) lwiod with all its drivers – specifically the share database will move into the registry
c) netlogond
d) eventlogd
e) gpagentd – all group policy configuration will move into the registry as well

Finally, Brian Koropoff has begun work on the lwscm – the Likewise Service Control Manager. And if all goes well, 5.4 will ship with the Service Control Manager. What does the lwscm do? Well it is responsible for forking off Likewise services. So instead of having multiple init scripts and having to deal with different configuration formats for different platforms, every Likewise service is stored in the registry. The Likewise Service Control Manager is responsible for identifying service dependencies and ensuring dependent services are launched prior to starting the target service.

While the first incarnation of this Likewise Service Control Manager is about separate services running in indepenedent processes, the next incarnation will be the ability to run multiple services in a single process. Subsequently we hope to provide a “svchost” model where we provide a service container which developers can leverage. There should be no need to have to write housekeeping code, but get developers to focus on their specific service offering.
Brian has just demoed some of this working and it is pretty impressive. The best part for me is to get away from maintaining multiple init scripts across multiple platforms.

Also, I’ve started the preliminary framework for a “spooler” architecture. We were flying back from the Linux Plumbers Conference and I began doodling on the flight and while we have no commercial reason at this time to be building a spooler, I’d like to get to a place where other developers on the team can jump in, add an RPC call if they want and get back to what their primary focus should be. I think it could be a fun distraction.
Lastly, I’m going to start getting dev packages ready, so that developers can install binaries and get the appropriate header files and starting building to the Likewise platform. One of the things I used to do at Microsoft was to design fully functioning template code which developers could take and then run a simple sed script which would replace all occurrences of the word Sample with their word fragment of choice. Within 10 minutes they would have an easy to read fully functioning RPC client, RPC server and in our platforms case integrated with the registry, the eventlog, the service control manager and our lwmsg minimal client-server framework (as opposed to RPC). We plan to release this to the public domain so you can build open source programs or proprietary programs but build on top of the Likewise platform.
Thanks for reading..

SMB Blues: Much Ado about Nothing

SMB Blues: Much Ado about Nothing

Last week, Steven Vaughan-Nichols, a respected journalist wrote a blog posting on “The SMB Blues”.  My original intent was to ignore it. And my colleague Manny Vellon has already written a brilliant rebuttal. There were so many inaccuracies and his arguments were so biased that I’m inclined to believe that he had been misinformed.  Of course, this blog posting is Mr. Vaughan-Nichols personal opinion which he is totally entitled to.  But he’s seen fit to liberally quote me out of context, I thought I should weigh in as well. Most of his comments are not really important, but there are two particular areas which are noteworthy.

On “Forking  Samba” -Forking a code base means branching an existing code base and making modifications. Likewise has no Samba code in it whatsoever. Therefore it is not a fork. We are an alternative SMB infrastructure.  Samba is GPL3 and Likewise is GPL 2. We could not license code under the GPL 2.0 if we took any Samba code. Twice in his blog, he states that Likewise is forking Samba and then accurately states that the Likewise implementation is a clean room implementation.

On assigning copyrights – The intent of the GPL 2.0 is clear. If you take GPL 2.0 code and make modifications to it, then subsequently you are legally obliged to make available that source code to anyone you provide binaries to. So if Joe Programmer wishes to make modifications to Likewise code, and then distribute that code to the Acme Company, he is legally obligated to give those source code modifications to the Acme Company. He retains copyright on his modifications.

Now Joe doesn’t have to give any code back to Likewise. Remember he only has to give the source code to those who he has distributed object code to. He can most happily fork the code base and merge his own changes in and retain his copyright to his changes. That is the intent of the GPL. It’s just that Likewise won’t maintain his changes for him.

However if Joe wants Likewise to  take his modifications and push them into the Likewise source repository,  Likewise will require that Joe execute a contributor license agreement with Likewise . Several open source projects do exactly this. They do those so that they can prevent possible legal  disputes at a later time. It is normal practice.  Again, much ado about nothing.

In the same blog, Jeremy Allison touts personal copyright as an advantage of Samba. I am little disappointed with this answer. Personally as an engineer, I would have hoped that Mr. Allison would I’d hope that he would pick a technical debate on why Samba is a superior system.  He trumpets” They collaborate together for the good of all who use the code, with no one organization having control.”  The implication here is that there are good corporate citizens and those of us who ask for copyright assignments are machiavellian. But the truth of the matter is that most of these companies do sponsor their own open source projects where they require copyright assignment. Here is a smattering of open source projects and their copyright/contributor assignment policy.

1. Apache project contributor assignment agreement (by the way Likewise derived its copyright assignment from the Apache project )
2. Novell’s Mono project’s copyright assignment agreement-  http://mono-project.com/FAQ:_Licensing
3. MySQL contributor/copyright agreement http://www.sun.com/software/opensource/sca.pdf
4. IBM’s copyright/joint ownership agreement http://docs.google.com/gview?a=v&q=cache:8JeHedYFNxkJ:ibm-db.googlecode.com/files/IBM_DB_Python%2520Joint%2520Copyright%2520Assignment%2520-%2520ongoing.pdf+IBM+contributor+assignment&hl=en&gl=us&sig=AFQjCNEEx5o7zaLa29_DXlCXQAQfDufYFQ
5. Google  http://code.google.com/legal/individual-cla-v1.0.html

What I find amusing that Mr. Allison works for Google Open Source Program Office that administers the Google contributor agreement  program . Why is it okay for Mr. Allison’s company to require copyright/contributor license agreements for their open source projects, but it’s not okay for Likewise?  But Mr. Allison can rest easy. The fact of the matter is that it’s totally okay for Google to require a contributor license agreement and it’s totally okay for Likewise to require one.  Again, much ado about nothing!

Mr. Vaughan-Nichols blog redeems itself by making one excellent point. He asks the question why anyone would want to re-invent the wheel.  And that is the crux of the issue and I am grateful to him for focusing attention on this.

The simple answer is because we believe that the Likewise “architecture” is technically superior to the Samba architecture, and this is my opinion.  Others may think that this is not true or not possible and that’s okay – that’s your opinion. Over a period of three years and two Samba XP conferences, I described to the Samba team members (primarily to Jeremy Allison) that we would need several key pieces of technology infrastructure for our customers and end users.  I begged, pleaded, requested and warned that it would be difficult for us to continue to build on Samba if we didn’t have these features.  For these changes to be supported there would need to be radical architectural changes in Samba. It is my technical opinion (as someone who’s worked on Windows distributed systems for  decade) that it would be in our best interest, our customers and ultimately the Likewise community’s interest to start with a brand new blueprint. The Likewise architecture is diametrically different from the Samba architecture.  The next post will detail specific technical differences between Likewise and Samba.

If I could extend upon Mr. Vaughan-Nichols analogy, it’s not so much about re-inventing the wheel, but about the belief that it is time and it’s possible to build a superior form of transportation.

This does not take anything away from or diminish the contributions of the Samba team. And for the world at large, the Samba architecture may be right for them.  For seventeen long years they have labored alone in the cause of Linux-Windows interoperability. In my opinion, the Samba projects emphasis has been more about network forensics than about system architecture and that was the right strategy. How could you build a system if you didn’t understand what the traffic on the wire looked like? And when the protocol designers provided no documentation on the protocols, your first focus would be about deciphering the network traffic.   The seminal contribution of the Samba team has been the deciphering and analysis of the Windows server protocol suites. The entire open source community rightfully owes a debt of gratitude to Andrew Tridgell and the Samba team.

Mr. Vaughan-Nichols alludes to the Samba leadership being disappointed with this current turn of events.  That too is not accurate. By and large, the Samba team has been mostly positive and gracious. However like any large open community there is a fringe element that feels insecure. My hope is that they will come around to the realization that Samba is the prevailing open source Goliath for CIFS interoperability today and that this is not a zero-sum situation where Samba has to lose for Likewise to win and that the storage market is experiencing its greatest renaissance right now and there is room for all of us to prosper.

One of my favorite movies is “The American President”. In the movie, President Andrew Shepherd resists the urge to engage in a pointless debate on things that are really irrelevant. But in the end, he realizes that people need the truth and in the absence of the truth, they’ll believe anyone who has a microphone.[PS – I’m no fake  or real President , and I have no delusions of grandeur – they are regularly and effectively dispelled by my beautiful wife]

 But Mr. Allison, channeling my own Andrew Shepherd – Likewise has done you no harm.  We’ve worked hard and played by the rules and if you want to have a serious technical debate on the architecture of Likewise versus Samba, you’re going to have to come at us with much more than a copyright assignment agreement or unsubstantiated insinuations of “forking”. Just tell me when and where, and I’ll show up.

Postscript[7:04AM PST Sept 28, 2009]: Since this posting, Mr Vaughan-Nichol’s blog now seems to have been to corrected to “which in the past was based on Samba”, which perhaps is more accurate than saying “forked”, but still inaccurate since we’ve never ever released a file server product based on Samba. Likewise Open 5.3 is our first foray in an open source file server solution.

krb5-1.7 is now the default Kerberos stack for Likewise

Today, we dropped in krb5-1.7 into the Likewise Open 5.3  code base. By the end of today, we should be able to get our gss-ntlm stack working within the krb5-1.7’s SPNEGO router framework. Once we have this done, we’ll be able to cleanly demonstrate the NTLM authentication for our SMB server stack.

There are major ramifications for this work. Out of the box, we will be able to support NTLM authentication for DCE/RPC and NTLM authentication for open-ldap.

The lsass daemon is being restructured so that it is always on.  It made no sense to do this before, but 5.3 includes our local SAM database. Out of the box, we get local users and groups – therefore it makes sense that the daemon be always on.

By next Friday, we will have the  SMB server smoothly authenticating NTLM connections.

Stay tuned!

Getting ready for SNIA

Its has been over a month since I last posted. But things are as hectic as usual. We’ve been deluged with OEM customers licensing the LWIS and now L-CIFS software stacks

We’re about 3 weeks away from SNIA. Here is some of the highlights of what we’re working on.

- We’re running NetBench on a regular basis. This is hugely relevant. What it means is that our stack is now feature complete that it can cleanly pass NetBench. Our numbers are pretty amazing and  we’re yet to start detailed profiling of source code. Three things stand out – first robustness: the NetBench test suite generates a huge number of requests. We flawlessly handle all of these requests

second, protocol completeness:  we support every single protocol opcode that the NetBench test throws at us.

finally, performance: as I’ve said, we’ve more than exceeded our expectations  

- We’re re-written our entire NTLM authentication stack. We’re in the process of reintegrating it into our code base.  We support every possible NTLM scenario.  Six key scenarios are

1) Windows client currently logged on user (AD credentials) talking to  the LWIO file server

2) Windows client; user  provides supplemental credentials (AD credentials) talking to the LWIO file server

3) Windows client; user provides supplemental credentials (server’s local SAM database) talking to the LWIO file server

4) Linux client currently logged on user (AD credentials) talking to a Windows file server

5) Linux client; user provides supplemental credentials (AD credentials) talking to a Windows file server

6) Linux client; user provides supplemental credentials (server’s local SAM database) talking to the LWIO file server

- Finally, the new Likewise registry – we have most of the pieces in place. The next release 5.3 slated for the beginning of September will have the entire registry subsystem, a registry import-export tool and an interactive command-line shell program that allows you to browse, set and edit registry entries.  The subsequent release will have all of the Likewise daemons read their configuration information from the registry. Many customers have asked us to provide them with the ability to change the log level for debugging information at run time without stopping the daemon. We could do this smoothly, but persisting information to a text file was painful. The registry allows us to dynamically update configuration state through a running daemon without  stopping and restarting the daemon.

Thanks for reading.