Saturday, October 13, 2007

Functional Programming In Industry

Last week attended the International Conference on Functional Programming (ICFP Conference). I attended as what some functional programmers would call their enemy, since like most people, I stick to the widely used imperative languages. There's one difference though - I do have an interest in functional programming (see below)!

Putting my own experiences aside for the now, the focus of this post is functional programming in industry. As an outsider with an interest in the community I often wonder why it is so small and how the situation can be improved. Everyone keeps going on about how beneficial functional programming is over imperative programming: faster development, shorter code, less bug prone and easier to test to name just a few.

So with all this power, where's the massive rush? Most of the successful languages of late have been successful mainly due to their success in industry. Java is a prime example. So a good way to become successful is for functional languages to attract the attention of industry. What's holding industry back though?

The problem I fear is a viscous circle that's difficult to slingshot out of. The number of FP programmers is very low. Because of this functional programmers are more likely to work in very small groups or even on their own on major projects. This trend seems to continue in industry, especially when there is a very small number of functional programmers in a company vying for the use of FP on some project within the company.

So we have come to the conclusion that FP groups in industry are very small or even solo. But how is this a problem? While the team works on the project all appears to be going very well. What happens when someone leaves though? Someone has to take over his section of the project. This is where the problems are hit! Finding a replacement is a mission in its own right. This is mainly caused by the lack of functional programmers, but also due to the next problem - programming style.

Why is programming style a problem? Well, this is an issue with imperative languages as well. Everyone has their own style of programming and when you have a large mass of code it can be very difficult for someone to take over without decent documentation. This is why many software development companies have very strict style guidelines such that all programmers follow and become quickly familiar with the same, well documented style.

Where are the style guidelines for functional languages?? This is the crux of an interesting discussion I had with someone at the ICFP Conference last week. All the above leads to this sad conclusion. Style guidelines are seriously underused by functional programmers. And this is one serious problem, because it takes so long for someone to take over a mass of code if it doesn't follow such guidelines. This might sound finicky, but it's true! In the conversation he gave a couple of examples such as one where it took a year to find a replacement Lisp programmer and a year for that replacement to understand the code enough to start maintaining it properly!

So I did a quick search for style guides for some common FP langauges and this is what I dug up:

  • Haskell: This style guide comes with an automatic style checker. It appears to be rather rigorous, which is great!
  • OCaml: This covers some aspects, but does not appear to be as thorough as the Haskell guide.
  • Lisp: This is a fairly detailed style guide. It is still, however, in draft phase.
  • Erlang: Appears to be lacking!!
So style guides do exist for the major functional languages. But how widely are the adopted? It's difficult for me to say as I'm not in the community, but my general impression is that functional programmers are the type of people that like their own style and stick to it. With such small teams this works in the short term, but fails miserably in the long term.

So, back to the beginning of this post. Why aren't people rushing to FP? There are these small success stories in industry, but not very many that have lasted over the long term. I believe that this vicious circle of functional programmers not wanting to let go of their favoured styles and change to a style guide is holding many people back from joining this elite community.

Is there anything that can be done to improve the situation? I believe that the community is too full of high quality programmers that the jump is just too far that most people within reach are also the type of people that far prefer following their own style. Gosh, even I prefer following my own style!

My functional programming experiences:

While I still cannot claim to be able to program in a functional language, the idea has fascinate me for some time. Ever since I've learned Python I have made use of several functional aspects it offers. And I love it! Earlier this year I started perusing through some OCaml tutorials, however the syntax really put me off it. I soon after started looking at Haskell. After sifting through some mediocre tutorials I finally settled on YAHT. I reached chapter 8 after I got sucked into other things that prevented me from progressing any further for some time. So I know a fair amount of the theoretical side of Haskell, although I've had next to zero practical experience.

8 comments:

  1. Too much ego?

    I had my fun share learning scheme and haskell, and back in those days, there weren't any hint how to structure your codes, as long as they work. When I looked at my old codes, some of them have consistent style, and some of them haven't.

    I suppose to break the cycle, a full of initiative person can start up to build a reasonable coding style for a functional language (like what is done by Mike Vanier for his Haskell class in Caltech). And then, have any existing industry which uses that language to start enforcing them. Things should follow on from there.

    But then again, I suppose this is more like a hypothetical suggestion than a real suggestion, given the individualistic nature. What's important is to have some prominent figures to endorse a good style, and time should tell the rest of the story.

    ReplyDelete
  2. There is an Erlang style guide at

    http://www.erlang.se/doc/programming_rules.shtml

    I think the interesting question to ask is

    - are there companies who bet their entire existence on FPLs? not whether they use some style guide.

    Answer yes - several.

    Are they profitable?

    Yes

    Are they more profitable than the
    equivalent company using Java/C++?

    Pass - we thing so - but there is no hard evidence here.

    ReplyDelete
  3. Yes, ego has a lot to do with it. I mean, I also prefer going on and doing my own thing. I don't like having to follow what other people say. But it has proved beneficial with imperative languages and therefore I can't see why this doesn't extend to functional languages.

    Breaking the cycle is going to be difficult. With imperative companies, it is usually a select few pushing the use of FP. Since their managers probably don't know much about FP they aren't going to really enforce anything since they are the "clueless" ones. But yes, initiative needs to be taken to further develop such styles. The Haskell one is a great example of what can be achieved.

    I missed that Erlang guide, but a lot of it seems to be very generic stuff that applies to all languages.

    I know there are companies using FP and that they are profitable - I mentioned that in the post. However, the average functional programmer has a greater skill level and so it's difficult to make a comparison. And this is about expanding FP to the masses, not just an elite community. Existing functional programmers can set things up for new programmers to enter a FP project and that is where style guides can help.

    ReplyDelete
  4. I find this to be true with most people who tout these mighty also-ran languages: "...I know a fair amount of the theoretical side of Haskell, although I've had next to zero practical experience.".

    Interesting...

    ReplyDelete
  5. While learning any language I am interested in its current uses and where it's heading. I think it's only natural to question the future of a language for which one invests time in. And FP is a very interesting case. How much is it worth studying FP?

    ReplyDelete
  6. The style guide you've said is for OCaml is actually for Standard ML. You probably meant to link to this one:

    http://www.cs.caltech.edu/~cs20/a/style.html

    There is another one here:

    http://www.seas.upenn.edu/~cis500/cis500-f06/resources/programming_style.html

    I don't really see the point of trying to dictate how OCaml code should be formatted when OCaml can format code itself...

    ReplyDelete
  7. By "OCaml can format code itself" I suppose you mean things such as indentation, etc. By style I was referring to a more complete style than just format. Good examples of this in the Haskell guide are MAGIC_NUMBER, USELESS_CODE and USAGE_STMT. There are many more examples, but hopefully you get the idea.

    When I was at Google they had extremely comprehensive style guides. They are a pain in the beginning, but help a lot once you get used to it. I'd hate to work in even a small group of people working on the same code, each with very different style.

    ReplyDelete
  8. Just curious, can you list some profitable FP companies?

    ReplyDelete