Visualizing F# Advent Calendar contributors

This post is part of the F# Advent Calendar in English 2015 project. Check out all the other great posts there! And special thanks to Sergey Tihon for organizing this.

A few weeks ago, while at BuildStuff, I walked next to an image built with words, like a tag cloud picturing some company’s logo. My first reaction was: “That looks nice, but has someone been paid to manually placed all these words? I bet it can be automated…”

I started putting together a script, in order to fill an image with words. My first attempt had disastrous performance and memory characteristics. Several hours of work later (quite a few, actually), and with contributions from my co-workers, we ended-up with a much better algorithm.

F# is a wonderful language to experiment with ideas. The convenience of quickly defining types with almost no ceremony lets you focus on what you’re really trying to build. The power of the .NET ecosystem lets you use all the standard APIs that you expect to find on a serious platform. This is really enjoyable and helps you to build simple things first, and evolve to more complicated ones while staying focused.

In the end we’re able to produce images such as:

In order to produce such an image, we need an image to use as the model, and a list of words to fill the shape with. We had much fun doing it, both trying to identify strategies to make the images look good and get the generation time down. You can find the full source code on Github: https://github.com/pirrmann/Wordz

The goal: build a nice picture

Another area where F# shines is related to getting data from various sources and manipulate it. This enables so many scenarios. What if we could get pictures, and related text, from some place such as https://sergeytihon.wordpress.com/2015/10/25/f-advent-calendar-in-english-2015/ for instance, and generate all sorts of images?

Step 1: parse the the posts list

I’m happy @TeaDrivenDev hasn’t published his first post idea about generating the RSS feed from the FsAdvent page, because I also have to parse the page in order to get my data… so first thing, we need to get the posts URLs from the page. It gives me a chance to mention the HTML Type Provider, which is part of the FSharp.Data library. Accessing the list of posts and downloading all the posts is as easy as:

type adventCalendarPage =
    HtmlProvider<"https://sergeytihon.wordpress.com/[...]/">
type PageRow =
    adventCalendarPage.FAdventCalendarInEnglish2015.Row

let postsTable =
    adventCalendarPage
        .GetSample()
        .Tables
        .``F# Advent Calendar in English 2015``

let extractPostsLinks (tr:HtmlNode) =
    let lastTd = tr.Elements("td") |> Seq.last

    lastTd.Elements("a")
    |> List.map (fun a -> a.AttributeValue("href"))

let postLinks =
    postsTable.Html.Descendants("tr")
    |> Seq.tail
    |> Seq.map extractPostLinks

let downloadPostsAsync links = 
    links
    |> Seq.map HtmlDocument.AsyncLoad
    |> Async.Parallel

let postsDownloads =
    postLinks
    |> Seq.map downloadPostsAsync
    |> Async.Parallel
    |> Async.RunSynchronously

let allPosts = Array.zip postsTable.Rows postsDownloads

From the previous sample, you may notice how the HTML type provider also allows you to access the elements of the page through the DOM. This is quite representative of how a good library design gracefully lets you use the underlying technology when the high-level API doesn’t suit your needs.

Step 2: extract words

We have now obtained all published FsAdvent posts in the form of HtmlDocument objects, but we don’t have their content yet… No magic type provider here, all posts are hosted on different blogging platforms, I had to use several strategies and fallbacks to identify the HTML element containing the post in each case. Semantic web is really not there yet! The search function finally looks like this:

let search =
    findDivWithEntryClassUnderDivWithPostClass
    |> fallback findWintellectContent
    |> fallback findDivWithClassStoryContent
    |> fallback findArticle
    |> fallback findElementWithClassOrIdPost
    |> fallback findH1TitleParentWithEnoughContent
    |> fallback findScottwContent
    |> fallback findMediumContent
    |> fallback wholePageContent

The next step of interest is how to extract the most important words from each post. Another handy feature of F# are computation expressions, and in particular the built-in ones such as async an seq. Returning a sequence while recursively traversing a tree is as easy as:

let getMultiplier (node:HtmlNode) =
    match node.Name() with
    | "h1" -> 16 | "h2" -> 12 | "h3" -> 8
    | "h4" -> 4  | "h5" -> 3  | "h6" -> 2
    | _ -> 1

let rec collectAllWords multiplier (node:HtmlNode) = seq {
    match node.Elements() with
    | [] ->
        let text = node.InnerText()
        let words =
            text.Split(delimiters)
            |> Seq.filter (not << String.IsNullOrWhiteSpace)
            |> Seq.filter (fun s -> s.Length > 1)
            |> Seq.map (fun s -> s.ToLowerInvariant())
        for word in words do
            if not(filteredWords.Contains word) then
                yield word, multiplier

    | children ->
        let multiplier' = getMultiplier node
        yield! children
               |> Seq.collect (collectAllWords multiplier')
}

The last step once we’ve obtained all the words from each post with is to group them and sum their weight in order to get an indicator of their importance in the post. We define a function for that:

let sumWeights words =
    words
    |> Seq.groupBy fst
    |> Seq.map (fun (word, weights) -> word,
                                       weights |> Seq.sumBy snd)
    |> Seq.sortByDescending snd
    |> Seq.take 100
    |> Seq.toList

Step 3: get Twitter profile pictures

So we have all words from the FsAdvent posts published so far, and their relative importance in each post. But in order to create a visualization, we need model images, right? Let’s download the Twitter profile pictures of the authors! The profile urls can be extracted from the FsAdvent page (actually some of them are not links to twitter, so I had to cheat a bit, and this is why the function extractCleanLink is omitted…)

let extractProfileLink (tr:HtmlNode) =
    let lastTd = tr.Elements("td") |> Seq.item 1
    let a = lastTd.Elements("a") |> Seq.head
    a.InnerText(), a.AttributeValue("href")

let profilesLinks =
    postsTable.Html.Descendants("tr")
    |> Seq.tail
    |> Seq.map extractProfileLink

let downloadAndSavePictureAsync (name:string, twitterProfileUrl:string) = async {
    let fileName = getFileName "profiles" name

    let cookieContainer = new Net.CookieContainer()

    let! twitterProfileString = Http.AsyncRequestString(twitterProfileUrl,
                                                        cookieContainer = cookieContainer)
    let twitterProfile = HtmlDocument.Parse(twitterProfileString)

    let profileImageLink =
        twitterProfile.Descendants("a")
        |> Seq.find(fun a -> a.HasClass("ProfileAvatar-container"))
            
    let profilePictureUrl = profileImageLink.AttributeValue("href")

    let! imageStream = Http.AsyncRequestStream profilePictureUrl

    use image = Image.FromStream(imageStream.ResponseStream)
    image.Save(fileName, Imaging.ImageFormat.Png)

    return () }

profilesLinks
|> Seq.map (extractCleanLink >> downloadAndSavePictureAsync)
|> Async.Parallel
|> Async.RunSynchronously
|> ignore

Step 4: cut the profile pictures into layers

My early experiments with this did not have any feature to cut pictures into layers, but I didn’t want to manually edit each picture profile! So what I’ve done to build “layers” is just to cluster pixels by color similarity (actually I’ve tried to really cluster them with a K-means cluster but I ended up using my custom clustering with magic numbers, as it performs better on that specific set of pictures).

This topic could be a blog post of its own so I’ll just sum up the steps I’ve performed:

  • In order to get rid of outliers, first blur the image
  • then generate color clusters using the following algorithm:
    • considering each pixel, if it’s close enough (below a given threshold) to an existing cluster, put it in the cluster. If too far from any existing cluster, create a new cluster with the single pixel in it.
    • after having seen each pixel, merge clusters that are close enough
    • finally, merge clusters that are too small with their nearest neighbour, regardless of the distance

This way, we can go from one image to several ones with the grouped pixels:

Step 5: fill images with words

As some authors have decided to blog more that once, I’ve decided to merge their posts and generate a single image per author.

let mergePosts posts =
    let allImportantWords =
        posts
        |> Seq.collect (fun post -> post.Words)
        |> Seq.toList
    { Seq.head posts with Words = sumWeights allImportantWords }

let mergedPosts =
    parsePosts allPosts
    |> Seq.groupBy (fun post -> post.Author)
    |> Seq.map(fun (_, posts) -> mergePosts posts)
    |> Seq.toArray

Using sumWeights to merge posts is like considering that each post is a paragraph containing only its 100 most important words.

I then consider each layer generated previously from the Twitter profile picture of an author, and fill it with the most important words from his/her posts. This generates one file per layer on the disk.

Step 6: generate the calendar!

So… 62 posts (before the last one) – 2 which were not published – 1 because I decided to group Steffen’s posts (even if scheduled on 2 different dates) + 1 because Steffen’s post have been unmerged (as I neeeded 60 posts in the end) = 60 posts (yeah!). This gives me a nice 8 * 8 square, with a 2 * 2 tile in the middle for the F# foundation! All is needed is to iterate on the generates layers filled with words, and place them on the calendar. I’ve randomly shuffled the posts list, can you identify everybody? Here is the result, enjoy!

(click on the calendar to view the full size image)

I’ve just updated this post with all posts written so far…

Posted in F sharp love | Tagged , | 6 Comments

Fun with turtles

F# is a great language to build internal DSLs, but I had no idea that you could go as far as what I’m going to present here…

Let’s start from the start: my goal was to design some way for me to explain my daughter what programming is about and how it works. There are some nice graphical tools for that, like Scratch or more recently Hour of code, however I wanted to show something which is closer to what I actually do on a daily basis: write (and read) code. There are some nice educational programming languages, some of them are even usable in a localized way (French, in my case). LOGO is a good example of this, and I could have used an existing tool, but where is the fun if you don’t build your own?

It appears that building an internal LOGO-like DSL is surprisingly easy, and requires almost no code! What you need is just to define the basic types to describe your actions:

type Distance_Unit = STEPS
type Rotation_Unit = GRADATIONS
type Rotation_Direction = | LEFT | RIGHT
let STEP = STEPS
let GRADATION = GRADATIONS

type Color = | RED | GREEN | BLUE

type Action =
    | Walk of int * Distance_Unit
    | Turn of int * Rotation_Unit * Rotation_Direction
    | LiftPenUp
    | PutPenDown
    | PickColor of Color

type Turtle = Action seq

And then a computation expression to do the trick of transforming sentences to sequences of actions:

type AS_word = AS
type TO_word = TO
type THE_word = THE
type PEN_word = PEN
type UP_word = UP
type DOWN_word = DOWN
type TIMES_word = TIMES
type WHAT_word = WHAT
type DOES_word = DOES

type TurtleBuilder() =
    member x.Yield(()) = Seq.empty
    [<CustomOperation("WALK", MaintainsVariableSpace = true)>]
    member x.Walk(source:Turtle, nb, unit:Distance_Unit) =
        Seq.append source [Walk(nb, unit)]
    [<CustomOperation("TURN", MaintainsVariableSpace = true)>]
    member x.Turn(source:Turtle, nb, unit:Rotation_Unit, to_word:TO_word,
                  the_word:THE_word, direction:Rotation_Direction) =
        Seq.append source [Turn(nb, unit, direction)]
    [<CustomOperation("LIFT", MaintainsVariableSpace = true)>]
    member x.LiftPenUp(source:Turtle, the_word:THE_word, pen_word:PEN_word,
                       up_word:UP_word) =
        Seq.append source [LiftPenUp]
    [<CustomOperation("PUT", MaintainsVariableSpace = true)>]
    member x.PutPenDown(source:Turtle, the_word:THE_word, pen_word:PEN_word,
                        down_word:DOWN_word) =
        Seq.append source [PutPenDown]
    [<CustomOperation("PICK", MaintainsVariableSpace = true)>]
    member x.PickColor(source:Turtle, the_word:THE_word, color:Color, pen_word:PEN_word) =
        Seq.append source [PickColor color]
    [<CustomOperation("DO", MaintainsVariableSpace = true)>]
    member x.Do(source:Turtle, as_word:AS_word, turtle:Turtle) =
        Seq.append source turtle
    [<CustomOperation("REPEAT", MaintainsVariableSpace = true)>]
    member x.Repeat(source:Turtle, nb:int, times_word:TIMES_word, what_word:WHAT_word,
                    turtle:Turtle, does_word:DOES_word) =
        Seq.append source (List.replicate nb turtle |> Seq.collect id)

let turtle = new TurtleBuilder()

And with nothing more, you can now write this kind of plain English instructions:

turtle {
    LIFT THE PEN UP
    WALK 4 STEPS
    TURN 3 GRADATIONS TO THE RIGHT
    PICK THE GREEN PEN
    PUT THE PEN DOWN
    WALK 4 STEPS }

Now, this doesn’t solve my initial problem, I want a French DSL. But I just need to define another builder, and a few translation functions:

type Tortue = Turtle

type Unite_De_Distance = PAS with
    member x.enAnglais = match x with | PAS -> STEP

type Unite_De_Rotation = | CRANS with
    member x.enAnglais = match x with | CRANS -> GRADATION
let CRAN = CRANS

type Sens_De_Rotation = | GAUCHE | DROITE with
    member x.enAnglais = match x with
                         | GAUCHE -> LEFT
                         | DROITE -> RIGHT

type Couleur = | ROUGE | VERT | BLEU with
    member x.enAnglais = match x with
                         | ROUGE -> RED
                         | VERT -> GREEN
                         | BLEU -> BLUE

type Mot_A = A
type Mot_DE = DE
type Mot_LE = LE
type Mot_STYLO = STYLO
type Mot_COMME = COMME
type Mot_FOIS = FOIS
type Mot_CE = CE
type Mot_QUE = QUE
type Mot_FAIT = FAIT

type TortueBuilder() =
    member x.Yield(()) = Seq.empty
    member x.For(_) = Seq.empty
    [<CustomOperation("AVANCE", MaintainsVariableSpace = true)>]
    member x.Avance(source:Tortue, de:Mot_DE, nb, unite:Unite_De_Distance) =
        Seq.append source [Walk(nb, unite.enAnglais)]
    [<CustomOperation("TOURNE", MaintainsVariableSpace = true)>]
    member x.Tourne(source:Tortue, de:Mot_DE, nb, unite:Unite_De_Rotation,
                    a:Mot_A, sens:Sens_De_Rotation) =
        Seq.append source [Turn(nb, unite.enAnglais, sens.enAnglais)]
    [<CustomOperation("LEVE", MaintainsVariableSpace = true)>]
    member x.Leve(source:Tortue, le:Mot_LE, stylo:Mot_STYLO) =
        Seq.append source [LiftPenUp]
    [<CustomOperation("POSE", MaintainsVariableSpace = true)>]
    member x.Pose(source:Tortue, le:Mot_LE, stylo:Mot_STYLO) =
        Seq.append source [PutPenDown]
    [<CustomOperation("PRENDS", MaintainsVariableSpace = true)>]
    member x.Prends(source:Tortue, le:Mot_LE, stylo:Mot_STYLO, couleur:Couleur) =
        Seq.append source [PickColor couleur.enAnglais]
    [<CustomOperation("FAIS", MaintainsVariableSpace = true)>]
    member x.Fais(source:Tortue, comme:Mot_COMME, tortue:Tortue) =
        Seq.append source tortue
    [<CustomOperation("REPETE", MaintainsVariableSpace = true)>]
    member x.Repete(source:Tortue, nb:int, fois:Mot_FOIS, ce:Mot_CE, que:Mot_QUE,
                    tortue:Tortue, fait:Mot_FAIT) =
        Seq.append source (List.replicate nb tortue |> Seq.collect id)

let tortue = new TortueBuilder()

And I can write my instructions in French!

tortue {
    AVANCE DE 5 PAS
    TOURNE DE 6 CRANS A DROITE
    AVANCE DE 5 PAS
    TOURNE DE 6 CRANS A DROITE
    AVANCE DE 5 PAS
    TOURNE DE 6 CRANS A DROITE
    AVANCE DE 10 PAS }

The next steps involved:

  • Writing a Windows Forms client to actually see the turtle draw things
  • Making it possible to send actions to the turtle using FSI
  • Hand-coding every letter of the alphabet
  • Adding a new keyword to my turtle builders

And please welcome my “Hello world” sample:

turtle {
    WRITE "MR T. SAYS:\nHELLO WORLD!\n\n"
}

As usual, all the code is available on my github.

Posted in F sharp love, Syntax Puzzles | Tagged , , | Comments Off

F# |> I <3

I’ve been writing about F# for a little while now, and how it has influenced the way I code in C# on an everyday basis for 3 years. During the last couple weeks, I’ve finally had the chance to use F# at work for a real project (I’ll probably talk about that later). Yesterday evening (Europe time) at Build, Roslyn was open-sourced. That’s nice and people are probably going to talk about it for a while. But for me there was another very nice announcement : Visual F# is now taking contributions! F# is already an exciting language, full of clever features. With this new step, I’m convinced that it will go even further and I’m already looking forward to F# 4.0!

So don’t expect to see many more posts about C# on this blog, I’ve totally gone F#.

And now, the latest sample from Arolla’s Code Jam. The aim was to write code to express values with a range of uncertainty, and operations which manipulate those numbers. For instance, a value 3.5 +/- 0.5, multiplied by 1.0 +/- 0.1. The goal of the kata was to write code that expresses intent (the why), and not implementation details (the how).

In F#, I’ve just defined a type to wrap both the value and its precision, and an operator to conveniently build those values :

type Number = | Number of float * float

let (+-) n a = Number (n, a)

This code alone already allows me to build values just by writing expressions such as “4.0 +- 1.0”. The next step for me was to write some expressive tests, using FsUnit.

let [<test>] ``A number with accuracy is equal to itself`` () =
    (1.0 +- 0.5) |> should equal (1.0 +- 0.5)

let [<test>] ``Sum of two numbers sums the numbers and accuracies`` () =
    ((5.0 +- 0.5) + (3.0 +- 0.5)) |> should equal (8.0 +- 1.0)

Although we could handle distinctly every operator with specific logic, the goal of this kata is to express the intent in a generic fashion, so here is a combine method that has no specific meaning of what an operator does:

let combine op (Number(n1, a1), Number(n2, a2)) =
   let all =
      seq {
         for x in [n1-a1; n1+a1] do
         for y in [n2-a2; n2+a2] do
         yield op x y
      }
   let minimum = all |> Seq.min
   let maximum = all |> Seq.max
   Number((minimum + maximum) / 2.0, (maximum - minimum) / 2.0)

Finally, we just have to add operator overloading to the type, the full code becomes:

type Number = | Number of float * float
    with override x.ToString() = match x with | Number(a, b) -> sprintf "%f +- %f" a b
         static member private combine op (Number(n1, a1), Number(n2, a2)) =
            let all =
                seq {
                    for x in [n1-a1; n1+a1] do
                    for y in [n2-a2; n2+a2] do
                    yield op x y
                }
            let minimum = all |> Seq.min
            let maximum = all |> Seq.max
            Number((minimum + maximum) / 2.0, (maximum - minimum) / 2.0)
         static member (+) (x, y) = (x, y) |> Number.combine (+)
         static member (-) (x, y) = (x, y) |> Number.combine (-)
         static member (*) (x, y) = (x, y) |> Number.combine (*)
         static member (/) (x, y) = (x, y) |> Number.combine (/)
         static member Pow (x, y) = (x, y) |> Number.combine ( ** )

Thanks to @luketopia for the interactions on Twitter, and the trick which allows to pass the operator ** as an argument: use spaces between the parenthesis and the operator!

Posted in F sharp love | Tagged , | Comments Off

Pattern matching in C# – part 4 – more tuples

DISCLAIMER: this post has been written months ago and never posted. By that time I was still trying to bring F# goodness to C#. This might be easier today with Roslyn, but that is not the path I’ve taken. Since the last post of this series, I’ve been doing more and more F#, even during my day job. I don’t think I’ll ever post this kind of ‘’How to do F# in C#” posts. What you can expect from this blog is now mostly functional programming in F#…

Today’s post will be a short one, as an answer to question from several months ago:

But if we want to handle tuples with more values… do we have to write all the overloads ?!

The answer is yes. With the current implementation, there has to be a method overload for every combination of patterns that you want to be able to match…

But wait a minute… For a given tuple of n values, there are 3n combinations… That makes a lot of methods to write ?!

You don’t have to actually write all these overloads yourself! T4 templates are there exactly for that purpose. If we want to generate all the extensions methods for the tuples of 2 elements up to 4 elements, for instance, we just have to build a .tt file, and define the 3 individual patterns as a string array :

string[] argumentsPatterns = new[]
{
    "T{0} testValue{0}",
    "Expression<Func<T{0}, bool>> testExpression{0}",
    "MatchCase any{0}"
};

Then and include a first loop :

for(int cardinality = 2; cardinality <= 3; cardinality++)
{
    [...]

As we are going to write the type arguments quite a few times, we store them in a string, as well as the type of the selector argument:

string typeArguments = string.Join(
    ", ",
    Enumerable.Range(1, cardinality)
              .Select(n => string.Concat("T", n.ToString())));

string selectorArgument = string.Concat(
    "Expression<Func<",
    typeArguments,
    ", TResult>> selector");

Inside the first loop we then loop over all the 3cardinality combinations :

for(int caseNumber = 0;
    caseNumber < System.Math.Pow(3, cardinality);
    caseNumber++)
{
    [...]

And we decompose the caseNumber into an array of int whose values can either be 0, 1 or 2, corresponding to the index of the pattern associated with each argument. For instance, with a cardinality of 4 and a caseNumber or 4, we would get an array containing [0, 0, 1, 1] :

int[] argumentsPatternsCases = new int[cardinality];
int argumentFlags = caseNumber;
for(int argumentIndex = 0;
    argumentIndex < cardinality;
    argumentIndex++)
{
    argumentsPatternsCases[cardinality - argumentIndex - 1] =
        argumentFlags % 3;

    argumentFlags = argumentFlags / 3;
}

From all this, we can generate the signature of the method :

public static PatternMatcher<Tuple<<#= typeArguments #>>, TResult>
    Case<<#= typeArguments #>, TResult>(
    this PatternMatcher<Tuple<<#= typeArguments #>>, TResult> pattern,
<#
    for(int argumentIndex = 0; argumentIndex < cardinality; argumentIndex++)
    {
#>
    <#= string.Format(argumentsPatterns[argumentsPatternsCases[argumentIndex]], argumentIndex + 1) #>,
<#
    }
#>
    <#= selectorArgument #>)

To understand what happens for the body of the method, the best is to look at the generated code for a particular sample :

public static PatternMatcher<Tuple<T1, T2>, TResult> Case<T1, T2, TResult>(
    this PatternMatcher<Tuple<T1, T2>, TResult> pattern,
    Expression<Func<T1, bool>> testExpression1,
    T2 testValue2,
    Expression<Func<T1, T2, TResult>> selector)
{
    ParameterExpression testParam = Expression.Parameter(typeof(Tuple<T1, T2>), "t");
    MemberInfo[] members = GetTupleMembers<T1, T2>();

    List<Expression> testExpressions = new List<Expression>();
    testExpressions.Add(
        Expression.Invoke(
            testExpression1,
            Expression.MakeMemberAccess(testParam, members[0])));
    testExpressions.Add(
        Expression.Equal(
            Expression.MakeMemberAccess(testParam, members[1]),
            Expression.Constant(testValue2)));

    Expression aggregateExpression = testExpressions.CombineAll();

    var testExpression = Expression.Lambda<Func<Tuple<T1, T2>, bool>>(
        aggregateExpression,
        testParam);

    var selectorExpression = GetSelectorExpression<T1, T2, TResult>(selector, members);

    return new PatternMatcher<Tuple<T1, T2>, TResult>(
        pattern,
        new MatchCase<Tuple<T1, T2>, TResult>(
            testExpression,
            selectorExpression));
}

I’ve just extracted two methods here :

  • an extension method CombineAll that combines test expressions into a single one by aggregating them using AndAlso expressions :
private static Expression CombineAll(
    this IEnumerable<Expression> expressions)
{
    Expression aggregateExpression =
        expressions.Aggregate(
            (Expression)null,
            (a, e) => a == null ? e : Expression.AndAlso(a, e));

    return aggregateExpression ?? Expression.Constant(true);
}
  • a GetSelectorExpression method, because the selector expression is generated exactly in the same way for each case number of a given cardinality.

The whole sample is available on my Github, including hand-crafted unit-tests for tuples of cardinality 2 and 3…

Posted in Functional Inspiration, Syntax Puzzles | Tagged , , , , | 2 Comments

Scala.io, seen by a .NET guy

As you might have noticed, I’m a .NET guy. I work on a Microsoft platform, with almost only Microsoft tools. But I happen to also be a functional programming fanboy now. My main interest is in F#, but I’m also curious about many other platforms…

Last year, I took Martin Odersky’s Scala course on coursera, and enjoyed it. I also watched the whole series of Haskell lectures by Erik Meijer on Channel9. And in a week, for my birthday, both of them offer me a new “Reactive programming in Scala” course on coursera!

With all this background, I went last week to Scala.io, a brand new conference in Paris, mostly about Scala, but also other functional languages. As a not-working-on-the-JVM guy, and not using Scala in my work, I was not focused on tools and practical uses of  Scala, but rather on getting ideas from a wide range of practices and languages, and interoperability in mind. As you’ll see, I chose a polyglot track.

I’ve attended several talks during the two days, but I won’t go into the details of each of these. All the videos should be available soon, so if any of these talks seem interesting to you, watch them!

Keynote day 1, Victor Klang

The opening keynote, given by Victor Klang, head of engineering at Typesafe, focused on the importance of failure, and the need to embrace validation and error handling as part of the software. He envisioned a world of micro-services, concurrent and compartmentalized, location transparent: typed endpoints producing typed streams of data. Of course, he insisted on the importance of type-safety…

ZeroMQ and Scala, François Armand

ZeroMQ is a reaction to Message Oriented Middlewares, and the AMQP protocol, which took ages to be specified. ZeroMQ is not a MOM, it is a socket library acting as a concurrency framework. Is is asynchronous and scalable, and has numerous languages integrations. For this reason, it is a convenient communication channel between heterogeneous systems. The talk presented several demos of different communication patterns, using both Scala (with or without Akka) and Python.

SPL: Scala Powered Lisp, Stefan Chris

This talk was about a project to parse and interpret a LISP is Scala. Its goal was to show how parser combinators could be used in Scala to build an AST, and how to build an interpreter and a REPL on top of it.

After a quick intro about Lisp, and sample expressions evaluated in a REPL, Stefan showed us the definitions of the basic data types of the language, and the Lisp AST he had built. In Scala, all the types were represented as case classes, that can be pattern-matched. After showing the basic constructs, he moved on to anonymous functions, bindings, and more complex expressions. Then came the interpreter, environment and closures.

Lazy on a Thursday afternoon

During the afternoon, I saw some Clojure (I wasn’t that impressed, I should see another more advanced talk some time), reified trees sent to the GPU, JVM Garbage Collection theory and tuning, and then came…

Brain damage with Haskell, Yves Parès

This talk was a general presentation about Haskell, accelerated. It was a very nice talk, first focusing on the non-strict evaluation model of Haskell, where the evaluation is in the hands of the programmer. Yves then showed us several standard functions, and… finally introduced the classical vocabulary of Haskell developers: functors, applicatives, monad. He finally presented several monads usage: IO monad, state monad.

And after some more Scala patterns, and a nice evening party with the attendees and speakers…

Keynote day 2, Sadek Drobi

This keynote was about web modern challenges, and the motivations behind the Play2 framework. To sum up, Play2 is about composability and functional programming toolset. This really seems to be a pretty nice framework.

Functional groovy, Guillaume Laforge

This was a very nice talk! I had never taken the time to look at Groovy, well this is now solved. Groovy is a superset of Java with APIs and toolset. It is a great fit for DSLs, and offers a seamless integration with Java. It has many functional characteristics: closures are first-class citizens, it supports genericity through duck-typing and operator overloading.

A very clever feature is the ability to write the last argument of a function, if it is a closure, after the closing parenthesis of the function call. This brings a very nice syntax. Groovy also supports transformations (macros), which allows to automatically add behaviors such a structural equality or memoization to existing code. It’s definitely worth a look.

Several other talks

I attended a talk about Gatling2, a stress test tool that builds a DSL on top of Scala, a talk about Scalaz and Shapeless, Nicolas Martignole talking about his experience working in a startup, and finally Fabrice Croiseaux talking about an app combining Scala, Akka and Backbone for a reactive web application. I have really enjoyed all these talks, but I won’t go into the details of each of these.

And a conclusion!

These two days were really nice, there are so many new things I want to learn now… I was already playing with parser combinators in F#, I only want to play with them even more. I am also going to look at structures such as HLists. And of course, reactive programming in Scala.

I really think that it is when we cross boundaries and go outside of our comfort zone that real useful exchanges are made. I had a good time at Scala.io and hope to be there again next year, maybe even doing a talk about F#!

Posted in Events, Functional Inspiration | Tagged , , , , , | Comments Off