Come one, come all, and witness the glorious sight that is Dinko Srkoč explaining what he does for his bread & butter! We are going to have a Sparky session this time, so go ahead – read the abstract below and get inspired!
* * *
Once upon a time, there was a little computer with a dream. It dreamed of processing vast amounts of data. Unfortunately, having only one CPU and limited RAM and storage made it very hard to live the dream. Rather than abandoning it, the computer decided to seek the help of its peers.
Quite unexpectedly, it turned out that many shared its dream. They all got together, shared hands and good wishes and made plans. For a moment life was good, full of positive emotions and optimism, but it was soon clear they had a problem. How to partition the data among themselves, how and when to share it, who will calculate what, how to coordinate, who will drive the process? What if data comes in as a stream, what algorithms to use on the data … and, by god, all those formats – who is going to read them all?
They realized that good will is not enough. They needed more, and they started the search. It was long, it was exhausting, but the effort payed off. In a remote Apache village they found the Spark.
… to be continued …
Join us at the meetup where we’ll continue with the tale, albeit expect a slightly more technical vocabulary.
PS: What does Spark have to do with the Scala meetup, you may ask. For one, Spark is written in Scala, and is, arguably, the most important driver for Scala adoption. Oh, and a certain figure, known as M.O. likes the way collections are done in Spark.
