Functors


Knoldus

While programming in Scala we  often come across a term called Functor. A functor is an extremely simple but powerful concept. In this blog, let us discuss that in more detail.

Theoretically functor is a type of mapping between categories. Given two categories A and B, a functor F maps the objects or entities of A to objects or entities of B. We can simply call it a function of objects or entities.

In programming the functors come into play when we have types or values wrapped inside context or containers. This wrapping up inside context, blocks application of normal functions on the values. This happens because the result on application of function is dependent on the context.

The solution to above scenario is a function that knows how to apply functions to the values wrapped in a context. Internally speaking this function should have a potential to

  • unwrap(fetch) the value…

View original post 265 more words

Building Analytics Engine Using Akka, Kafka & ElasticSearch


Knoldus

In this blog , I will share my experience on building scalable, distributed and fault-tolerant  Analytics engine using Scala, Akka, Play, Kafka and ElasticSearch.

I would like to take you through the journey of  building an analytics engine which was primarily used for text analysis. The inputs were structured, unstructured and semi-structured data and we were doing a lot of data crunching using it. The Analytics engine was accessible by the rest-client and web-client(Built In with engine)  as shown in below diagram.

architecture-00-b
Here is a quick overview on technology stack :

  1. Play Framework  as Rest Server & Web Application  (Play is MVC  framework based on  lightweight, stateless and web friendly architecture.)
  2. Akka cluster as processing engine.(Akka is a toolkit and runtime for building highly concurrent,distributed, and resilient message driven applications on the JVM.)
  3. ClusterClient (It was contributed module) for communication with Akka cluster. It used to run on rest…

View original post 987 more words

Introduction to Spark 2.0


Rklick Solutions LLC

Overview of Dataset , Dataframe and RDD API :

Resilient Distributed Datasets (RDD) is a fundamental data structure of Spark. It is an immutable distributed collection of objects. Each dataset in RDD is divided into logical partitions, which may be computed on different nodes of the cluster.

But due to facing issue related to advanced optimization move to dataframe.

Dataframe brought custom memory management and runtime code generation which greatly improved performance. So in last year most of the improvements went into Dataframe API.

A DataFrame is a Dataset organized into named columns. It is conceptually equivalent to a table in a relational database or a data frame in R/Python, but with richer optimizations under the hood. DataFrames can be constructed from a wide array of sources such as: structured data files, tables in Hive, external databases, or existing RDDs.

Though dataframe API solved many issues, it was not a…

View original post 742 more words

Effective Programming In Scala – Part 3 : Powering Up your code implicitly in Scala


Discover type annotations versus ascriptions, adding methods to class in implicit way and monkey patching.

Knoldus

Hi Folks,
In this series we talk about the concepts that provide a better definition to the code written in scala. We provide the methods with some definitions that lead to perform a task in a better way. Lets have a look at what we have done in the series so far,

Effective Programming in Scala – Part 1 : Standardizing code in better way
Here we covered the better solution to code, so that it cannot lead the any kind of code styling or formatting errors behind. Provided the way to use scala properties and collections in a better way according to their behaviour.
Effective Programming In Scala – Part 2 : Monads as a way for abstract computations in Scala
Here we provided short hand solutions to the long and lengthy writing of code to perform some task in a predefined sequence. Provided the better use of comprehensions…

View original post 762 more words

Initializing data with eager binding in play


Knoldus

I just came across a scenario where I had not to choose Play GlobalSettings to execute some tasks on start up of the application, as it is being deprecated in Play 2.5. So as the alternative what we can do to make things happen at the time when application just gets started.

So whatever was done using GlobalSetting.onStart and GlobalSetting.beforeStart needs to be implemented using the dependency injection.

For this, we will just provide the eager binding for the class implementation that needs to perform some tasks just before the application actually gets started.

(Eager binding is used to start up a service when an application starts.)

Then provide those implementations in the constructor of the class for which we have provided eager binding.

So lets start going step by step to replace our onStart/beforeStart implementations with dependency injected class.

Firstly create a class say, MyDBModule which extends AbstractModule in which we will bind the class implementation that will…

View original post 248 more words

Integrate JWT with Akka HTTP


Knoldus

In this article we will discuss about , how to implement  authentication or authorization in Akka HTTP routes using JWT .

As we know Akka HTTP is full implementation of  server  and client side HTTP stack on top Akka actor and Akka stream . Now Let’s  we  talk about JWT.

what is JWT ?

JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA.

What is the JSON Web Token structure?

JSON Web Tokens consist of three parts separated by dots “.”, which are:

  • Header
  • Payload
  • Signature

Let’s dive into  detail of each part.

Header :  The header typically…

View original post 214 more words

A basic application to handle multipart form data using akka-http with test cases in Scala


Knoldus

In my previous blogs, I have talked about file upload and its test cases using akka-http.

Now, in this blog I am going to explain the handling of multi part form data in akka-http with the help of a basic application which contains the code and its test cases.

So Let’s start with the dependencies for the application.
You have to add the following dependencies in you build.sbt :

Now, we have to create a handler to handle multipart form data :

Whenever a file will come in request, a new file will be created in temp directory of your system.

Below, you can find the test cases for the handler :

I hope, you enjoyed it and it will be helpful for you.

You can get full code here

Happy Blogging !!!

View original post

Dynamic Typing in Scala


Knoldus

Scala provides different ways to share common data between the files. In a simplified way, a class is often used to share the values by storing them in methods in that class. But those methods are statically typed. What we mean by statically typed ?

Here we mean by statically typed, is that a class having a method declared and used everywhere we want to assign information and fetch from it. Now if there is a requirement to have a unique method every time we store some information in and fetch.

As Scala provides the dynamic typing, we have the following profits,

  • Code written in Scala will be more understandable due to well formed method names and arguments,
  • Dynamic typing will create the method at the execution time, we need not to type it statically,
  • Need not to create many variables, methods to store or calculate some common calculations with…

View original post 519 more words

Introduction to Knockout js with Play template


Knoldus

Knockout is a JavaScript library that is based on Model–view–viewmodel (MVVM) that helps to build responsive UI display. This makes clear separation between model(Domain model/ stored data), view(User Interface) and view-model(presentation of model in which binder communicates between view and data-binder).

Why Knockout js?

1. Declarative bindings : This is a convenient way to bind the html to the data model. When we try to manipulates the DOM with the declarative binding all the bounded element remain connected

2.Dependency Tracking : Whenever you modify or update your data model, It will automatically updates the associated UI.

3.MVVM Design Pattern: knockout build a connection between ViewModel and View, and it any updation take place it will automatically update the UI element.

4.Extensible – This implements custom behaviors as new declarative bindings for easy reuse in just a few lines of code..

5. and many more

1…

View original post 214 more words