Robust and declarative machine learning pipelines for predictive buying

Proof of concept of how to use Scala, Spark and the recent library Sparkz for building production quality machine learning pipelines for predicting buyers of financial products.
The pipelines are implemented through custom declarative APIs that gives us greater control, transparency and testability of the whole process.

Read Full Story

Setting up IntelliJ for Spark

Brief guide to setting up IntelliJ to build Spark applications.
Create new Scala Project
Select:
Create New Project
Scala Module
Give it an appropriate name
Setup Directory Structure
Move to the project root. Run the following:mkdir -p src/main/scala
mkdir -p src/test/scala
mkdir project
Setup gen-idea plugin
In the project directory you just created, create a new file called plugins.

Read Full Story

Scala and Pyspark specialization certification courses started

 
Scala & Spark Specialization 
Data science is a promising field, Where you have to continuously update your skill set by learning the new technique, algorithms, and newly created tools. As the learning journey never ends, we would always seek to find the best resources to start learning these new skill sets.

Read Full Story