We build Visual Spark, enhancing Apache Spark to build the simplest and most powerful machine learning platform.
All our engineers work on the entire product. You'll be driving our execution service, written in Play/Scala that spins up and manages Spark clusters, generates Spark code and gets high performance for large scale jobs and latency sensitive interactive development. You'll manage how clusters are spun up on AWs using Kubernetes, docker.
- Should have developed an application in Spark - Machine Learning or ETL - using SparkML, MLLib, Spark SQL with substantial data sizes
- Should have good knowledge in developing Web Services preferably using Scala and Play Framework
- Should have good knowledge of deployment and containers - AWS, Kubernetes, Docker, security on AWS
- Should have good understanding of compilers and interpreters