Senior Scala Developer in Toronto at APEX Systems

Date Posted: 5/23/2018

Job Snapshot

Job Description

Job #:  851781

please apply to rsafiyari@apexsystems.com

Job Description

 

We are looking for an experienced Big Data Engineer that will work on the collection, storage, processing, and analysis of large data sets as part of a new  initiative at Securities-–the design and implementation of the Market Risk reporting platform. The successful candidate will be responsible for the design and development of a software solution that captures data from a variety of sources and makes it available to multiple stakeholders ranging from other IT applications to business teams. This is a unique opportunity to work on cutting-edge technologies to solve problems of scale and speed.

 

Responsibilities

  • Provide leadership on the development and maintenance of robust batch and real-time data processing solutions using Spark and Hadoop
  • Implement ETL processes to ingest large data sets from multiple data sources
  • Monitor performance and advise any necessary application and/or infrastructure changes
  • Leverage best practices in continuous integration and delivery
  • Help drive transformation by continuously looking for ways to automate existing processes, test, and optimize data quality
  • Explore new capabilities and technologies to drive innovation
  • Mentor team members on the use of software development best practices and agile methodologies

Skills and Qualifications

 

Must Have

  • Proficient understanding of distributed computing principles
  • Advanced knowledge and hands on experience building solutions using the Spark ecosystem (e.g., Spark Core, Spark SQL, Spark Streaming)
  • Practical experience building solutions leveraging the Hadoop ecosystem (e.g., HDFS, YARN, MapReduce, Oozie)
  • Practical experience implementing Big Data ingestion solutions from multiple data sources using tools such as Flume, Sqoop, and NiFi  
  • Good understanding of relevant architectures (e.g., Lambda Architecture, Kappa Architecture, etc.) along with their advantages and drawbacks