Sr. Data Engineer

Company: The Judge Group
Location: Newark
Posted on: January 12, 2021

Job Description:

Location: Newark, CADescription: Our client is currently seeking a Sr. Data Engineer Data Engineer and BI : IOT Tools - MQTT, Kafka, Apache Spark and data frameLanguages: Java, PythonTechnical/Functional Skills -MUST HAVE SKILLS --- 6+ years of experience in Data Engineering and Business Intelligence. --- Proficient in IoT tools such as MQTT, Kafka, Spark --- Proficient with AWS, S3, Redshift --- Experience with Presto and Parquet/ORC --- Proficient with Apache Spark and data frame. --- Experienced in containerization, including Docker and Kubernetes --- Expert in tools such as Apache Spark, Apache Airflow, Presto --- Expert in design and implement reliable, scalable, and performant distributed systems and data pipelines --- Extensive programming and software engineering experience, especially in Java, Python, --- Experience with Columnar database such as Redshift, Vertica Roles & Responsibilities --- Hands-on design and develop streaming and IoT data pipelines. --- Developing streaming pipeline using MQTT, Kafka, Spark Structure Streaming --- Orchestrate and monitor pipelines using Prometheus and Kubernetes --- Deploy and maintain streaming jobs in CI/CD and relevant tools. --- Python scripting for automation and application development --- Design and implement Apache Airflow and other dependency enforcement and scheduling tools. --- Hands-on data modeling and data warehousing --- Deploy solution using AWS, S3, Redshift and Docker/Kubernetes --- Develop storage and retrieval system using Presto and Parquet/ORC Contact: rragab@judge.comThis job and many more are available through The Judge Group. Find us on the web at www.judge.com

Keywords: The Judge Group, Newark , Sr. Data Engineer, Other , Newark, California