Lead Big Data Engineer (Hadoop)

Comcast - New York, NY

- Design and build data processing pipelines using tools and frameworks in the Hadoop ecosystem
- Design and build ETL pipelines to automate ingestion of structured and unstructured data
- Design and Build pipelines to facilitate data analysis
- Implement and configure big data technologies as well as tune processes for performance at scale
- Manage, mentor, and grow a team of big data engineers

5+ years of work experience of relevant work experience
- 3+ years of experience working with big data technologies
- Proficiency in a programming language, ideally Python, Java, or Scala
- Proficiency and knowledge of best practices with the Hadoop (YARN, HDFS, MapReduce)
- Experience and knowledge of best practices with big data interactive query technologies like Spark, Impala, or Hive
- Leadership experience with small and/or mid-size software development teams
- Excellent written and oral communication skills

 

Preferred Qualifications
- Experience sprint planning
- Experience leading code reviews
- Proficiency and knowledege of best practices in Spark
- Experience with a workflow management framework (Luigi, Oozie, Azkaban, etc)
- Experience leading in an agile environment
- Experience with version control (Git preferred)
- Experience with Jenkins

---

Posted: 30+ days ago

Apply Now