Data Engineer - Apache NiFi

Job Title : Data Engineer (Nifi) Location : Remote Job Type : 6 months contract (extendable) Experience Required : 4 years in developing and configuring workflows on Apache Nifi, including AWS-related connectors. 3 years in Apache Airflow, including writing jobs/DAGs/tasks using Python and experience with HTTP, S3, and Database operators. Experience with AWS Step Functions, Glue, and Athena is a must. Strong proficiency in Python. Experience with Apache Spark (good to have). Experience with PowerBI (a plus). Workflow optimization experience in Nifi, Glue jobs, and query optimization. Roles and Responsibilities : Apache Nifi Workflows : Develop, configure, and optimize workflows using Apache Nifi, focusing on AWS-related connectors. Apache Airflow DAGs : Write, configure, and manage Airflow DAGs/jobs using Python, including scheduling, monitoring, and alerting for various tasks. AWS Step Functions & Glue : Implement and optimize workflows using AWS Step Functions, Glue, and Athena to streamline data processing. Python Development : Write and maintain Python scripts for various data engineering tasks, ensuring code quality and performance. Data Processing Optimization : Optimize Nifi workflows, Glue jobs, and database queries to enhance performance and reduce costs. Spark and PowerBI (Optional) : Work with Apache Spark for large-scale data processing (preferred) and PowerBI for data visualization (a plus). Collaboration : Work closely with other engineers, data scientists, and stakeholders to ensure smooth project execution. Required Skills : Nifi : primary Alteryx : secondary skill Apache Nifi : Must have Apache airflow : good to have PowerBI : Must have Glue : good to have Step Functions : good to have Lambda : good to have Big Data : Must have (ref:hirist.tech)

Location: in, IN

Posted Date: 5/1/2025