Senior Data Engineer

FlatGigs is pleased to announce an excellent career opportunity with one of our distinguished clients. We are seeking a highly experienced Senior Data Engineer to join a fast-paced and innovative team. This role offers the chance to design and implement cutting-edge data solutions that support advanced analytics and machine learning initiatives.

Role Overview

The Senior Data Engineer will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure. The role involves collaboration with cross-functional teams, including data scientists, analysts, and software engineers, to ensure efficient and innovative data-driven solutions.

Responsibilities

1. Data Architecture and Engineering

• Develop and maintain scalable data pipelines and infrastructure for optimal data extraction, transformation, and storage.
• Evaluate and select appropriate technologies and tools for data processing.

2. Data Integration and ETL

• Design and implement efficient ETL processes to integrate data from multiple sources.
• Ensure data quality through validation, cleansing, and transformation.

3. Big Data Technologies

• Utilize big data tools such as Hadoop, Spark, and Kafka for processing and analysis.
• Optimize workflows for performance and cost-efficiency.

4. Data Modeling and Warehousing

• Design and implement data models and data warehousing solutions to meet operational and analytical requirements.
• Manage data lakes and warehouses, ensuring security and compliance.

5. Collaboration and Leadership

• Work closely with data scientists and engineers to deliver scalable data solutions.
• Mentor junior team members and promote best practices.

6. Monitoring and Optimization

• Monitor data pipelines and infrastructure to identify and resolve issues.
• Continuously improve performance and efficiency of data processing and storage solutions.

7. Documentation and Compliance

• Maintain comprehensive documentation of data workflows and architecture.
• Ensure adherence to data privacy regulations and organizational policies.