Principal Data Engineer
Caliberly
- Dubai
- Permanent
- Full-time
- Data Pipeline Development: Building and optimizing data pipelines for ingesting, processing, and transforming large volumes of data from various sources into usable formats for analysis and reporting.
- Data Integration: Integrating disparate data sources and systems to ensure data consistency, accuracy, and reliability across the organization.
- Performance Optimization: Optimizing data processing and query performance to ensure timely access to data for analytics and reporting purposes.
- Data Governance and Security: Implementing data governance policies and procedures to ensure data quality, integrity, and security. This includes defining data access controls, encryption, and compliance with regulatory requirements.
- Team Leadership: Providing technical leadership and guidance to a team of data engineers. This may involve mentoring junior team members, conducting code reviews, and driving best practices in data engineering.
- Collaboration: Collaborating with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver solutions that meet business objectives.
- Data Modeling and Database Skills: Expertise in relational and NoSQL databases, as well as data modeling techniques. Experience with database technologies like PostgreSQL, MySQL, MongoDB, or Cassandra is often required.
- Big Data Technologies: Knowledge of big data technologies and platforms, such as Apache Hadoop ecosystem (HDFS, MapReduce, Hive, HBase), Apache Spark, and cloud-based data platforms like AWS, Azure, or Google Cloud Platform.
- ETL and Data Warehousing: Experience with Extract, Transform, Load (ETL) processes and tools, as well as data warehousing concepts and methodologies. Familiarity with tools like Apache Airflow, Talend, or Informatica is beneficial.
- Problem-Solving Skills: Strong analytical and problem-solving skills, with the ability to troubleshoot complex data issues and devise effective solutions.
- Communication and Leadership: Excellent communication and interpersonal skills, with the ability to communicate technical concepts to non-technical stakeholders. Experience leading and mentoring a team of data engineers is often required.