Data Engineer -2

Role: Data Engineer - 2 Responsibilities: • Designing, developing, and maintaining data architecture, data pipelines, and data workflows. • Building and optimizing large-scale data processing systems for both structured and unstructured data. • Collecting, integrating, and transforming data from various sources, including databases, APIs, and file systems. • Collaborating with data scientists, analysts, and other stakeholders to understand data requirements and design effective data solutions. • Developing and implementing data quality and data governance processes to ensure data accuracy, consistency, and reliability. • Writing efficient and scalable code/scripts to automate data processing tasks and optimize data workflows. • Monitoring and troubleshooting data pipelines to ensure their performance, availability, and reliability. • Identifying and implementing improvements to enhance data infrastructure, including data storage, processing, and retrieval mechanisms. • Keeping up to date with the latest technologies, tools, and trends in the field of data engineering. • Mentoring and providing guidance to junior data engineers, as needed. Qualifications: • Bachelor's degree in computer science, data science, or a related field. Advanced degree preferred. • Proven experience as a data engineer or in a similar role, handling large-scale data processing and analytics projects. • Strong programming skills in languages like Python, Java, or Scala, with experience in data manipulation, scripting, and automation. • Proficiency in working with relational databases (SQL) and NoSQL databases. • Experience with big data technologies and frameworks, such as Hadoop, Spark, Kafka, or Hive. • Knowledge of data integration and ETL (Extract, Transform, Load) processes. • Familiarity with cloud platforms, such as AWS or GCP, and their data-related services. • Understanding of data modelling concepts and data warehousing= principles. • Strong problem-solving and analytical skills, with the ability to troubleshoot and optimize data processing workflows. • Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.