Overview
We are seeking a highly skilled Data Engineer to design, implement, and maintain robust data pipelines and scalable data storage solutions in a
dynamic, MLOps-friendly environment. The ideal candidate will have a minimum of 2–3 years of industry experience, a relevant bachelor’s degree,
and strong expertise in programming and data infrastructure. This role is essential for supporting our enterprise’s mission to leverage real-time
analytics and advanced AI integrations for improved decision-making.
Key Responsibilities
Architecture & Pipeline Development
Experiment Tracking & Service Maintenance
Advanced Analytics & Distributed Processing
Cloud & Storage Technologies
Vector Databases & AI Integration
Software Development & DevOps
System Administration & Linux Proficiency
Collaboration & Mentorship
Core Technical Skills:
Programming:
Proficiency in Python and Java for data processing and application development.
Database Management:
Strong experience with SQL databases; familiarity with NoSQL solutions.
Ability to administer and maintain databases and data services.
Experiment Tracking & MLOps Tools:
Hands-on experience with MLflow (or equivalent) for experiment tracking, model registry, and deployment.
Distributed Computing:
Experience with Apache Spark or similar frameworks (e.g., Apache Flink) for scalable data processing.
Cloud & Storage:
Experience with AWS S3 and knowledge of open-source S3 alternatives such as MinIO.
Operating Systems:
Advanced proficiency with Linux, including system administration and troubleshooting.
DevOps & CI/CD:
Familiarity with CI/CD practices and tools (e.g., Jenkins, GitLab CI/CD).
MLOps Integration:
Understanding of AI/ML concepts and integration of machine learning models into production data pipelines.
Agile Methodologies:
Experience working in Agile environments with effective version control practices.
Soft Skills
Communication & Collaboration:
Excellent written and verbal communication skills to clearly articulate technical concepts.
Proven ability to work collaboratively in cross-functional teams.
Problem-Solving & Strategic Thinking:
Strong analytical skills with a keen eye for designing scalable and sustainable solutions.
Adaptability & Continuous Learning:
Commitment to staying updated on emerging trends and technologies in data engineering and MLOps.
Plus Points
Experience with containerization (e.g., Docker) and orchestration (e.g., Kubernetes).
Familiarity with Infrastructure as Code (IaC) tools like Terraform.
Contributions to open-source projects or personal projects in data engineering and MLOps.
Additional experience with big data technologies and distributed processing frameworks beyond Apache Spark.
Work Environment and Benefits
Flexible Work Options:
Opportunities for remote or on-site work based on company policy.
Career Growth:
Dynamic, international environment with ample opportunities for professional development.