Job Opportunity: Data Engineer – Recommender Systems & Big Data

Location: Tehran, Iran
Company: Ofogh Kourosh Chain Stores
Employment Type: Full-time

About Us

Ofogh Kourosh is the largest hard-discount retail chain in Iran with over 4,700 stores nationwide. We are transforming retail with cutting-edge analytics and AI. Our flagship initiative is building a large-scale recommender system powered by big data pipelines (Spark/Hadoop) to personalize shopping experiences for millions of customers.

Your Role

As a Data Engineer, you will be the backbone of our Big Data & AI team. You will design, implement, and maintain the data infrastructure and pipelines that fuel our recommender system—from raw transaction logs in HDFS to feature stores, Spark-based models (ALS, K-Means), and GPU-powered deep learning systems. You will work closely with data scientists and ML engineers to ensure scalable, reliable, and high-quality data flows.

Key Responsibilities

• Design, build, and maintain scalable ETL pipelines for billions of retail transactions and customer data.
• Manage and optimize a Hadoop/Spark cluster for large-scale data processing.
• Implement data ingestion workflows (e.g., Airflow + PySpark + SQL Server) to bring daily transactional data into the data lake (HDFS/Parquet).
• Build and maintain customer feature stores (RFM scores, frequency, recency, segmentation).
• Develop baseline models (ALS, K-Means) in Spark MLlib and prepare training datasets for deep learning models (PyTorch/TensorFlow on GPU).
• Collaborate with ML engineers to integrate deep learning models into the data pipeline.
• Ensure data quality, lineage, and governance across all stages.
• Automate workflows with Apache Airflow and monitor system health/performance.
• Support deployment of results into databases (SQL Server) and API layers (FastAPI).
• Experience with Docker/Kubernetes and Linux system administration.

Requirements

Must Have:

• Bachelor’s/Master’s degree in Computer Science, Software Engineering, or related fields.
• Strong hands-on experience with Apache Spark (PySpark, DataFrame API) and distributed data processing.
• Advanced SQL (SQL Server) – schema design, query optimization, large joins.
• Solid Python programming skills (pandas, pyarrow, ETL scripting).
• Familiarity with big data storage formats (Parquet, Delta Lake) and HDFS.
• Experience with workflow orchestration (Apache Airflow).
• Understanding of ETL, data modeling, and data warehouses.

Nice to Have (big plus):

• Exposure to recommender systems, personalization, or customer analytics.
• Familiarity with machine learning pipelines (MLlib, scikit-learn, or PyTorch/TensorFlow).
• Knowledge of GPU environments (CUDA, NVIDIA drivers) for deep learning.
• Previous work with retail or e-commerce data at scale.

What We Offer

• Be part of one of the largest recommender system projects in Iran’s retail industry.
• Work with cutting-edge big data & AI technologies (Spark, Hadoop, PyTorch, TensorFlow, Airflow).
• Access to powerful hardware infrastructure (multi-node CPU clusters + GPU servers).
• A dynamic, fast-growing team at the intersection of AI and retail.
• Competitive salary, benefits, and career growth opportunities.