Job Opportunity: Data Engineer – Recommender Systems & Big Data

Location: Tehran, Iran

Company: Ofogh Kourosh Chain Stores

Employment Type: Full-time

Your Role

As a Data Engineer, you will be the backbone of our Big Data & AI team. You will design, implement, and maintain the data infrastructure and pipelines that fuel our recommender system—from raw transaction logs in HDFS to feature stores, Spark-based models (ALS, K-Means), and GPU-powered deep learning systems. You will work closely with data scientists and ML engineers to ensure scalable, reliable, and high-quality data flows.

Key Responsibilities

Design, build, and maintain scalable ETL pipelines for billions of retail transactions and customer data.

Manage and optimize a Hadoop/Spark cluster for large-scale data processing.

Implement data ingestion workflows (e.g., Airflow + PySpark + SQL Server) to bring daily transactional data into the data lake (HDFS/Parquet).

Build and maintain customer feature stores (RFM scores, frequency, recency, segmentation).

Develop baseline models (ALS, K-Means) in Spark MLlib and prepare training datasets for deep learning models (PyTorch/TensorFlow on GPU).

Collaborate with ML engineers to integrate deep learning models into the data pipeline.

Ensure data quality, lineage, and governance across all stages.

Automate workflows with Apache Airflow and monitor system health/performance.

Support deployment of results into databases (SQL Server) and API layers (FastAPI).

Experience with Docker/Kubernetes and Linux system administration.

Requirements

Must Have:

Bachelor’s/Master’s degree in Computer Science, Software Engineering, or related fields.

Strong hands-on experience with Apache Spark (PySpark, DataFrame API) and distributed data processing.

Advanced SQL (SQL Server) – schema design, query optimization, large joins.

Solid Python programming skills (pandas, pyarrow, ETL scripting).

Familiarity with big data storage formats (Parquet, Delta Lake) and HDFS.

Experience with workflow orchestration (Apache Airflow).

Understanding of ETL, data modeling, and data warehouses.

Nice to Have (big plus):

Exposure to recommender systems, personalization, or customer analytics.

Familiarity with machine learning pipelines (MLlib, scikit-learn, or PyTorch/TensorFlow).

Knowledge of GPU environments (CUDA, NVIDIA drivers) for deep learning.

Previous work with retail or e-commerce data at scale.

What We Offer

Be part of one of the largest recommender system projects in Iran’s retail industry.

Work with cutting-edge big data & AI technologies (Spark, Hadoop, PyTorch, TensorFlow, Airflow).

Access to powerful hardware infrastructure (multi-node CPU clusters + GPU servers).

A dynamic, fast-growing team at the intersection of AI and retail.

Competitive salary, benefits, and career growth opportunities.