Data Engineer PySpark
__jobinformationwidget.freetext.LocationText__
Chennai, Tamil Nadu, India
- Sopra Steria
- Engineering, Development, Applications
- 6 to 10 years
- Standard
- India
- All Sectors
This vacancy has now expired. See all jobs
About Sopra Steria
Sopra Steria, a major Tech player in Europe with 56,000 employees in nearly 30 countries, is recognized for its consulting, digital services and software development. It helps its clients drive their digital transformation and obtain tangible and sustainable benefits. The Group provides end-to-end solutions to make large companies and organizations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a fully collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. In 2023, the Group generated revenues of €5.8 billion.
The world is how we shape it.
We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will collaborate closely with our Data Scientists to develop and deploy machine learning models. Proficiency in below listed skills will be crucial in building and maintaining pipelines for training and inference datasets.
Responsibilities:
• Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines.
• Utilize PySpark for data processing, transformation, and preparation for model training.
• Leverage AWS EMR and S3 for scalable and efficient data storage and processing.
• Implement and manage ETL workflows using Stream sets for data ingestion and transformation.
• Design and construct pipelines to deliver high-quality training and inference datasets.
• Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities.
• Optimize and fine-tune pipelines for performance, scalability, and reliability.
• Ensure IAM policies and permissions are appropriately configured for secure data access and management.
• Implement Spark architecture and optimize Spark jobs for scalable data processing.
Requirements:
Mandatory
• Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop.
• Proven expertise in designing and deploying data pipelines.
• Strong problem-solving skills and ability to work effectively in a collaborative team environment.
• Excellent communication skills and ability to translate technical concepts to non-technical stakeholder
Desirable
• Hands-on experience with Airflow, S3, and Stream sets or similar ETL tools. [ can be trained locally ]
• Understanding of real-time or near real-time inferencing architectures.
- •Basic Knowledge on Kafka ,AWS IAM, AWS EMR and Snowflake.
Total Experience Expected: 06-08 years
BE
At our organization, we are committed to fighting against all forms of discrimination. We foster a work environment that is inclusive and respectful of all differences.
All of our positions are open to people with disabilities.
Discover what working at Sopra Steria looks like...
Are you looking for a place where you can free your creativity and take initiatives, supported by tech experts?
Join us on this adventure where every idea counts and every talent steps up.
Job offers that might interest you
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
Total Experience Expected: 06-08 years Work Location: Noida / Bangalore Mode of work: Hybrid (3 days at office every week) Required Skills: Strong expertise in SAP ABAP development, including ABA
Reference
b09908dc-1ff7-42bf-93e3-63f2b6f001dc
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
We are seeking a highly skilled 8-10 yrs Experienced SAP MM Public Cloud Consultant to join our team. In this role, you will be responsible for implementing, configuring, and supporting SAP Materials
Reference
fa2d9715-a6b1-4831-beb3-34a087922eff
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
We are seeking a highly skilled 8-10 yrs Experienced SAP EWM Public Cloud Consultant to join our team. In this role, you will be responsible for implementing, configuring, and supporting SAP Extended
Reference
f9e3445e-c240-423a-89e4-e620770d6c1d
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
Bachelor’s degree in computer science, Computer Engineering, IT, Business Computer or related fields 5+ years of experience in SAP Production Planning module and worked on S/4 HANA Data Migration proj
Reference
96825d2c-3e4e-43da-8a04-248d764daf6a
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
Bachelor’s degree in computer science, Computer Engineering, IT, Business Computer or related fields 5+ years of experience in SAP Finance and Controlling on S/4 HANA Data Migration projects or ETL ba
Reference
7c9ddbc2-671d-4bf1-9e82-3a441a82d0d1
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
Job Description Role & Responsibilities: Team member should have Minimum 8-10 Yrs experience in SD in S/4HANA, Must have completed 1-2 S4H Greenfield Implementation. Participates in requirements revie
Reference
dc2f8cb3-0b43-449b-a1e2-06ec38a920f3
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
· Minimum of 8-10 years of SD experience in full cycle implementation as well as in support projects. · Minimum 2-3 E2E SD Implementation experience. · Knowledge of working experience o
Reference
1c1de706-07b1-4b69-b691-838f350f65d7
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
Job Description: Work Location : Chennai / Noida SAP EWM Functional experience between 8 and 10 years. Minimum 1 E2E implementation & 2 support projects. Work self-directed and independently. Able to
Reference
d5aef09a-9fad-44d7-bca5-b1dd1bcb6f08
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
Primary Skill: SAP Functional GTS Job Description: 1) SAP GTS consultant(6-10 years of experience) must act as E2E responsible for all GTS Topics (Compliance, Customs, Trade Preference etc.) to prov
Reference
4f23f6e0-f2a7-44a8-a78c-7ee7f22be2b7
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Job Type
Standard
Experience Level
6 to 10 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Location
Noida, Uttar Pradesh
Description
We are seeking a highly skilled SAP SD Public Cloud Consultant with 8-10 yrs of experience to join our growing team. In this role, you will be responsible for implementing, configuring, and supporting
Reference
c2bca834-41ee-4fd5-9811-f14f7b385f30
Expiry Date
Jan 1, 0001