Data Engineer PySpark
__jobinformationwidget.freetext.LocationText__
Noida, Uttar Pradesh, India
- Sopra Steria
- Engineering, Development, Applications
- 3 to 5 years
- Standard
- Noida, Uttar Pradesh
This vacancy has now expired. See all jobs
About Sopra Steria
Sopra Steria, major Tech player in Europe recognised for its consulting, digital services and software development, helps its clients drive their digital transformation and obtain tangible and sustainable benefits. It provides end-to-end solutions to make large companies and organisations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a fully collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. With 50,000 employees in nearly 30 countries, the Group generated revenue of €5.1 billion in 2022.
The world is how we shape it.
We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will collaborate closely with our Data Scientists to develop and deploy machine learning models. Proficiency in below listed skills will be crucial in building and maintaining pipelines for training and inference datasets.
Responsibilities:
• Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines.
• Utilize PySpark for data processing, transformation, and preparation for model training.
• Leverage AWS EMR and S3 for scalable and efficient data storage and processing.
• Implement and manage ETL workflows using Streamsets for data ingestion and transformation.
• Design and construct pipelines to deliver high-quality training and inference datasets.
• Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities.
• Optimize and fine-tune pipelines for performance, scalability, and reliability.
• Ensure IAM policies and permissions are appropriately configured for secure data access and management.
• Implement Spark architecture and optimize Spark jobs for scalable data processing.
Total Experience Expected: 04-06 years
BE
Requirements:
Mandatory
• Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop.
• Proven expertise in designing and deploying data pipelines.
• Strong problem-solving skills and ability to work effectively in a collaborative team environment.
• Excellent communication skills and ability to translate technical concepts to non-technical stakeholder
Desirable
• Hands-on experience with Airflow, S3, and Stream sets or similar ETL tools. [ can be trained locally ]
• Understanding of real-time or near real-time inferencing architectures.
- •Basic Knowledge on Kafka ,AWS IAM, AWS EMR and Snowflake.
At our organization, we are committed to fighting against all forms of discrimination. We foster a work environment that is inclusive and respectful of all differences.
All of our positions are open to people with disabilities.
Discover what working at Sopra Steria looks like...
Are you looking for a place where you can free your creativity and take initiatives, supported by tech experts?
Join us on this adventure where every idea counts and every talent steps up.
Job offers that might interest you
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Description: Ideal candidate must be self-motivated with a proven track record in working on ServiceNow Platform Having an experience in ServiceNow Development from medium to complex needs. Able to Un
Reference
655f37c4-1a6f-4c4a-9f2e-7b57f2e12c80
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Working experience on ABAP (RICEF) Objects with at least 1 Implementation project experience in SAP Proficient in OOABAP, IDOC, ODATA and ABAP new syntax. Good knowledge of Module Pool programming, Sm
Reference
c27d2815-f2d1-4c80-8166-648953c3779a
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Job description 6-8 years of work Experience on different EDI Integration plat forms like IBM Sterling Integrator, Axway B2bi, Open Text, Seeburger etc. Strong Mapping Experience using various formats
Reference
14db983a-29f9-44ab-9981-827b33858166
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Must-Have: Strong experience in Angular 10+ development Good to have knowledge of Vue.js Experience with NgRx for state management Proficient in SCSS and XSLT development with live server and hot relo
Reference
ce70c84e-4f35-4807-acc7-929a8f61b8b6
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
We’re seeking a skilled Support Analyst with hands-on experience in the Microsoft stack (.NET, AngularJS, Webforms, WCS), SOAP services, SQL, Jenkins, Camunda, PowerShell, and YAML. This role focuse
Reference
5a340d2f-170c-4601-b97d-3ea6bef5d83a
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Experience: 4yr to 6yr Work Location: Noida / Bangalore Mode of work: Hybrid (3 days office every week) Profound ABAP development experience and good in Performance Tuning Knowledge on all RI
Reference
48ff99e2-2b1a-4e1a-a3b8-d5abb5e83309
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Experience: 4 to 6 years Work Location: Noida / Bangalore Mode of work: Hybrid (3 days office every week) Strong working experience on ABAP (RICEF) Objects on SAP S4 HANA system Proficient in OOABAP
Reference
02293f95-6e5c-457b-8d1a-873143b409b2
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
Role: Google Cloud Engineer Skillset: GCP, Kubernates, Docker, CI/CD Experience: 4-6 years Location: Noida and Chennai Required Skillset: 4-6 years of experience in DevOps, with a focus on GCP. Strong
Reference
a2105003-48f1-4491-822d-4ffee9e28d96
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
• Order Management, Procurement, I-Procurement, Inventory • Expert Knowledge on Business Processes 4-6 Plus years’ of experience with Oracle Application, at least 4 + years of experience in Oracle E
Reference
cb69143a-3282-4103-b894-bf3acf355f17
Expiry Date
Jan 1, 0001
Salary
Location
Noida, Uttar Pradesh, India
Vacancy language
English
Job Type
Standard
Experience Level
3 to 5 years
Department
Engineering, Development, Applications
Brand
Sopra Steria
Sector
All Sectors
Remote
No
Location
Noida, Uttar Pradesh
Description
SAP MM Functional Consultant: Expert in design/implementation/consulting with SAP Material Management. SAP MM certification is plus At least 3 End to End Implementation in SAP MM Module (ECC) – Prefer
Reference
f97076d3-fb60-415c-a317-3fb2e1593cb3
Expiry Date
Jan 1, 0001