Title: Data Engineer- Pyspark
Experience: 0-2 Years
• Hands on experience in creating pipelines in Azure Data Factory using activities like Move &Transform, Copy, filter, for each, Get Metadata, Lookup, Data bricks etc
• Hands on experience working with different file formats like json, csv, Avro, parquet etc.. using Databricks and Data Factory
• In-depth knowledge of Spark Architecture including Spark Core, Spark SQL, Data Frames and Spark Streaming
• Extensive knowledge and Hands on experience implementing cloud datalakes like Azure DataLake Gen1/Azure DataLake Gen2
• Experience working with ETL process
Job Summary:
Data Engineer Data Mapping : The position is intended for Data Mapping of Insurance Business Systems Data (Policy/Claims/Billing) in Big Data/ (Azure) cloud environment to support the needs and objectives of the Organizational Units. The individual will perform mapping for migration of on-prem workloads to the cloud, and facilitate governance and streamlining operations of (Azure) cloud computing resources.
The role requires a self-starter that can be productive with minimal direction.
Key Responsibilities:
Specific KEY duties include but are not limited to:
• Drives “Cloud First strategy for all initiatives
• Analyzes complex legacy systems and come up with cloud migration strategies with full stack solution designs
• Collaborate with team to define overall data approach
• Perform data mapping and data profiling of Insurance data
• Confirm data readiness for feeds and conversion based on target system and business process requirements.
• Data Lake/ Data Vault with IBM Insurance data model experience on Azure with Pyspark coding
Other Responsibilities:
Reviews project requirements, evaluating industry-strength XaaS solutions and providing architectural recommendation based on best-fit and/or best-of-breed considerations
Works in onsite/offshore model. Person will work closely with offshore delivery team to drive quality and deliverable.
Education and Experience:
Bachelor’s degree (4-year) in Engineering or Computer Science; graduate degree is preferred
Minimum 10 years of experience in IT project delivery; must have at least 2 years hands-on Logical Data Mapping from Relational to Big Data/ Cloud
Nice to have (But not mandatory) Cloudera or AZURE Certification
Skills & Other Requirements:
Must have strong written and oral communications skills
Must have knowledge of IT project execution and delivery methodologies
Must have technical understanding of common applications, server software, and database technologies
Understanding of budget planning/development and cost allocation processes is desirable
USA.info@XDuce.com
+1 732-465-9100