Experience:
•2 years’ full-time experience in Data engineering and AWS Cloud Technologies
•Mandatory experience in Mid to Expert Level programming capabilities in a large-scale enterprise
•Experience with Distributed Versioning Control environments such as GIT
•Familiarity with development tools - experience on either IntelliJ / Eclipse / VSCode IDE, Build Tool Maven
* Experience on AWS cloud including DMS, Glue , Pyspark
Responsibilities
* Responsible to Ingest data from files, streams and databases. Process the data with Hadoop, Scala, SQL Database, Spark, ML, IoT
* Develop programs in Python & Pyspark and AQL as part of data cleaning and processing
* Responsible to design and develop distributed, high volume, high velocity multi-threaded Ingestion pipeline
* • Provide high operational excellence guaranteeing high availability and platform stability
* Implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Pyspark, Glue, and AWS Cloud computing etc.
* Anticipate and resolve complex problems that impact data design
* Assist in critical technical and design decisions during the development process
Key Performance indicators
• Ensure Data Pipeline is robust and is monitored with respect to schemas, distributions and data completeness
• Evaluate the time required for data to be available for analysis to baseline by monitoring the data velocity against baseline
• Maintain and Monitor the Query to report conversion rate
• Maintain and monitor the optimal time needed for data to be reported in form of KPIs
• Identify issues in codes or bugs that have entered the production stage
• Drive performance optimization by developing optimized codes (Pipelines) for efficient resource utilization
Qualifications:
•Bachelor's degree in Computer Science, Software Engineering or a related technical discipline
•Knowledge of any of the programming languages: Python, Scala or Java
•Primary Skill: AWS , AWS Glue , AWS DMS , Cloudwatch , Pyspark , SQL
•Domain Knowledge: Life Insurance domain is preferred . Familiarity with Finance data and subject is highly preferred
Interested? Upload the detail at: https://forms.gle/7BHCKLnBHEJqHb2D6 or share your cv at hr@mvss.online
