Job description
Job Purpose
This position is responsible to gather and analyze data to solve and address highly complex business problems and evaluate scenarios to make predictions on future outcomes and support decision making. Also responsible to apply ML/AI technologies across the Portfolio to drive greater business value and competitive differentiation to achieve business goals
Key Responsibilities
Gather and process raw data at scale (including writing scripts, web scraping, calling APIs write SQL queries, etc.)
Design, implement and maintain high performance big data infrastructure /systems, & big data processing pipelines scaling to structured and unstructured business requirements
Create web services / API based interfaces to modularize capabilities across lending products
Developing prototypes and proof of concepts for the selected solutions, and implementing complex big data projects
Drive data investigations across organizations and deliver resolution of technical, procedural, and/or operational issues to completion
Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)
Adapting and learning new technologies surrounding Big Data ecosystems and ensure learning and growth of team members
Enhance/optimize data migration procedures to include data that is relevant for building analytic systems
Work with business domain experts, data scientists and application developers to identify data relevant for analysis and develop the Big Data solution.
Job Requirements
Qualifications
Masters / Graduate in Computer science, Engineering (CS, ECE, IT etc.), Mathematics, Computational physics
Experience
4+ years in data engineering and warehousing roles
Functional Competencies
Big data tech: Hbase, MapReduce, Flume, Sqoop, Oozie, Zeppelin etc
Data pipelines with Kafka streams, RESTful APIs with SpringBoot, API gateways etc
Heterogeneous data environments: Cloud (AWS, Azure), Relational, NoSQL, GraphDBs in production
SQL and scripting languages like Python, Java / JS, Perl etc
S/w development practices: Agile, devOps CI/CD, automated testing / deployments, WebServices
Behavioral Competencies
Hands-on contributor, passionate about applying tech in businesses
Strong Analytical & Interpersonal skill
Eye for detail
