Kanhaiya Pandey

Logo

Data Engineer
Location: India


Senior Data Engineer-Tech Lead

 

Dynamic Data Engineer professional with over 12 years of experience in Data Analysis and Business Intelligence(BI) solutions, including strong skills in Power BI,ADF, Azure, ETL, PySpark ,Data Modelling(Erwin), SQL and Agile. Worked on variety of Projects Implementation and development of ETL Data flow /Reports/Dashboard in numerous domains, including Banking, Finance, Healthcare and HR on Analytical Tools and Technology, spanning multiple phases from Feasibility Study to Go Live to Support. Significant contribution to a project using ADF and Power BI resulting in 30% reduction in analysis time. Good Exposure of handling Islamic Bank Projects. Worked in Kuwait for 2 years.

Microsoft Certified in AZ-900(Azure), DP-900(Azure Data), DA-100(Power BI)

 

EDUCATION

●   MBA | IT & Telecom| BIMM, Pune |AICTE| 2009-11 | Aggregate: 62.00%

●   B.Tech | Nagpur University |Nagpur | 2005-09 | Aggregate: 70.00%

●   H.S.C. | D.AV. Public School, Dhanbad |C.B.S.E.|2003-04 | Aggregate:77.50%

●   S.S.C. | S.S.V.M  |C.B.S.E.|2001-02 | Aggregate:79.00%

 

SKILLS SUMMARY

●   Languages :        SQL, Python,Pyspark, R , DAX, Shell ,BIAL

●  Frameworks:        Pandas, NumPy, Matplotlib

●   Azure Stack:        Azure Data Factory, Databricks ,Azure Data Lake, Azure SQL DB, Logic App, Microservices

●   Tools:                   Power BI, Tableau, Erwin, MSBI, AWS Athena, Air flow,  SAP BO / BOBJ, BODS, SAP HANA Studio

●   Skills:                     ETL Data pipelines, BI Reporting , PySpark, Data Modelling, SQL DB, Azure Cloud

●  Platforms:            PyCharm, Jupyter Notebook, Visual Studio Code, Rest API, Postman, Swagger

●  Soft  Skills:          Team Management, Stakeholder management,  People Management

 

WORK EXPERIENCE

Senior Data Engineer – Tech Lead                                                                                                                 Pune, Maharashtra
Globant India (Client- Ernst and Young, US)                                                                                              Dec 20 – till date

○  Designed and implemented complex ETL data pipelines using Databricks and Azure Data Factory to efficiently ingest and transform data from multiple Rest APIs and ingest to Azure SQL DB and improved transactional data processing time by 25%

○  Analyzed, Designed and Built modern data solutions using Azure API service to trigger ADF from Frontend UI and reduced cost by 15%

○  Designed the data pipeline architecture for the cyber security product that quickly scaled up from one region tenants to multi regions for multi tenants.

○  Managed Azure Infra right from CI/CD pipeline, Code Repo-creation, automate tasks using Shell scripting, Devops release pipeline and deploy codes to multi prod environment from singe source of truth, Deployment automation with YAML file.

○  Led a team of Data Engineers in refactoring the application architecture and implementing Job scheduling using APIs, utilizing Agile methodology

○  Successfully designed ETL data pipeline to automate resource provisioning and identifying fraud email and deploying the solutions in 6 regions while optimizing efficiency by 30%.

○  Used Azure AI service Form Recognizer as an automated data processing system that uses AI and OCR to quickly extract text and structure from documents.

○  Getting the requirements from Business users, analyzing the data and providing estimates to business stakeholder.

○  Build complex pipelines to bring the data from third party API to ADF then automate the ADF to onboard the new client, provision the Azure Cloud infrastructure to ingest Data into respective Provisioned Database Tables

○  Used various sources to pull data into Power BI such as SQL Server, SAP BW, SQL Azure and streaming data.

○  Streamlined data collection and reporting procedures, reducing processing time by 20% enhancing efficiency.

○  Responsible for designing, creating, and maintaining logical and physical data models using the Erwin data modeling software, ensuring data consistency ,quality, data governance standard.

○  Created 15+ compelling power BI reports that provided actionable insight for key stakeholders.

 

 

 

 

 

 

Data Engineer                                                                                                                                                 Kuwait city, Kuwait
KASP (Client – AUB Bank and Central Bank of Kuwait, Kuwait)                                                               Sep 2019 – Nov 2020

○  Built and managed Data Pipelines and Jobs in Azure Data Factory to streamline data processing that helped user to improve the efficiency by 20%

○  Automated data processing and improved efficiency using Data Pipelines (DLT) API and Web app service, enhancing overall data processing accuracy.

○  Managed data integrity by handling missing values and encoding categorical variables, enhancing quality by 25%.

○  Involved in CI/CD pipelines in Azure DevOps to automate development implementation with Data bricks and Azure Data Factory, enabling faster and more reliable deployment of data pipelines.

○  Developed Settlement and Reconciliation Power BI Dashboard with review and reconcile third-party transaction details along with Problematic Transaction and Provide detailed information about them.

○  Good understanding of Retail Banking and Corporate Banking Domain (Especially Retail, Deposit, Card& their different products).

○  Developed SQL queries/Scripts to validate the Data such as Checking duplicates, Null Value to ensure correct data aggregation

○  Proficiency in Native HANA views development (Especially HANA Artifacts Calculation views, Decision Table and Analytical Privilege and used Data provisioning Technique like SDA,SMQI,SLT for Data Replication)

○  Used k- means clustering, Key influences, decomposition tree to understand root cause and adhoc analysis.

 

 

BI Consultant                                                                                                                                                       Pune, Maharashtra
Saama Technology India Pvt. Ltd (Delta Dental Health Insurance, US)                                                Jun 2017 – Aug 2019

○  Worked on Complex HR Dashboard including Personal Information, Leave Status along with Responsibility.

○  Explored data in a variety of ways and across multiple visualizations using Power BI. Strategic Expertise in design of experiments, data collection, analysis and Visualization.

○  Designed Power BI Data Viz using utilizing Tree Map, Heat Map, Scatter Plot & Bubble Chart, Pie and Donut.

○  Developed SQL Queries using Stored Procedures, CTEs to support Power BI and SSRS Reports.

○  Worked with Microsoft team on Product Limitations and communicate limitations and Tool Capabilities with Snowflake Cloud Data warehouse using import and DQ mode.

○  Worked on Lifecycle management tool and promotion management tool to export the reports from one server to other like DEV-Quality-Prod.

○  Worked on Data Modeling, Data Provisioning, and Reporting aspects of SAP HANA.

○  Created Free hand SQL and Custom WebI Report in order to improve report performance.

○  Created visualizations using Slicers, Pies, Histograms, Maps, Scatter, Bullets, Heat Maps, Tree maps.

○  Worked on report and universe retrofitting/remediation, investigate report Failure and implement fixes.

○  Experienced in DAX language in row context /filter context, variables, and performance techniques.

○  Implemented various data security profile at universe level for data segregation at Business

 

 

Senior Tech Consultant                                                                                                                                     Noida, Delhi NCR
Fujitsu India Pvt. Ltd                                                                                                                                       Jul 2016 – May 2017

○  Actively played Role in Root Cause Analysis (RCA) and fixing many production related issues raised by users by Service Now incident management Tool.

○  Have implemented BO Authorization on Dashboard report without having any Universe as Semantic Layer.

○  Requirement gathering from stakeholder & analysis, preparation of universe designing documents.

○  Experience in HANA Data Models including Attribute Views, Analytical Views, Calculation Views, basic SQL Scripting, and Analytical Privileges.

○  Scheduling and Load Monitoring of Daily, weekly, and Monthly, Data Loads using Process Chains.

○  Report designing, test cases preparation, report functional and technical design document preparation.

○  Worked on creation of complex reports using multiple data providers, Drill Features and Hyperlinks, Alerts, Prompts etc.

○  Worked on Lifecycle management tool and promotion management tool to export the reports from one server to other like DEV-Quality-Prod.

 

 

 

 

Reporting Data Analyst                                                                                                                                  Pune, Maharashtra
Barclays, Bank (B.T.C.I.)                                                                                                                                 Jan 2015 – June 2016

○  Designing of schema, universe designing , creating connection, handling loops and traps, context defining,  creation of condition/filter at universe level, defining hierarchy, Creation of Prompts, Row/Object level security implementation, Performance tuning at universe level, self-testing of universe, migration to DEV to UAT and Production. Designing of WebI reports, Xcelsius Dashboard connection with universe and creation of report level filter/parameters.

○  Requirement gathering from stakeholder & analysis, preparation of universe designing documents.

○  Report designing, Regression testing in WebI, test cases preparation, report functional and technical design document preparation.

○  Performance tuning at universe level, export to repository, self-testing of universe, migration to DEV to UAT and Production.

○  Worked on the CMC Module in creating user groups /folders/ managing Security like Row level security.

○  Requirement gathering from stakeholder & analysis, preparation of universe designing documents.

○  Experience in HANA Data Models including Attribute Views, Analytical Views, Calculation Views, basic SQL Scripting, and Analytical Privileges.

○  Scheduling and Load Monitoring of Daily, weekly, and Monthly, Data Loads using Process Chains.

○  Report designing, test cases preparation, report functional and technical design document preparation.

○  Worked on creation of complex reports using multiple data providers, Drill Features and Hyperlinks, Alerts, Prompts etc.

○  Worked on Lifecycle management tool and promotion management tool to export the reports from one server to other like DEV-Quality-Prod.

 

 

 

 

Software Engineer                                                                                                                                       Pune, Maharashtra
Ramboll India Pvt. Ltd                                                                                                                                Feb 2012- Dec 2014

○  Retail Sales Vertical in telecom industries, providing support to the AVAYA. Receiving client request, Incident, and Problem of Client every day via ticket raised by the business user. Based on priority resolving the tickets issue.

○  Working with Ticketing tool, Resolving Tickets on daily basis, based on SLA. High, Low, Incident. Tickets trouble shooting and resolving.

○  Universe enhancement and designing of new universes as per requirement. Designing the WebI enhancement and modification in the report. Migrating the reports to Dev to QA and Production Environment.

○  Worked on the CMC Module in creating user groups and folders and managing Security like Row level security and performed many troubleshooting related to report access.

○  Team Handling, Working with CA Ticketing tool, Resolving Tickets on daily basis, based on SLA P1 to P4 Tickets trouble shooting and resolving, interacting with business audience/user, and resolving the tickets related to business object to WebI Reports error, Xcelcious maintenance and support.

○  Work closely with development, SQL and SAP BW teams and ensure end to end support is provided within SLAs/OLAs.

○  Responsible for the production support, maintenance, and configuration

 

 

 

 

 

CERTIFICATES

Microsoft Certified: Power BI Data Analyst Associate | DA-100                                                                              Mar 2022
○  Provide meaningful business value through easy-to-comprehend data visualizations.

○  Acquired expertise in Power BI report creation and automate them.

Microsoft Certified: Azure Fundamentals | AZ-900                                                                                                   Jun 2023
○  Learned about compute, networking and storage in DB.

○  Helped to understand all basic fundamentals of Azure.

Microsoft Certified: Azure Data Fundamentals | DP-900                                                                                         Jul 2023
○  Learned the fundamentals of database concepts in a cloud environment.

○  Acquired knowledge of Azure storage for non-relational data and the fundamentals of Azure Cosmos DB.

  • IT/Technology
  • Updated 1 year ago

Only Employers can view Contact Details. Request Access