Description

The DP-203 Data Engineering on Microsoft Azure certification training course from CloudThat offers candidates proper training and relevant study material to prepare and successfully clear the DP-203 exam.

This exam measures candidate’s ability to accomplish the following technical tasks: design and implement data storage; design and develop data processing; design and implement data security; and monitor and optimize data storage and data processing.

An Azure Data Engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints. In Azure Data Engineer Associate track, there are two certifications namely DP-200 & DP-201 respectively. After attending both the trainings candidates can design and implement the management, monitoring, security and privacy of data using the full stack of Azure data services to satisfy business needs. However, soon both these exams are going to be replaced with one exam.

Note: EXAM DP-203 IS REPLACING EXAMS DP-200 AND DP-201. DP-200 and DP-201 will retire on June 30, 2021.

Course Outline

• Module 1: Design and Implement Data Storage Design a data storage structure.
• Module 2: Design and Develop Data Processing Ingest and transform data.
• Module 3: Design and Implement Data Security Design security for data policies and standards.
• Module 4:Monitor and Optimize Data Storage and Data Processing Monitor data storage and data processing.

To know more about the course outline, click here.

Why Choose CloudThat?

At CloudThat, we offer world-class teaching experience catered to the diverse needs of technology professionals. From personalized mentoring sessions to self-paced learning modules, we work towards providing a growth-oriented learning experience to nurture their future.

CloudThat has trained over 350K industry professionals from fortune 500 companies in Cloud since 2012. Empowering tech professionals in unleashing the potential of Cloud computing technologies with best-in-class training, CloudThat has proven its excellence over the years and is a proud winner of Microsoft 2020 Global Partner of the Year Finalist award.

Considering the recent technological developments, Cloud certifications are in huge demand. That is why CloudThat is offering an in-depth BootCamp course to help IT professionals become a certified Azure expert.

• A Microsoft worldwide Partner and Microsoft Gold Partner.
• Delivered highest number of Azure advance training in India in the last 4 months.
• CloudThat CEO is the MCT Regional Lead for India.
• Robust consulting division complements and feeds practical knowledge to trainers.
• Trained 3,50,000+ professionals worldwide in instructor-led classroom and in virtual instructor-led classroom.
• Our training modules are equipped with 50%-60% hands-on labs sessions.
• Highly interactive virtual and classroom teaching.
• Qualified instructor-led Training and mentoring sessions.
• Practice lab and projects aligned to Azure learning modules.
• Integrated teaching assistance and support.

Why Microsoft Azure is the Best Certification?

The job market and opportunities for Microsoft Azure certified professionals have multiplied over the last few years. With increase in demand in Cloud computing domain, Azure certification remains the most popular and widely chosen certification course. Here is why:

• According to reports, around 1,000 new customers daily sign up to Azure which means every year over 365,000 new companies adopt Azure.
• Azure is the fastest-growing enterprise Cloud platform leveraged by leading IT companies across the globe.
• Over 57% Fortune 500 companies use Microsoft Azure.
• Thanks to Azure, Microsoft’s commercial Cloud revenue increased at an annual run rate of $6.3 billion.
• In the US, even the government and their partners leverage Microsoft Azure Government to transform their critical workloads to the secure and compliant Cloud environment.
• 21% of respondents said they received a job offer or promotion upon sharing their Microsoft digital badge with a future employer.
• 41% of respondents said they received positive feedback from their professional network after sharing digital badges.

Benefits of IT Professional Certification

• 85% of enterprises committed to integrate Cloud architecture in their organizations.
• Gives you more professional credibility in any job-related interactions.
• By 2021, more than 38% of IT positions will be Cloud related.
• 19% found a new job.
• 65% experienced their first benefit of certification within 3-4 months.
• 67% acknowledged positive impact on confidence, professional image, and reputation.
• Is more effective at reducing skills gaps.
• Helps keep your skills up to date, making you more valuable to your employer.

Course Fee: $1595

Objectives

After successful completion of this DP-203 exam training course, candidates will be able to:

• Design and implement data storage
• Design and develop data processing
• Design and implement data security
• Monitor and optimize data storage and data processing

Who Should Attend

Azure Data Engineers who integrate, transform, and consolidate data from various structured and unstructured data systems into structures that are suitable for building analytics solutions.

Prerequisites

• A candidate must have solid knowledge of data processing languages, such as SQL, Python, or Scala, and they need to understand parallel processing and data architecture patterns.
• Candidate should have subject matter expertise integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building analytics solutions.

Course Outline

Module 1: Design and Implement Data Storage  Design a data storage structure

• Design an Azure Data Lake solution
• Recommend file types for storage
• Recommend file types for analytical queries
• Design for efficient querying
• Design for data pruning
• Design a folder structure that represents the levels of data transformation
• Design a distribution strategy
• Design a data archiving solution
• Design a partition strategy
• Design a partition strategy for files
• Design a partition strategy for analytical workloads
• Design a partition strategy for efficiency/performance
• Design a partition strategy for Azure Synapse Analytics
• Identify when partitioning is needed in Azure Data Lake Storage Gen2

Design the serving layer

• Design star schemas
• Design slowly changing dimensions
• Design a dimensional hierarchy
• Design a solution for temporal data
• Design for incremental loading
• Design analytical stores
• Design metastores in Azure Synapse Analytics and Azure Databricks

Implement physical data storage structures

• Implement compression
• Implement partitioning
• Implement sharding
• Implement different table geometries with Azure Synapse Analytics pools
• Implement data redundancy
• Implement distributions
• Implement data archiving

Implement logical data structures

• Build a temporal data solution
• Build a slowly changing dimension
• Build a logical folder structure
• Build external tables
• Implement file and folder structures for efficient querying and data pruning

Implement the serving layer

• Deliver data in a relational star schema
• Deliver data in Parquet files
• Maintain metadata
• Implement a dimensional hierarchy

Module 2: Design and Develop Data Processing Ingest and transform data

• Transform data by using Apache Spark
• Transform data by using Transact-SQL
• Transform data by using Data Factory
• Transform data by using Azure Synapse Pipelines
• Transform data by using Stream Analytics
• Cleanse data
• Split data
• Shred JSON
• Encode and decode data
• Configure error handling for the transformation
• Normalize and denormalize values
• Transform data by using Scala
• Perform data exploratory analysis

Design and develop a batch processing solution

• Develop batch processing solutions by using Data Factory, Data Lake, Spark, Azure Synapse Pipelines, PolyBase, and Azure Databricks
• Create data pipelines
• Design and implement incremental data loads
• Design and develop slowly changing dimensions
• Handle security and compliance requirements
• Scale resources
• Configure the batch size
• Design and create tests for data pipelines
• Integrate Jupyter/IPython notebooks into a data pipeline
• Handle duplicate data
• Handle missing data
• Handle late-arriving data
• Upsert data
• Regress to a previous state
• Design and configure exception handling
• Configure batch retention
• Design a batch processing solution
• Debug Spark jobs by using the Spark UI

Design and develop a stream processing solution

• Develop a stream processing solution by using Stream Analytics, Azure Databricks, and Azure Event Hubs
• Process data by using Spark structured streaming
• Monitor for performance and functional regressions
• Design and create windowed aggregates
• Handle schema drift
• Process time series data
• Process across partitions
• Process within one partition
• Configure checkpoints/watermarking during processing
• Scale resources
• Design and create tests for data pipelines
• Optimize pipelines for analytical or transactional purposes
• Handle interruptions
• Design and configure exception handling
• Upsert data
• Replay archived stream data
• Design a stream processing solution

Manage batches and pipelines

• Trigger batches
• Handle failed batch loads
• Validate batch loads
• Manage data pipelines in Data Factory/Synapse Pipelines
• Schedule data pipelines in Data Factory/Synapse Pipelines
• Implement version control for pipeline artifacts
• Manage Spark jobs in a pipeline

Module 3: Design and Implement Data Security Design security for data policies and standards

• Design data encryption for data at rest and in transit
• Design a data auditing strategy
• Design a data masking strategy
• Design for data privacy
• Design a data retention policy
• Design to purge data based on business requirements
• Design Azure role-based access control (Azure RBAC) and POSIX-like Access Control List (ACL) for Data Lake Storage Gen2
• Design row-level and column-level security

Implement data security

• Implement data masking
• Encrypt data at rest and in motion
• Implement row-level and column-level security
• Implement Azure RBAC
• Implement POSIX-like ACLs for Data Lake Storage Gen2
• Implement a data retention policy
• Implement a data auditing strategy
• Manage identities, keys, and secrets across different data platform technologies
• Implement secure endpoints (private and public)
• Implement resource tokens in Azure Databricks
• Load a DataFrame with sensitive information
• Write encrypted data to tables or Parquet files
• Manage sensitive information

Module 4:Monitor and Optimize Data Storage and Data Processing  Monitor data storage and data processing

• Implement logging used by Azure Monitor
• Configure monitoring services
• Measure performance of data movement
• Monitor and update statistics about data across a system
• Monitor data pipeline performance
• Measure query performance
• Monitor cluster performance
• Understand custom logging options
• Schedule and monitor pipeline tests
• Interpret Azure Monitor metrics and logs
• Interpret a Spark directed acyclic graph (DAG)

Optimize and troubleshoot data storage and data processing

• Compact small files
• Rewrite user-defined functions (UDFs)
• Handle skew in data
• Handle data spill
• Tune shuffle partitions
• Find shuffling in a pipeline
• Optimize resource management
• Tune queries by using indexers
• Tune queries by using cache
• Optimize pipelines for analytical or transactional purposes
• Optimize pipeline for descriptive versus analytical workloads
• Troubleshoot a failed spark job
• Troubleshoot a failed pipeline run

About the Trainer

A Certified Microsoft Azure Trainer.

Course Fee

$1595