Scribd Logo
Upload

Welcome to Scribd!

  • 10 views

    Azure Book 127

    Uploaded by

    jayantascribe

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Azure Book 127

    Uploaded by

    jayantascribe
    10 views1 page

    Original Title

    Azure book 127

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    10 views1 page

    Azure Book 127

    Uploaded by

    jayantascribe

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 1

    114

    Criteria Azure Data Factory Azure Synapse Analytics


    Data sharing Can be shared across different No sharing of data
    data factories
    Solution templates Provided with Azure Data Factory Provided with Synapse Work-
    template gallery space Knowledge center
    Integration Runtime cross region Support Cross region data flows Does not support cross region
    support data flows
    Monitoring Integrated with Azure Monitor Diagnostic logs in Azure Monitor
    Monitoring of Spark Jobs for Not supported Supported by the Synapse Spark
    Data Flow pools

    Design a strategy for hot, warm, and cold data


    paths
    Traditionally data was stored on premises without taking into consideration its usage and lifecycle. In the
    cloud, data can be stored based on access, lifecycle, and other compliance requirements. Let us learn how
    the concept of hot, warm, and cold data path determines how we store and compute data.

    Understand Warm Path


    Let us look at the common scenario of IoT device data aggregation. The devices could be sending data
    while not really producing anything. This highlights a very common challenge when trying to extract
    insight out of IoT data: the data you are looking for is not available in the data you are getting. Therefore,
    we need to infer utilization by combining the data we are getting with other sources of data, and apply-
    ing rules to determine of whether or not the machine is producing. In addition, these rules may change
    from company to company since they may have different interpretations of what “producing” is.
    The warm data path is about analyzing as the data flows through the system. We process this stream in
    near-real time, save it to the warm storage, and push it to the analytics clients.
    The Azure platform provides many options for processing the events and one popular choice is Stream
    Analytics.
    Stream Analytics can execute complex analysis at scale, for example, tumbling/sliding/hopping windows,
    stream aggregations, and external data source joins. For even more complex processing, performance
    can be extended by cascading multiple instances of Event Hubs, Stream Analytics jobs, and Azure
    functions.
    Warm storage can be implemented with various services on the Azure platform, such as Azure SQL
    Database and Azure Cosmos DB.

    Understand Cold Path


    The warm path is where the stream processing occurs to discover patterns over time. However, there
    might be a need to calculate the utilization over a period in the past, with different pivots, and aggrega-
    tions and to merge those results with the warm path results to present a unified view to the user.
    The cold path includes the batch layer and the serving layers. The combination provides a long-term view
    of the system.

    You might also like

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy