Azure Databricks

Azure Databricks

Databricks is an Apache Spark based analytics platform available as a first party service on Azure. Optimised for Microsoft’s various cloud services, Azure Databricks integrates deeply with Azure Active Directory, Azure Data Services, Power BI and more.

Read More

Getting Started with Apache Spark

Getting Started with Apache Spark

Apache Spark is a unified analytics engine for big data processing with built-in support for interactive queries (Spark SQL), real-time analytics (Spark Streaming), machine learning (MLib) and graph processing (GraphX).

Read More

Azure Data Lake Series: Working with JSON - Part 3

Azure Data Lake Series: Working with JSON - Part 3

In part 3, we ratchet up the complexity once more to see how we handle multiple JSON files with schema structures that require extraction at multiple levels (i.e. highly hierarchical). Using U-SQL via Azure Data Lake Analytics we will transform semi-structured data into flattened CSV files to glean insights on the top restaurants across London.

Read More

Getting Started with Azure Data Factory

Getting Started with Azure Data Factory

Azure Data Factory is a fully managed data integration service that allows you to orchestrate and automate data movement and data transformation in the cloud. In Microsoft's latest release, ADF v2 has been updated with visual tools, enabling faster pipeline builds with less code.

Read More

Azure Data Lake Series: Working with JSON - Part 2

Azure Data Lake Series: Working with JSON - Part 2

In part 2, we ratchet up the complexity to see how we handle JSON schema structures more commonly encountered in the wild (i.e. an array of objects, dictionaries, nested fields, etc). Using U-SQL via Azure Data Lake Analytics we will transform semi-structured data into flattened CSV files.

Read More

Azure Data Lake Series: Working with JSON - Part 1

Azure Data Lake Series: Working with JSON - Part 1

In part 1, we begin by learning how to set up our environment in order to query JSON documents using U-SQL via Azure Data Lake Analytics, then storing the flattened results into a CSV file back into Azure Data Lake Store.

Read More

Upload a file to Azure Data Lake Store using .NET

Upload a file to Azure Data Lake Store using .NET

Learn how to upload a file from a local machine to an Azure Data Lake Store using the .NET SDK.

Read More