Azure Data Lake : A wonderful Scalable Cloud Storage Solution for all your Big Data Needs
Use Cases and Deployment Scope
Stored Terabytes of Healthcare data in a cost-optimized solution on-cloud using Azure Data Lake Storage Gen2 in containerized fashion. We utilized Azure Data Lake Storage containers as a Destination in our Data Engineering Streasmets Pipelines. Loaded Data became available further to multiple downstream applications in an automated and faster way using Azure Data Factory. Also turned out a better, cost-optimized, and faster solution than HDFS for our different business use cases like the migration of huge data from RDBMS to Data Lake.
Pros
- Setting up Azure Data Lake Storage account, container is quite easy
- Access from anywhere and easy maintenance
- Integration with Azure Data Factory service for end to end pipeline is pretty easy
- Can store Any form of data (Structured, Unstructured, Semi) in faster manner
Cons
- UI search feature can certainly be improvised e.g. inclusion of wildcards to search a particular file in container
- Sometimes gets Hanged/lagged while monitoring
- Probably the new UI feature can address above issues.
Likelihood to Recommend
Azure Data Lake storage is well suited for applications/use cases within organizations where capturing and storing large amounts of data in any format is required, primarily for storing and processing purposes. It's an easy and cost-effective cloud solution for your application data. The ability to integrate with other Azure Services like Azure Databricks and Azure Data Factory is superb.
