Home United States USA — software Data Security Considerations in Cloud Data Warehouses

Data Security Considerations in Cloud Data Warehouses

110
0
SHARE

In this article, we will discuss the move to cloud data warehouses (CDWs) and security considerations when using them.
Join the DZone community and get the full member experience. This is an article from DZone’s 2022 Data Pipelines Trend Report. For more:
Cloud data warehouses (CDWs) are making rapid growth in the way organizations are analyzing data at scale. As cloud storage is elastic and cheap, and as modern data pipelines simplify ETL processes, they commonly scale to store more data than on-premises data warehouses. This obviously includes sensitive data, leading to challenges in security, privacy, governance, and compliance. In this article, we will discuss the move to CDWs and security considerations when using them. Over a decade ago, public cloud companies started releasing data warehouses in a Platform-as-a-Service (PaaS) model. Significantly, Google BigQuery in 2010 and Amazon Redshift in 2012 enabled organizations to deploy a CDW in minutes, without the need to install the databases or configure the servers. This was followed by the launch and gradual growth of other vendors, later becoming the largest software IPO with its data cloud. The move to a CDW (which is considered part of the modern data stack) has significant implications for the ease for data consumers and producers to access data. Cloud computing allows elasticity, storage becomes cheaper and easier to increase (or decrease), and database administration becomes much simpler (for example, in some of the CDW platforms, there are no indexes that DBAs need to maintain). When organizations use CDWs, the basics of information security remain the same, but some things are different versus having an on-premises data warehouse, or even a data warehouse you manually install on a cloud infrastructure. Part of the difference is because of the shared responsibility model. Your provider assumes full responsibility for certain things, such as physical security, operating system security and patching, and even maintaining the database software. This leaves you narrower areas of focus. However, in many cases, the move to the cloud also allows more users to access and make value from your data (in what’s referred to as “data democratization”). The move from specific teams using data to many teams accessing it and having data that keeps changing brings more challenges. In addition, some assumptions of on-premises data storage (including that it’s disconnected from the internet by default) are not always correct in cloud data storage. The complexity in applying data security to CDWs is also in the people managing them in organizations. These are mostly data engineers and not security professionals, and in many companies, this creates a situation where the CDW is a “black box” for the security teams who don’t have full control over the security policies.

Continue reading...