Understanding Wide-Column Stores

To understand the concept  more clearly, let’s start with an example of Relational database:

Customers
Id
Product
FN
Date
Country
1
Dell
Harry
17/04/2018
India
2
Dell
Harry
17/04/2018
India
3
Apple
Ron
17/04/2018
India
4
Sony
Ron
17/04/2018
South Africa
5
Sony
Hermione
17/04/2018
South Africa

When we convert this to a Column Store, this is how it looks like:
FN
LN
Product
Date
Country
Id
Value
Id
Value
Id
Value
Id
Value
Id
Value
1-2
Harry
1-2
Potter
1-2
Dell
1-5
17/04/2018
1-3
India
3-4
Ron
3-4
Weasley
3
Apple
4-5
South Africa
5
Hermione
5
Granger
4-5
Sony

Now a Wide Column store will Group the columns which are accessed frequently together into one , like

Id
Name
1-2
FN
Harry
LN
Potter
3-4
FN
Ron
LN
Weasley
5
FN
Hermione
LN
Granger

Some key points are,

  • Columns are created for each row rather than being predefined by the table structure. So, each row can have different columns.
  • We can thing of wide columns as a key-value collection, where each value in the collection is either a simple data type or another key-value collection.
  • In most systems, a third piece of data is usually held for each column: a timestamp that records when the data was added to the database or last modified. In above example, there would be one more column along with FN and LN to store timestamp.
  • Unlike traditional databases, a column in a wide-column database is not something that you define in advance – they are created as needed when sending data to the database.
The advantage of this design is processing power and ability to work with truly massive data sets.
Two most popular NoSQL wide-column databases are Cassandra, HBase and Amazon SimpleDB.

Comments

Post a Comment

Popular posts from this blog

Two pizza team!

Image
Came across this terminology while reading about APIs based on microservices architecture. So, what are two pizza teams? Yes, they are small enough to be fed by 2 large pizzas! But how are they related to microservices?! We will find out. Amazon, being a popular online store, wanted to be able to innovate in IT space quickly. But their monolith architecture was not allowing them to make software changes quickly. Monolith architectures are the old school enterprise architectures where all the components of the software are tightly coupled. These APIs are powerful and perform lot of disparate functions. Even a small change in such systems needs a lot of planning, impact analysis, availability from other teams. This definitely slows down the software release cycle. Typically in such environments there are different teams like test team, deployment team, UI team, backend development team etc who work in silos. Any small or big change goes through a long process of build, test, deploy...
Read more

Data governance and (hence) Metadata Management

Data is a strategic asset for businesses today and it is growing exponentially. Regulations like GDPR, BCBS 239, Basel 3 are making it extremely important for organisations to have full control over their data. Data governance is about availability, usability, integrity and security of data. Is this something new we are learning here? Actually, No! Data governance has been in place since years. Every system has some or other form of data governance in place. Documents and excel sheets are traditional form of data governance. Could be access control mechanism implemented at system level. Operational fields in a databases like created by/ updated by, are also part of this governance. Audit logs provides useful information for governance purposes.   But is this sufficient? We are implementing new systems, new functionalities every day. Is the organisation working in silos where individual departments might have different standards? Is managing metadata centrally in excel she...
Read more