Posts

Understanding Wide-Column Stores

To understand the concept  more clearly, let’s start with an example of  Relational database : Customers Id Product FN Date Country 1 Dell Harry 17/04/2018 India 2 Dell Harry 17/04/2018 India 3 Apple Ron 17/04/2018 India 4 Sony Ron 17/04/2018 South Africa 5 Sony Hermione 17/04/2018 South Africa When we convert this to a  Column Store , this is how it looks like: FN LN Product Date Country Id Value Id Value Id Value Id Value Id Value 1-2 Harry 1-2 Potter 1-2 Dell 1-5 17/04/2018 1-3 India 3-4 Ron 3-4 Weasley 3 Apple 4-5 South Africa 5 Hermione 5 Granger 4-5 Sony Now a  Wide Column store  will Group the columns which are accessed frequently together into one , like Id Name 1-2 FN Harry LN Potter 3-4 FN Ron LN Weasley 5 FN Hermione LN Granger Some key points are, Columns
Post a Comment
Read more

Relational and Non-relational databases

In old days, when we used to think of 'database', we would only picture tables and columns, primary and foreign keys, complex joins, intersection tables etc. We often use ER diagrams, relationships to design and define such models. These databases are widely used and are stable. They adhere to  ACID  principles, provide  immediate consistency  which are most important when it comes to transaction handling. The most widely used products are  Oracle, Teradata, SAP Hana, IBM db2, MySQL, Amazon Aurora, Maria db (comes by default on linux distributions), postgreSQL . Then why is there a need of non-relational database? The reason is Data itself. Data is now considered most valuable resource, even more than Oil!! No wonder that social networking site like Facebook and twitter actually run the world. They decide the trends, what people read, what they think. All from the power of Data. Anyways that’s a topic for another post. Data from web, genomics data, data from IoT devices
Post a Comment
Read more

Two pizza team!

Image
Came across this terminology while reading about APIs based on microservices architecture. So, what are two pizza teams? Yes, they are small enough to be fed by 2 large pizzas! But how are they related to microservices?! We will find out. Amazon, being a popular online store, wanted to be able to innovate in IT space quickly. But their monolith architecture was not allowing them to make software changes quickly. Monolith architectures are the old school enterprise architectures where all the components of the software are tightly coupled. These APIs are powerful and perform lot of disparate functions. Even a small change in such systems needs a lot of planning, impact analysis, availability from other teams. This definitely slows down the software release cycle. Typically in such environments there are different teams like test team, deployment team, UI team, backend development team etc who work in silos. Any small or big change goes through a long process of build, test, deploy
4 comments
Read more

Google Cloud on Air

So, I have spent my day today listening to this global online live conference by Google. I know quite a bit about Amazon AWS platform, but didn’t really had much idea about Google cloud platform before joining this conference. And needless to say, google cloud on air, was awesome. Well, some sessions were not as good, but most of them were very informative and they encourage you to explore the google cloud platform even more. The sessions were not only talks and presentations (which were extremely informative anyways), but they were filled with demos and hands on exercizes to get us started on the platform.   ‘Cloud on air’ had 3 parallel tracks. It was a tough choice to choose between them. But then the cloud on air was being broadcasted in 3 timezones, so you can easily watch the session you have missed in the subsequent broadcast. The three tracks were Machine Learning/ AI, Big Data and Industry Solutions. And all of them were amazing.   Right from developing the services ha
Post a Comment
Read more

People who bought also bought..

Recommendation systems are everywhere. Right from amazon.com , netflix, youtube, super market’s advertising..and where not! When it all started a few years back, I used to think, wow! How does amazon knows which dress I might like next? That was really interesting. Now we all know that it’s the magic of data science. Machine learning algorithms are continuously capturing the data, learning from it and providing the recommendations. Today we are going through it in a bit more detail.   So, how data can predict what the user is gonna like or gonna buy? Based on prior knowledge. And here comes our first Association Rule Learning Algorithms, Apriori. The name tells us, that it’s going to predict something based on prior knowledge. Apriori works on 3 factors. Support, Confidence and Lift. Lets say, out of 100 people, 10 people like the movie ‘fantastic beasts and where to find them’. So, we can predict that the probability is 10%, for any other set of netflix watchers. This is g
Post a Comment
Read more

Into the Cloud

I started my career with CRM industry in 2007. 10 years have passed since then. And look how everything has changed! It’s not only Siebel CRM to SFDC or only a change in a CRM platform. It’s not only move from waterfall to agile. There is more to it. Siebel CRM was a revolutionary product in itself. And so as others at that time. Oracle, SAP, these companies started to come up with Products, rather than only services. Now there was no need to start coding from scratch to build your CRM or ERP platform. Only customisations to these ready made products were needed. These products were spread across various horizontals and verticals, making them almost ready to use for any industry. No need to write thousands of lines of code! These technologies proved cutting edge technologies in those times. And then Cloud walked in. Companies spent a lot of money, time in managing the infrastructure. With cloud, this burden was taken off IT. Now you have freedom to utilise these efforts in mo
Post a Comment
Read more

Pursuit of Happyness

It was a wonderful movie. I had tears in my eyes when he finally gets the Job! Beautiful really. 'Happiness' is philosophical, isn’t it? We all think, living a good life within the norms of the society, will give us happiness. What is normal? Get a degree, get a job, earn money, fall in love, get married, have children, have a perfect family..this is the normal cycle in the society. Any one missing link from these …and we look different from others. And it is completely okay to be different. We don’t need the society to accept us. Only We need to accept ourselves. That is all is needed. Happiness is relative. Problem starts when we lose ourselves in our unconscious (or conscious! ) efforts in trying to fit in the society. It could be anything… like, getting an education in a field you never wanted to be in. Or your job is not giving you anything other than the monthly paycheque. Ya.. we all work for money. There is no doubt on importance of earning and being able to
Post a Comment
Read more