The almighty NoSQL database to power mobile sites with real-time analytics for better customer experience
Began as the storage system for the web search index at Google. Made available as a public service via Google Cloud Platform in 2015 and currently powers some of the most prominent web and mobile applications such as Spotify, Google Maps and Google Search. It’s no other than Cloud Bigtable, which is neither suitable for complex joins, ad-hoc interactive queries nor ideal for supporting an online transaction processing (OLTP) system.
So what is it and why might we need a massive database like Cloud Bigtable one day?
You’ll learn:
- The real-world challenge: Site speed and personalisation customer engagement
- A primer…
How to avoid those dreaded pitfalls and “gotcha” moments when selecting databases for your next application?
Asking any developer of an enterprise application, and you’ll know how much they are feeling frustrated with the limitations of relational databases. Too much to the extent that in 2009, a meetup was held in San Francisco to discuss with his peers about open-source, distributed, non-relational databases.
Originally chosen to make a good Twitter hashtag for that meetup, NoSQL caught on like wildfire, yet resulted in no generally accepted definition till now. Fast forward to 2021, businesses have an abundance of choices when it comes to data storage solutions, both relational and non-relational ones.
Yet one thing remains the same…
In a post-COVID-19 future, businesses must be prepared to respond and adapt rapidly. Does the old Hadoop technology even have a place in this new world?
In the emerging world of 2021, big data and machine learning are still the holy grail while the need for superior speed and agility continues to accelerate cloud adoption. Does the old Hadoop technology even have a place in this new world? To shed some light on that question, let’s reflect on what Hadoop is and how the cloud is impacting Hadoop.
In this article, you’ll:
- Get to know Hadoop
- Explore Hadoop on the cloud
- Look to the future and beyond with modern cloud platforms
Get to know Hadoop
Why was Hadoop a big deal?
Once upon a time, as our business grew, we started to generate too much data…
When all you want is simple steps to get good data scattered everywhere into a single location for quick business insights, don’t let the complexities of coding and infrastructure hold you back.
Knowing how to integrate data from various sources and perform simple transformation to address data quality issues is the first step towards extracting insights from big data. In this blog post, we will explore how to build and deploy simple ETL data pipelines without coding via Cloud Data Fusion on Google Cloud Platform (GCP).
You’ll learn:
- Why do we need Cloud Data Fusion?
- How to build and deploy ETL pipelines with Cloud Data Fusion?
Why do we need Cloud Data Fusion?
The problem: Your first hurdle towards great insights from big data
Despite the rising popularity of sophisticated data analytics and machine learning techniques, something has never changed. The first hurdle towards great insights is usually data integration…
Computer vision has the potential to improve the consumer experience, reduce costs and enhance security. Understanding the fundamentals of computer vision is the first step towards unlocking a crucial competitive edge. Are you ready?
Computer vision is one of the most extraordinary gifts coming out of the artificial intelligence world. With computer vision, many companies have attempted to see the world through computers’ eyes and made great strides in solving complex business problems such as identifying product defects in real-time, verifying customers’ identification or automating insurance claims process. Overlooking such real-life applications of computer vision could represent missed opportunities to unlock growth, productivity and cost-savings for businesses. So what is computer vision and how can it help?
You’ll learn:
- What is computer vision?
- How is computer vision applied in today’s world?
- How do we…
Making Sense of Big Data
How do you obtain business insights when all you have on hand is raw unprocessed data scattered everywhere?
Born in the 1980s, about 40 years old and still counting. It’s no other than the data warehouse, which requires some hefty investment, might take years to build. Yet the chance of failure is sky-high. Fast forward to 2021, the data warehouse has been evolving with time and will continue to be the backbone for business insights across organisations all over the world. So what is it? Why do we need a data warehouse in the first place? As a data professional, what do you need to know about data warehouse at the bare minimum?
In this article, you’ll learn:
- …
Effortless approach to determine what is in the BigQuery dataset and which tables are useful for analysis with INFORMATION_SCHEMA and TABLES
Metadata! I bet you might have heard this term before and may have asked yourself what it is and why it is important. Let’s explore this concept with Google BigQuery.
In this article, you’ll discover:
- What is metadata?
- Why query table metadata in Google BigQuery?
- How to query table metadata with INFORMATION_SCHEMA and TABLES?
What is metadata?
Many sources define metadata as “data about data”. But I personally find it too vague and difficult to understand. So here is my attempt to define metadata in layman’s terms.
Stepping up your SQL query game with these 2 advanced SQL concepts is easier than you think. Don’t miss out!
A good understanding of arrays and structs could be extremely powerful when analyzing big data because we can query faster and more efficiently with pre-joined tables from object-based schemas such as JSON or Avro files. In this blog post, we will explore arrays, structs and how to make use of their full potential in Google BigQuery through lots of examples.
You’ll learn
- Why do we need to know arrays and structs?
- Arrays and how to work with them
- Structs and how to combine them with arrays to create nested records
Enough talking, let’s start!
Why do we need to know arrays and structs?
It’s time to take your queries up a notch with Date Partition, Analytic Window Functions and WITH clause
I’m starting 2021 with one of the essential New Year’s resolutions: Practice more complex SQL queries. If you’re on the same boat, join me to explore 3 useful SQL features with Google BigQuery.
You’ll learn how to:
- Create tables with date partitions
- Aggregate over a group of rows with (analytic) window functions
- Break down complex queries using WITH clause
Depending on your background, these might seem like basic features that appeared in other relational databases or they may appear exotic. Either way, I have included detailed examples and approach on how I tackled each query. …
With HyperLogLog algorithm, we trade off some accuracy to get the job done faster. But how much faster and at the expense of what level of accuracy?
HyperLogLog (HLL) is an algorithm that estimates how many unique elements the dataset contains. Google BigQuery has leveraged this algorithm to approximately count unique elements for a very large dataset with 1 billion rows and above.
In this article, we’ll cover 2 points.
- What’s HLL?
- How does HLL compare with other methods?
What’s HLL?
The HyperLogLog algorithm can estimate cardinalities well beyond 10⁹ with a relative accuracy (standard error) of 2% while only using 1.5kb of memory.
-HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm by Philippe Flajolet et al.
HLL is one of many cardinality estimation algorithms that are all…
Skye Tran
Love data, problem-solving, and storytelling | Observe the world through technology-driven lens | Cherish order from chaos
