Learning Objectives
By the end of this chapter, you will be able to:
- Define Big Data and its key characteristics (the Vs).
- Identify sources, tools, benefits, and challenges of Big Data.
- Define Blockchain and its core concepts of decentralization and immutability.
- Differentiate between public and private blockchains, and between a blockchain and a traditional database.
- Evaluate business uses and limitations of blockchain.
Big Data
Big Data refers to the extremely large and complex datasets that cannot be easily managed or analyzed with traditional data processing tools. It is often characterized by the “Vs”:
- Volume: The massive scale of the data.
- Velocity: The high speed at which the data is generated.
- Variety: The different forms the data can take (structured, unstructured).
Two more Vs are often added:
- Veracity: How accurate and trustworthy the data is.
- Value: Whether the data can be turned into useful business decisions.
Business examples: a telecom operator analyses millions of call and data records to spot network congestion; a digital wallet such as eSewa or Khalti watches transaction patterns to flag fraud; an online store such as Daraz uses browsing and purchase history to recommend products.
Types of data
| Type | Description | Example |
|---|---|---|
| Structured | Organized in rows and columns | Bank transactions, sales records |
| Semi-structured | Has tags or markers but no fixed table form | Emails, JSON and XML files, web logs |
| Unstructured | No predefined format | Photos, videos, social media posts, voice calls |
Most of the world’s new data is unstructured, which is why traditional databases alone cannot handle it.
Sources of Big Data
- Social media posts, likes, and comments.
- Transactions from POS terminals, ATMs, and wallets.
- Sensors and IoT devices (smart meters, GPS trackers, traffic cameras).
- Mobile phones — call records, app usage, location.
- Websites — clicks, searches, and browsing behaviour.
Tools and technologies
Big Data is processed by spreading work across many computers. Common tools include Hadoop (distributed storage and batch processing), Apache Spark (fast in-memory processing), NoSQL databases (such as MongoDB and Cassandra), data lakes (storing raw data of all types), and cloud services that rent large-scale analytics on demand.
Benefits and challenges
| Benefits | Challenges |
|---|---|
| Better understanding of customers | Privacy concerns and data-protection rules |
| Real-time fraud detection and risk management | High cost of storage, tools, and skilled analysts |
| Improved forecasting and planning | Poor data quality (veracity) misleads decisions |
| New products and personalized services | Security of huge data stores |
| Operational efficiency (routes, energy, stock) | Finding real value in the “noise” |
Blockchain
Blockchain is a distributed and immutable digital ledger. It is the technology that underlies cryptocurrencies like Bitcoin.
How it works
- A user starts a transaction (for example, sending value or recording a land title).
- The transaction is broadcast to a network of computers called nodes.
- The nodes verify it and group many verified transactions into a block.
- Each block stores the hash (a digital fingerprint) of the block before it, which links the blocks into a chain.
- Once the network agrees on the new block through a consensus mechanism, every node adds it to its copy of the ledger.
flowchart LR
B1[Block 1: data, hash A] --> B2[Block 2: data, previous hash A, hash B]
B2 --> B3[Block 3: data, previous hash B, hash C]
Common consensus mechanisms are Proof of Work (computers compete to solve puzzles, as in Bitcoin; uses a lot of energy) and Proof of Stake (validators are chosen based on the value they lock up; far more energy-efficient).
Key properties
- Distributed/Decentralized: The ledger is copied and spread across a network of computers. There is no central authority.
- Immutable: Once a block of transactions is added to the chain, it is extremely difficult to alter or remove. Changing one block changes its hash, which breaks the link to every block after it.
- Transparent: Participants can see and verify the same record.
This structure makes blockchain very secure and transparent.
Public vs private blockchains
- Public (permissionless): Anyone can join, read, and validate (e.g., Bitcoin, Ethereum).
- Private / permissioned: Only approved organizations can participate (e.g., a group of banks sharing a ledger for trade finance). Faster and more controlled, but less decentralized.
Blockchain vs traditional database
| Aspect | Traditional database | Blockchain |
|---|---|---|
| Control | Central administrator | Shared among many nodes |
| Changing records | Records can be updated or deleted | Append-only; past records cannot be changed |
| Trust | Users trust the database owner | Trust comes from cryptography and consensus |
| Speed | Very fast | Slower due to consensus |
| Best for | Most business applications | Shared records among parties who do not fully trust each other |
Uses beyond cryptocurrency
- Supply chain tracking: recording each step a product passes through, from farm to shop.
- Land and asset registries: tamper-resistant ownership records.
- Digital certificates: universities issuing degrees that employers can verify instantly.
- Smart contracts: agreements that execute automatically when conditions are met.
- Cross-border payments and trade finance: faster settlement between banks.
Limitations of blockchain
- Slower and more costly than a central database for ordinary tasks.
- High energy use for Proof of Work systems.
- Mistakes are hard to reverse because records are immutable.
- Legal and regulatory uncertainty, and scams involving crypto tokens.
- Not needed when a single trusted organization already maintains the records.
Nepal context: Nepal Rastra Bank treats trading and mining cryptocurrency as illegal in Nepal, but the underlying blockchain technology itself is not banned and can still be studied and applied to record-keeping.
Summary
Big Data and Blockchain are two of the most talked-about trends in technology. Big Data refers to the challenge and opportunity of working with massive, fast-moving, and varied datasets — structured, semi-structured, and unstructured — from sources like social media, transactions, and sensors, processed with distributed tools such as Hadoop and Spark. Blockchain is a revolutionary technology that provides a secure, decentralized, and immutable way to record transactions using linked blocks and consensus, with potential applications far beyond just cryptocurrency, though it is slower than central databases and not always needed.
Key Takeaways
- Big Data is characterized by its volume, velocity, and variety (plus veracity and value).
- Most new data is unstructured and needs distributed tools to process.
- Blockchain is a distributed, immutable ledger secured by hashes and consensus.
- Blockchain is the technology behind Bitcoin, but has wider uses such as supply chains and certificates.
- Blockchain fits shared records among parties who do not fully trust each other; otherwise, a database is usually better.
Discussion Questions
- What are some of the sources that generate “Big Data”?
- What are the main advantages of a decentralized system like blockchain compared to a traditional centralized database?
- Besides cryptocurrency, what is another potential application for blockchain technology?
Exam-Style Questions
Short answer
- Define Big Data. Explain its 5 Vs briefly.
- What is a hash in blockchain?
- Differentiate between public and private blockchains.
Long answer
- Explain the sources, benefits, and challenges of Big Data for businesses.
- What is blockchain? Explain how it works and discuss its applications and limitations.
