Install · Connect MongoDB Atlas · Build your first live panel
| Time | Segment | Activity |
|---|---|---|
| 0–10 min | Recap | Module 3 wrap-up — from streaming to visualising |
| 10–25 min | What Is Grafana? | Architecture, who uses it, industry context |
| 25–40 min | Grafana Mental Model | Data sources, queries, panels, dashboards |
| 40–55 min | Setup & Connection | Docker install, MongoDB plugin, connect Atlas |
| 55–75 min | Hands-On Lab | Build first panel: stock prices time series |
| 75–85 min | Panel Types | Time Series, Stat, Bar Gauge, Table overview |
| 85–90 min | A3 Preview | What students will build for Assignment 3 |
In Modules 2 and 3, you built the full backend: Kafka producers, consumers, MongoDB storage, alert routing, and windowing. All that work produces data — but the data lives in a database only engineers can query. Grafana closes this gap. It turns database queries into visual panels that refresh automatically, putting your streaming data in front of the people who need to act on it.
| What It Is | What It Isn't |
|---|---|
| A dashboard and visualisation platform | A database |
| A query builder (UI on top of your data) | A streaming system |
| An alerting layer with notification routing | A replacement for MongoDB/Kafka |
| Open source + free for self-hosted use | An analytics/BI tool (that's Metabase, Tableau) |
2014 — Created by Torkel Odegaard as a fork of Kibana
2021 — Grafana Labs valued at $3 billion
2024 — ~20 million users, 200,000+ organisations
Default monitoring dashboard at:
Netflix, PayPal, Booking.com, Zepto, Swiggy, Paytm, HDFC Securities
Career context: When you walk into a data-forward company's operations centre and see a wall of live charts, there is a 70%+ chance you are looking at Grafana. Understanding this tool positions you to have meaningful conversations about data infrastructure — as a manager, analyst, or product leader.
Unlike static BI tools, Grafana treats time as the primary dimension. Every dashboard has a time range (last 5 min, last 1 hour, custom). This makes it ideal for streaming data — you can zoom into the exact window when an alert fired.
docker run -d \ --name grafana \ -p 3000:3000 \ -e GF_SECURITY_ADMIN_PASSWORD=admin \ grafana/grafana-oss:latest # Open: http://localhost:3000 # Login: admin / admin (Grafana will prompt you to change it)
docker exec -it grafana grafana-cli plugins install grafana-mongodb-datasource docker restart grafana
Left sidebar → Connections → Data Sources → Add data source
Search "MongoDB" → select the community plugin
Connection string: mongodb+srv://<user>:<password>@cluster0.xxxxx.mongodb.net/
Database: sda_course
Click "Save & Test" — should show ✅ "Data source is working"
Build a time series panel showing INFY stock price over the last 30 minutes, auto-refreshing every 10 seconds — powered by the stock_prices collection from Lectures 8 & 9.
Left sidebar → Dashboards → New → New Dashboard → Add visualisation → Select MongoDB data source.
{
"collection": "stock_prices",
"aggregate": [
{ "$match": { "symbol": "INFY" } },
{ "$sort": { "_saved_at": 1 } },
{ "$project": { "_id": 0, "time": "$_saved_at", "price": 1 } }
]
}
Map Time field → time | Value field → price
Add a Stat panel for open alert count:
{
"collection": "stock_alerts",
"aggregate": [
{ "$match": { "status": "open" } },
{ "$count": "value" }
]
}
Thresholds: 0 = green | 5 = amber | 10 = red
Teaching moment: When a price spike fires an alert, students see both the time series move AND the alert counter increment in real time. This is the "aha" moment that makes the full pipeline visible — from Kafka producer to live dashboard.
| Panel Type | Best For | Example Use |
|---|---|---|
| Time Series | Trends over time | Stock price history, transaction volume per minute |
| Stat | Single important number | Current price, total open alerts today |
| Bar Gauge | Progress / comparison | Alert count per symbol (INFY vs TCS) |
| Bar Chart | Categorical comparison | Transactions per city in last hour |
| Table | Raw data / detail view | Latest 10 alerts from stock_alerts |
| Gauge | Single value with thresholds | Risk score: green/amber/red |
Full hands-on build session: multi-panel stock monitor, variable dropdowns, the transactions dashboard, and Assignment 3 guidance. Keep Grafana running and your producer streaming before class.