Linux Architecture

Summary

This video provides a comprehensive overview of Sisense's cloud-native architecture, focusing on deployment options, services, and microservices that support the system. It explains how Kubernetes, namespaces, and microservices like Prometheus, Grafana, and RabbitMQ play key roles in the infrastructure. The video details deployment configurations, from single-node to multi-node setups, with an emphasis on scalability and resilience. Additionally, it dives into the build and query flows, showing how requests are processed within Sisense’s architecture. The session concludes by summarizing the architecture, deployment options, and core services that power Sisense’s data management and visualization capabilities.

Takeaways

• ☁️ Sisense is designed as a cloud-native architecture using Kubernetes from the ground up.
• 📦 Kubernetes in Sisense is organized into three main namespaces: Sisense, Monitoring, and Kube System.
• 🖥️ The Sisense namespace contains application pods and logging tools like fluent-bit and fluentd.
• 📊 The Monitoring namespace uses Prometheus for metrics, Grafana for visualization, Alert Manager for alerts, and fluentd for monitoring support.
• 🔧 The Kube System namespace handles infrastructure tasks including scheduling, API proxying, metrics collection, networking, and shared storage.
• 💽 Sisense can be deployed in single-node or multi-node configurations, supporting various OS like Ubuntu, CentOS, and Amazon Linux 2.
• 🗂️ Multi-node deployments use shared storage to ensure data distribution and resiliency without duplicating data across servers.
• 🌐 Sisense services include client interfaces (web, mobile, REST APIs), core components (API Gateway, query/build/live connections), and infrastructure services (RabbitMQ, MongoDB, Zookeeper).
• ⚙️ Build flow involves the client request going through API Gateway, Identity Pod for authorization, Management Pod for orchestration, then data is processed and stored in Monet DB and shared storage.
• 📈 Query flow supports dashboards and widgets, converting client requests from JQL to SQL, enforcing data security via Galaxy Pod, and retrieving data from Monet DB or live sources.
• 🔌 Sisense is extensible through plug-ins and AI features like NLQ, Blocks for custom widget extensions, and pivot table support.
• ✅ Overall, Sisense provides a scalable, secure, and flexible architecture for building, querying, and visualizing data in cloud-native environments.

Q & A

What is Sisense's cloud-native architecture built on?

Sisense's cloud-native architecture is built on Kubernetes, a container orchestration platform that helps in managing containerized applications across clusters of machines.

What are the main namespaces in the Sisense architecture?

The main namespaces in the Sisense architecture are the Sisense namespace, the Monitoring namespace, and the kube-system namespace. Each has a distinct function: Sisense manages the application, Monitoring handles monitoring services, and kube-system manages housekeeping tasks for Kubernetes.

What is the role of the Sisense namespace?

The Sisense namespace contains all the pods that support the Sisense application, including logging services like Fluent Bit and Fluent D.

How does Sisense handle monitoring?

Sisense uses the Monitoring namespace, which includes services like Prometheus (for collecting monitoring data), Alert Manager (for managing alerts), Grafana (for visualizing data), and Fluent Bit/Fluent D (for logging).

What does the kube-system namespace do in the Sisense architecture?

The kube-system namespace manages the Kubernetes infrastructure, including scheduling, proxy services, API management, and other essential services such as DNS and metric collection via Metricbeats.

What are the different deployment options for Sisense?

Sisense can be deployed in single-node or multi-node configurations. In a single-node setup, everything runs on one server, while in a multi-node setup, at least three nodes are required: one build node and two query app nodes.

What types of storage does Sisense support?

Sisense supports local storage for single-node deployments and shared storage for multi-node deployments, with options like FSx, NetApp ClusterFS, and others for data distribution and resiliency.

What are the main microservices supporting Sisense?

Sisense relies on several microservices, including RabbitMQ for messaging, MongoDB for metadata storage, Zookeeper for configuration management, and additional services like plugins/extensions and the elastic cubes for data storage.

How does Sisense handle requests for builds and queries?

For builds, requests go to the build pod, which ensures user authorization and connects to data sources. For queries, requests go through the API gateway and may interact with live data sources or elastic cubes, with metadata and security being applied before fetching the data.

What is the role of the API Gateway in Sisense's architecture?

The API Gateway is responsible for receiving all external requests, routing them to appropriate services or pods (like query or build pods), and ensuring requests are properly authorized and processed.