IITI Computer Science & Engineering, Big Data Lab is equipped with a small Apache Spark cluster setup version 2.4.0 with Hadoop version 2.7.3.
The Spark cluster consists of six nodes (clusters); Master Node configuration: Dell precision Tower 5810, RAM: 32 GB, 4 cores, Slave Nodes configuration: Intel(R) Core(TM) i7-77000 CPU @ 3.60 GHZ, 1TB storage (each node).
HDFS is used for storing data across the cluster and Spark standalone for resource management.
HPC System Specification: Total number of cores: 32, Total memory: 187 GB. Total disk: 12 TB.
HPC server machine is used for preprocessing of raw genome data. HPC system is added into Apache Spark clusters for feature extraction of huge genome data.
Outcomes: The setup is established to check the performance of developed scalable fuzzy clustering algorithms for handling big data in various domains of pattern recognition like genomics ( to increase the productivity of next generation plants based on the conserved region of plants), Stock exchange, disease diagnosis etc.
Outcomes: The setup is established to detect the occurrences of anomalous events in a video happen for fraction of time compared to normal videos. The primary aim is to detect the occurrence of an anomalous event in the video through artificial intelligence and categorize the type of anomaly.
HPC server machines is established to check the performance of develop machine learning/artificial intelligence algorithms, like softcomputing/datamining, clustering, hybrid quantum fuzzy neural network, evolutionary optimization techniques, One-class Classification, Kernel Learning, Online Learning, Non-iterative Approaches in Learning, Multi-label Classification etc.
PARAM SHAKTI from IITKGP: PARAM SHAKTI is a High-Performance Computing facility and datacenter ecosystem at IIT Kharagpur under the National Supercomputing Mission (NSM). The supercomputer PARAM Shakti is based on a heterogeneous and hybrid configuration of Intel Xeon Skylake processors, and NVIDIA Tesla V100. It consists of 2 Master nodes, 8 Login nodes, 10 Service/Management nodes and 442 (CPU+GPU) nodes with total peak computing capacity of 1.66 (CPU+GPU) PFLOPS performance. PARAM Shakti systems are based on Intel Xeon SKL G-6148, NVIDIA Tesla V100 with total peak performance of 1.6 PFLOPS. The cluster consists of compute nodes connected with Mellanox (EDR) infiniBand interconnect network. The system uses the Lustre parallel file system.
PARAM SIDDHI from CDAC PUNE : Param Siddhi is the high-performance computing-artificial intelligence (HPC-AI) supercomputer with Rpeak of 5.267 Petaflops and 4.6 PetaflopsRmax (Sustained) was conceived by C-DAC and developed jointly with the support of the Department of Science and Technology (DST), Ministry of Electronics and Information Technology (MeitY) under NSM.
