UT08x12: Revolutionizing Data Infrastructure for AI with WEKA

Storage software running on modern hardware can deliver incredible performance and capability to support AI applications. This episode of Utilizing Tech wraps up our season with a discussion of WEKA’s data platform for AI with Alan McSeveney, Scott Shadley of Solidigm, and host Stephen Foskett. Modern hardware is capable of incredible performance, but bottlenecks remain. The limiting factor for AI processors is memory capacity: GPUs are hungry for data and must be refreshed from storage quickly enough to keep them running at scale. Storage can also be used to share data between GPUs across the data center and to cache working data to accelerate calculation. The secret to scalability, from storage to applications to AI, is distribution and parallel processing. Modern software runs at incredible scale, and all elements of the stack must match. Technologies like Kubernetes allow applications to use huge clusters of workers all contributing to scale and performance. WEKA runs this way, matching the GPU clusters and web applications we rely on today.