Unboxing the Tenstorrent Grayskull AI Accelerator!

Summary

In this engaging conversation, Yasmina, a fellow at T-torrent, discusses the company's innovative approach to hardware-software co-design, emphasizing the importance of accessible programming for new architectures. She introduces T-torrent's gen one hardware, the Grull dev kit, designed for developers with a focus on ease of use and community feedback. The discussion highlights the balance between high-level and low-level programming entry points, the company's commitment to open-sourcing their bare metal software stack, and the future integration with PyTorch 2.0. Yasmina shares insights on the company's goals to empower developers and grow the AI hardware ecosystem collaboratively.

Takeaways

• 🎉 Yasmina is a guest on the channel, discussing the exciting developments at Tenstorrent, a company specializing in novel hardware and software.
• 🌟 Tenstorrent focuses on hardware-software co-design, emphasizing the importance of making new hardware accessible through programming.
• 🔧 Yasmina's background in FPGAs and high-level synthesis has been instrumental in her work at Tenstorrent, contributing to the development of their novel architectures.
• 💡 The company is releasing its first generation hardware, called 'Grull', in the form of a developer kit, with two versions: E75 and E150, differing in wattage and size.
• 📦 The Grull developer kit includes everything needed to set up and start working with the hardware, targeting developers and small to medium-sized businesses.
• 💻 The developer kit is designed to be used with Linux, with Windows support planned for the future, and it comes with tools and drivers for developers to get started.
• 🛠️ Tenstorrent offers two software stacks: a high-level entry point called 'Buddha' for easy model deployment, and a low-level 'bare metal' programming model for fine-grained control.
• 💸 The E75 version of the Grull developer kit is priced at $599, while the E150 version is priced at $799, positioning them as accessible to developers and businesses.
• 🔄 Tenstorrent is committed to open-sourcing their bare metal software stack, allowing developers to see and interact with the underlying APIs and system components.
• 🔄 The company aims for forward compatibility, ensuring that anything developers create for the current generation will work with future Tenstorrent hardware.
• 🤝 Tenstorrent values community engagement and feedback, encouraging developers to test, experiment, and provide input to improve their products.

Q & A

What is the significance of Yasmina's background in FPGAs to her current role at Tenstorrent?

Yasmina's background in FPGAs, including her doctorate and experience in high-level synthesis, is significant because it provides her with a strong foundation in hardware-software co-design. This expertise is crucial for developing novel architectures that are easy and enjoyable to program, which is a key focus at Tenstorrent.

What is the main goal of Tenstorrent's hardware-software co-design approach?

The main goal of Tenstorrent's hardware-software co-design approach is to create hardware and software environments that are accessible and engaging for developers. They aim to provide easy entry points for programming, empowering developers to create innovative applications and feel excited about the potential of new hardware and software integrations.

What are the two different versions of the Tenstorrent's Grail dev kit mentioned in the transcript?

The two different versions of the Tenstorrent's Grail dev kit mentioned are the e75, which is a smaller card with 75 watts, and the e150, which is slightly larger and has a higher wattage.

How much does the Tenstorrent's Grail dev kit cost, and who is the intended audience for this product?

The Tenstorrent's Grail dev kit costs $599 for the 75-watt version and $799 for the 150-watt version. The intended audience is developers and small to medium-sized businesses interested in exploring the utility of the hardware for their models.

What is the purpose of the 'Buddha' software stack offered by Tenstorrent?

The 'Buddha' software stack is a compiler developed by Tenstorrent. It serves as a high-level entry point for developers, allowing them to easily download models from sources like Hugging Face and run them on Tenstorrent's hardware without needing to change their environment or rewrite their models.

What is the 'bare metal programming model' that Yasmina refers to, and who is it intended for?

The 'bare metal programming model' is a low-level software stack that provides developers with fine-grained control over the workloads running on Tenstorrent's hardware. It is intended for developers who want to have a deep understanding of the underlying architecture and are interested in customizing their applications at a lower level, including writing custom operators and exploring novel hardware features.

What is Tenstorrent's policy on community feedback and open sourcing their software?

Tenstorrent encourages community feedback and plans to open source their full bare metal software stack. They are interested in hearing both positive and negative feedback to improve their products. They believe in empowering developers and fostering an open collaboration environment, which includes sharing APIs, kernel compilation methods, memory allocators, and other底层细节 to allow developers to fully explore and utilize their hardware.

How does Tenstorrent ensure backward compatibility for their APIs?

Tenstorrent aims to maintain backward compatibility for their host APIs, which are designed to be intuitive and familiar to developers, mimicking existing models like OpenCL and CUDA. For kernel APIs, they value backward compatibility but also recognize the potential for significant performance and functionality improvements in next-generation architectures, which might require them to allow for some changes while ensuring that everything written for the current generation will work on future generations.

What are the differences between high-level and low-level entry points for developers in Tenstorrent's software stacks?

High-level entry points, like the Buddha software stack, offer a quick path to desired outcomes without requiring developers to change their environment or rewrite their models. They are less susceptible to low-level changes and provide a consolidated, specialized API for certain purposes. Low-level entry points, on the other hand, give developers full access to the hardware and allow for fine-grained control, enabling them to write custom kernels and explore the hardware's capabilities in depth.

How does Tenstorrent support developers in accessing their hardware, both physically and through the cloud?

null

What is the significance of Tenstorrent's decision to open source their bare metal software stack?

By open sourcing their bare metal software stack, Tenstorrent aims to foster community engagement and collaboration. This transparency allows developers to see and understand the underlying mechanisms of the software, including how kernels are compiled, memory is allocated, and runtime arguments are managed. It also ensures that anything developers write for the current hardware will be forward-compatible with future Tenstorrent hardware generations.