We are happy to feature an interview with Jay Dawani, the Co-Founder & CEO of Lemurian Labs. Dawani and his team have embarked on a mission to democratize AI development, making it accessible, affordable, and sustainable for all. In this interview, we explore into Dawani’s journey, insights, the pivotal moments that shaped his career, the challenges of contemporary AI development, and the transformative potential of emerging technologies.
Jay, you’ve been a pioneer in integrating advanced technologies like AI and quantum computing into real-world applications. Can you share a pivotal moment in your career that led you to focus on making AI development more accessible and affordable?
Absolutely. At the start of 2018, I was working with my team on training a foundation model for general purpose autonomy. We had trained a simple model as a proof of concept and were starting to scale it up from 350 million parameters to 2 billion parameters on our 256 GPU cluster. When we realized the model would need to get a whole lot bigger we had to abandon the effort because the sheer cost was unjustifiable. We expected to need 40,000 GPUs, but it was closer to 200,000. That much compute, and the cost to run that machine is well beyond reach for our startup.
This is a problem that will continue to get worse as AI models become larger and more complex. Based on current scaling trends, we can expect the cost of training a frontier AI model to exceed a billion dollars any day now. This means only a few companies with their own AI supercomputers housed in their datacenters will be able to develop these models.
We started Lemurian Labs to figure out how to rein in the costs of AI as models become larger and more resource demanding, and serve these models at scale so everyone can use them in an energy efficient and economical way. This required a comprehensive reevaluation of both hardware and software. By emphasizing efficiency, utilization, and scalability, we successfully crafted an accelerated computing platform that not only delivers superior performance within the same energy constraints but also simplifies the process for developers to extract more performance. With our software and hardware-based approach, we aim to level the playing field and empower individuals and organizations of all sizes to harness the transformative potential of AI without barriers.
Your work at Lemurian Labs aims to democratize AI by addressing the compute crisis. Could you explain the core challenges in current AI development related to compute resources, and how your approach at Lemurian Labs is set to change that landscape?
The heart of the challenge in today’s AI landscape lies in the ever-escalating costs associated with compute resources, particularly in the training phase of large-scale AI models. As these models continue to expand in both complexity and size, it’s apparent that existing infrastructure is ill-equipped to handle such demand. Now legacy GPUs are burning an exorbitant amount of power to keep up, feeding the demand for additional data centers, and skyrocketing development costs while taking a significant toll on the environment. At Lemurian Labs, we’re tackling these issues head-on by fundamentally reshaping the economic dynamics of data centers, and democratizing AI development for all.
Our approach is two-fold. Optimize both hardware and software stacks to streamline operations and drive down costs while maintaining peak performance. Drive innovation to lessen the environmental footprint of AI thereby, reduce expenses and usher in sustainability as a part of data center management. In a major step towards this effort, we raised a $9M seed round last fall to develop our new number format PAL (parallel adaptive logarithm) that enabled us to design a processor capable of achieving up to 20 times greater throughput compared to traditional GPUs on benchmark AI workloads.
Through these concerted efforts, we envision a future where AI development is not only more financially feasible but also environmentally conscious, ensuring that the transformative power of AI is accessible to all.
Having worked at the frontier of AI and having advised many leading companies, you have a unique vantage point on the cutting edge of technology. What emerging trends or technologies do you believe will have the most significant impact on AI and automation in the next decade?
It’s hard to say for sure what the world will look like in a decade, especially given the pace of innovation and breakthroughs today. We live in a world of acceleration and exponentials.
The most interesting things almost always happen at the boundaries or intersection of fields. I think we will see radical innovation in cloud computing, datacenter infrastructure, computer architectures, and compilers. It is the convergence of them that will enable further progress in AI.
The framework we use at Lemurian is to understand changes in constraints and what technologies need to intersect to give us new capabilities. One in particular that we view as important is that software needs to be reimagined for a world where large scale heterogeneous computing is the norm. We see the need for better computer architectures and infrastructure, but their adoption is limited by the robustness of software. Existing software stacks limit the direction in which architectures are able to evolve, which imposes a limit on the kind of AI models that can take root.
The Lemurian Labs software stack will open up new opportunities for system design in the future which is ultimately our vision. In the shorter term we can change the economics of AI by giving greater utilization and throughput on existing hardware while making it easier for developers to train and deploy models, and making it less burdensome to adopt new alternative hardware architectures.
Sustainability in AI development is a growing concern, with the environmental cost of data centers and computing resources coming under scrutiny. How is Lemurian Labs addressing the sustainability aspect of AI development, especially regarding reducing power consumption?
Sustainability has to do with more than just choice of hardware, it is a full system problem. A large reason for the high cost is because a lot of these compute clusters are underutilized relative to their peak capabilities. This turns out to be a software problem. We don’t have the right software for this new world. At Lemurian Labs, we’re committed to addressing this challenge by building a software stack that unlocks the hidden performance in existing hardware so that more work can be done in less energy, thereby bringing more sustainability to AI. But this is just the first step in bringing down the energy cost of AI, there is still a lot more that needs to be done.
The most interesting things almost always happen at the boundaries or intersection of fields. I think we will see radical innovation in cloud computing, datacenter infrastructure, computer architectures, and compilers. It is the convergence of them that will enable further progress in AI.
The framework we use at Lemurian is to understand changes in constraints and what technologies need to intersect to give us new capabilities. One in particular that we view as important is that software needs to be reimagined for a world where large scale heterogeneous computing is the norm. We see the need for better computer architectures and infrastructure, but their adoption is limited by the robustness of software. Existing software stacks limit the direction in which architectures are able to evolve, which imposes a limit on the kind of AI models that can take root.
The Lemurian Labs software stack will open up new opportunities for system design in the future which is ultimately our vision. In the shorter term we can change the economics of AI by giving greater utilization and throughput on existing hardware while making it easier for developers to train and deploy models, and making it less burdensome to adopt new alternative hardware architectures.
Finally, on a more personal note, as someone at the forefront of technological innovation, what motivates you to keep pushing the boundaries, and what advice would you give to young entrepreneurs aspiring to make a difference in the tech world?
Personally, I really enjoy big, hairy, hard problems that are perceived as impossible to solve. Those problems are solvable, but they require you to bend your mind a bit and break away from conventional wisdom. You’re rarely battling with physics, but you are going up against the status quo. However, I am not interested in solving it just because it’s interesting, it has to matter and hold the potential to make a difference in people’s lives, otherwise it’s just not worth doing. And that’s a worthy pursuit in my book.
Stay humble, stay hungry, stay curious, and embrace your failures as best as you can
Jay Dawani
As for young entrepreneurs, that’s hard, because I’m still a young entrepreneur and I’m still learning everyday. There is far more to know that I will likely ever be able to know. That said, the best way to overcome that is by surrounding yourself with people with diverse knowledge and backgrounds and skill sets because they will help you think differently, so you all get smarter together. Outside of that, stay humble, stay hungry, stay curious, and embrace your failures as best as you can. In failing, I have learned the most.
