Fastly, a leading edge cloud platform, launches the Fastly AI Accelerator to improve performance and reduce costs of large language model applications. The tool aims to support developers in handling the growing demands of AI-driven applications by leveraging semantic caching and offering streamlined integration processes. This initiative reflects Fastly’s commitment to providing innovative solutions for rapid, secure, and efficient online experiences in the evolving digital landscape.
Fastly Unveils AI Accelerator Aimed at Enhancing Developer Experiences
San Francisco, June 13, 2024 — Fastly, Inc., a recognised name in global edge cloud platforms, has introduced the Fastly AI Accelerator, a tool aimed at optimising performance and cutting costs associated with large language model (LLM) applications. The launch underscores the company’s ongoing commitment to supporting developers with innovative solutions designed to create rapid, secure, and efficient online experiences.
In an industry where AI technologies and large language models are increasingly pivotal, Fastly’s AI Accelerator seeks to provide developers with a means to handle the growing demands of AI-driven applications. Stephen O’Grady, Principal Analyst at RedMonk, highlights the rising interest in medium and smaller models as alternatives due to their benefits in reducing costs, shortening training cycles, and operating on less powerful hardware.
Fastly’s new offering leverages semantic caching, a method that stores responses for repeated queries at the edge, thereby reducing the need for repeated API calls to AI providers. This technology is initially compatible with ChatGPT and will soon expand to support other prominent models. By caching frequently asked questions, the AI Accelerator can deliver quicker responses and cut down on costs associated with repeated queries.
“AI technologies generally and large language models specifically are aggressively reshaping the technology industry, and the way millions worldwide – developers included – work every day,” O’Grady observed. “There’s a lot of focus on the largest models. However, developers and enterprises alike are turning in large numbers to medium and smaller models. Whether it’s to lower costs, to shorten training cycles or to run on more limited hardware profiles, they’re an increasingly important option.”
Elaborating on the need for their new tool, Fastly’s Vice President of Developer Experience, Anil Dash, noted, “Fastly AI Accelerator gives developers exactly what they want, by making the experience of their favourite LLMs a lot faster and more efficient. This allows them to focus on innovating their unique applications and keeping users satisfied.”
The integration process for developers is streamlined; updating an application to utilise Fastly AI Accelerator typically involves modifying a single line of code to a new API endpoint. This adjustment facilitates the implementation of semantic caching without significant changes to the existing codebase.
To further support the development community, Fastly is expanding its free account tier. This initiative aims to ease the onboarding process for new developers, offering a suite of tools including a content delivery network (CDN), memory and storage allocations, uncapped redirects, and various security features. This effort is intended to provide additional resources for developers to set up websites, applications, or services efficiently.
The Fastly AI Accelerator represents just one part of the company’s broader strategy to enhance web performance and security through edge computing. The approach aims to deliver faster and safer user experiences on a global scale, a move in line with the company’s commitment to supporting significant digital entities such as Reddit, Stripe, and Universal Music Group.
As industry dynamics shift and the demand for more efficient AI solutions grows, tools like the AI Accelerator are likely to become valuable assets for developers seeking to optimise their applications and reduce operational costs. The introduction of this tool comes at a time when AI-driven technology continues to rapidly evolve, necessitating innovative approaches to handle the associated computational challenges.
While Fastly’s ambitions with the AI Accelerator are clear, the long-term impact of this tool will depend on its reception among developers and its effectiveness in real-world applications. The company remains optimistic, positioning itself as a key player in the ongoing digital transformation landscape.
