Columbus

At the start of this is a simple and yet perennial CEO question: “My gross-margin got hammered. Why did my gross margin decline, and what actions should I take?”

The complexity of modern enterprises means the answer is seldom straightforward, spanning across various domains from supply chain and production to sales and customer service.

GenAI in Modern Business

As I write the capabilities of Generative AI are pushing the boundaries of what’s possible, extending its reach beyond simple chat functionalities to encompass problem-solving, personalization, and automation. It’s these capabilities that will drive actual change in an enterprise, more so than simple chat.

Today’s enterprise systems are often large, unwieldy, siloed. Data within these systems is typically unstructured and scattered. And of different character in each system, making inter-operability difficult. A few years ago a first band-aid was developed: Robotic Process Automation (RPA). And yet — I simplify — if the precise robotic process can’t be followed RPA fails.

The key reason: ambiguity in enterprise data sets. That reason can be as simple as say an underlying product definition change in one system. This ambiguity can lead to inefficiencies and errors, further complicating the task of understanding and acting upon data insights.

The Future: An Insight & Autonomization Layer

What if the solution lies in enhancing communication not just between people but between machines themselves? Imagine a scenario where an enterprise system is able to “chat” with another system to solve a business challenge?

The combination of a number of technologies into a RAG+++ stack (everyone knows what a RAG stack is, we’ll expand over the next weeks what a RAG+++ stack is) allows for the first time to deal with this enterprise ambiguity.

A generic request of the type “I need to increase output capacity by 20%” is a multi-faceted and consequently multi-threaded help desk request. Watch this following video (https://go.squirro.com/agents ) to see how a clever combination of men and machine solves that using our Autonomization layer.

It’s the equivalent of a Air Traffic Controller (ATC). She is not flying the planes, she’s simply orchestrating the operations, focusing on what she does best — decision making in ambiguous situations while offloading the work to other agents in that complex system called airline operations. This exact same scenario becomes now possible for any business operations.

The result? A system where insights and actions are not just data-driven but dynamically responsive to the shifting landscapes of business needs. This integration of an Insight & Autonomization Layer across companies promises a more interconnected and intelligent enterprise. This layer will act as the central nervous system of the business, interpreting vast and varied data streams and converting them into actionable insights and automated responses.

Conclusion

For businesses, this fusion of various technologies into a RAG+++ stack represents not merely a tool but a transformative force that can enhance productivity across all crucial areas of operation.

The journey towards an autonomized, insight-driven enterprise is not without its challenges, but the potential rewards — increased efficiency, better decision-making, and enhanced productivity — are too significant to ignore.

Testing and Implementing RAG in Your Enterprise

If our strategies surrounding Generative AI and RAG have sparked your curiosity, there’s more to uncover. Here are our top three resources for C-level leadership and senior IT professionals:

• This whitepaper is the perfect introduction to RAG and its capabilities in an enterprise: Discover why you need Retrieval Augmented Generation in 2024
• Listen to our experts delve into RAG technology: Squirro’s RAG Revolution
• Get your copy of The Successful Enterprise Search & Generative AI Buyer and transform your data into actionable insights with Squirro — a trusted solution provider for leading organizations, including central banks.

Since December 2022, we at Squirro have been pioneering RAG technology to empower enterprises across various industries. Would you like to see a tailored demonstration and solutions leveraging RAG technology?

We offer an in-depth look at how it enhances enterprise use cases with Squirro — a trusted solution provider for leading organizations, including central banks.

At its core, automation enables systems, operations, or mechanisms to function independently without direct human intervention. Pursuing this concept has shaped technologies, transformed industries and economies, and left a significant mark on the labor force. From the early gears of industrial machinery to sophisticated algorithms, the journey of Automation reflects a relentless pursuit of efficiency, safety, and innovation

Strategic automation applications permeate various facets of our lives, from our smart home assistants regulating temperature in our homes to chatbots on websites, robot manufacturing, and the accelerated realm of autonomous vehicles.

In today’s landscape centered around Intelligent Automation, we are at the advent of the AI-powered evolution spanning all data-centric sectors. As every modern industry pivots to become an information-driven, rapid data processing enterprise, tomorrow’s digital landscape becomes ripe for hyper-automation. To gauge the full impact of the digital revolution, we must draw insights from the history of automation in order to harness a viable automation strategy.

Industrial Revolution: The First Wave of Automation

Industrialists sowed the seeds of Automation during the Industrial Revolution, a period marked by the transition to new manufacturing processes and the introduction of mechanical production facilities. The invention of machines that could perform tasks previously done by hand, such as the spinning jenny and the steam engine, initiated the first wave of automation. These machines didn’t operate entirely without human oversight, but they significantly reduced the labor required to produce goods, laying the groundwork for the automation concept.

The 20th Century: The Rise of Automation, Robotics and Computing

With a jump to the 20th century, we witnessed the rapid advancement of technology and the rise of automation, particularly in manufacturing. In the early 1900s, Henry Ford popularized and pioneered the development of the assembly line, which paved the way for the mass production of goods with minimal human intervention.

This era also saw the advent of computer technology and robotics, pushing the boundaries of what could be automated much further. The introduction of programmable logic controllers (PLCs) in the 1960s marked a significant milestone, enabling the automation of industrial processes with unprecedented precision and flexibility.

The Digital Age: Information Technology and AI-powered Automation

Furthermore, the advent of the digital age accelerated the evolution of automation exponentially, with information technology and artificial intelligence (AI) playing pivotal roles. The Internet and advancements in computing power expanded the possibilities of autonomous systems beyond physical tasks to include data processing, analysis, and decision-making.

AI and machine learning algorithms have enabled the creation of systems that can learn from data, adapt to new situations, and make decisions with minimal human guidance.

The 21st Century: The Intelligent Automation Platform Scaling Automation at Speed

Today, Intelligent Automation is permeating every facet of our daily lives. From autonomous vehicles (self-driving cars) navigating our roads to smart home devices managing our living environments. Drones conduct aerial surveys, robots perform surgeries with precision beyond human capability, and AI-driven platforms personalize our digital experiences.

The Internet of Things marks a significant milestone as integrating IoT devices further enhances connectivity, reshaping communication and engineering landscapes. We witness more tech innovation based on this trend: becoming more autonomous and interconnected, where a thriving digital ecosystem is formed. Cloud migration also accelerates digital transformation, empowering organizations with real-time analytics. The convergence of AI and machine learning enables end-to-end automation, making a profound turning point in how we harness our data technology.

The Future of Automated Intelligence: AI-powered Hyper Automation

Finally, we find ourselves at a critical intersection: navigating the way forward at a decisive moment as emergent technologies continue to deliver exponential growth in proportion to imminent risks. Startling advances in artificial intelligence have propelled pioneers like Mustafa Suleyman, to the urge of “Containment for AI” in public conversation, referring to the 20th Century’s Cold War strategy to a New Threat.

For instance, the future of automation promises even greater integration of autonomous technologies into our daily lives, potentially leading to smart cities, fully automated transportation systems, and AI-driven governance. Thus, this future arrives with unprecedented challenges, posing ethical, legal, and social challenges that must be navigated carefully.

As we respond to the new wave of innovation and continue advancing the frontiers of automation and beyond, human oversight remains an essential part of the collective evolution.

Conclusion: Implementing AI-Driven Automation Solutions Tailored For Success

The history and evolution of automation reflect a fascinating journey from mechanical Automation to AI-driven automation, unpacking astonishing technological discoveries. As we stand on the brink of further groundbreaking developments, automation will undeniably continue to shape the future growth trajectory of any industry.

As a must-have feature in any company’s AI strategy, the economic incentive for integrating AI-powered automation is evident. Yet, the critical determinant lies in implementing within an enterprise setting to harness its ultimate benefit.

Gain Expert Guidance on Enterprise Solutions:

C-level leaders must proactively implement enterprise-grade solutions to elevate operations and ensure the secure deployment of intelligent automation technologies.

For Seamless Implementation of AI Enterprise Solutions tailored to your unique needs and timely guidance on navigating changes in your industry, connect with Our Specialists and schedule a free demo – book your demo here.

Our Generative AI Buyer’s Guide is designed for decision-makers looking to advance with AI-powered Enterprise Solutions tailored to success:

Drive Your Enterprise Forward: Access Buying Guidance from a Leading Solutions Provider and get the full perspective here: “The Successful Enterprise Search and Generative AI Buyer.“

Last year Large Language Models (LLMs) took center stage, signaling a new era in technology and innovation. This transformative wave wasn’t just a blip on the radar for tech aficionados; it became a pivotal focus for Chief Information Officers and Chief Technology Officers across industries.

The initial enthusiasm led many organizations to start building their own LLMs based solutions, driven by the promise of revolutionizing how we interact with data and digital systems. However, the journey from excitement to execution unveiled a complex reality: while the first 80% of development seemed straightforward, the remaining 20%—involving data ingestion at scale, indexing, entitlements, and integration into existing enterprise systems—is a formidable challenge, consuming 80% of the effort. And while a must to make it work in an enterprise setup it’s not “exciting”, hence not much in focus with consequences down the line, when trying to go full “production”.

As we stepped into 2024, organizations had already earmarked significant budgets for AI, ready to invest heavily in what was perceived as the next frontier of competitive advantage. Nvidia’s stock prices soared with each new ChatGPT announcement, a testament to the market’s bullish outlook on AI’s potential.

Early pilots showcase the innovative capabilities of LLMs, yet the chasm between a successful pilot and operational deployment will become increasingly apparent. Contrary to – my view – popular belief, chat functionalities, while impressive, are not the ultimate application of LLMs. The real value is in automating internal processes, a more challenging endeavor but one with potentially transformative outcomes. By year end the initial euphoria will be coupled with scrutiny over the tangible returns on these hefty AI investments.

Looking ahead to 2025, a reality check will be on the horizon. The initial surge in budgets is likely to face cuts due to underwhelming results in some areas. This recalibration will see some organizations doubling down. The “it takes a tad longer” argument will be used to invest even more in hopes of unlocking the true potential of LLMs. Others will begin to question the initial hype surrounding chat functionalities. Market corrections are to be expected, with significant players like Nvidia and OpenAI likely to experience downturns as a consequence of changed market sentiment.

However, this period of disillusionment is not the end but rather a critical juncture. It mirrors the Gartner Hype Cycle’s trajectory, from inflated expectations to a trough of disillusionment, before climbing towards a plateau of productivity.

The true value of LLMs, as we will begin to see more clearly by mid-2025 and beyond, lies not in the flashy chat interfaces but in the less visible, yet far more impactful, automation of complex processes. This phase will usher in real productivity leaps and unveil new areas of progress, marking the transition from overhyped short-term expectations to meaningful, long-term outcomes.

In essence, the journey of LLMs from 2023 to 2025 is a microcosm of the broader tech innovation lifecycle. It underscores the importance of tempering initial excitement with a measured understanding of the challenges and complexities involved. For businesses, the key to leveraging LLMs effectively lies in realistic goal-setting, patience, and a strategic approach to integrating these technologies into their operational fabric. As we navigate this evolving landscape, the lessons learned will not only refine our approach to LLMs but also shape how we adopt and adapt to new technologies in the future.

As we navigate the ever-evolving AI landscape, bold predictions about its immediate and long-term impacts have become increasingly commonplace. The start of the year, in particular, brings forth a flurry of these prediction posts.

While these forecasts often swing between utopian and dystopian extremes, let’s take an “anti-prediction” approach here. My stance is simple: we tend to overestimate AI’s short-term effects while significantly underestimating its long-term implications.

The Short-Term Overestimation

In the short term, there’s a tendency to expect rapid, revolutionary changes following the introduction of new AI technologies. This anticipation is fueled by a mix of media hype, speculative fiction, and the tech industry’s penchant for dramatic announcements. However, the reality often falls short of these expectations. The reasons are multifaceted:

Technical Challenges

AI development often encounters unforeseen technical hurdles that delay progress. 
For instance, in an enterprise setup, integrating data residing in arcane systems is more complex than a simple tap and click.

Adoption Lag

The integration of AI into society and industry takes time, as it requires changes in infrastructure, skills, and societal attitudes. Many pilots have commenced in the last 12 months, but the leap from this to actual product-grade rollouts is significant. (We are working on several. If you are interested in how we got there – read our case studies and get in touch!)

Regulatory and Ethical Constraints

Legal and ethical considerations can slow down the deployment of AI technologies, particularly in sensitive areas like healthcare, autonomous vehicles and generally any regulated industry.

This short-term overestimation often leads to a cycle of hype and disillusionment, often referred to as the “AI hype cycle.”

The Long-Term Underestimation

Conversely, the anti-prediction perspective holds that we consistently underestimate the long-term impact of AI,  not just in terms of technological capabilities but also in how AI will reshape our social, economic, and political landscapes. Several factors contribute to this underestimation:

Cumulative Advances

Incremental progress in AI, while seemingly modest in the short term, can lead to profound changes over longer periods. Think back to the advent of the internet – I vividly remember a telco operator telling us even after 2000, that the demand for mobile internet would be limited. 

Convergence with Other Technologies

The interaction of AI with other emerging technologies like biotechnology, nanotechnology, and quantum computing could lead to synergistic effects that are hard to predict.

Societal Transformation

AI’s long-term impact extends beyond technology, potentially altering the very fabric of society – from the way we work to the nature of human relationships and the structure of governments. For instance, the way we apply for jobs will be altered forever.

Navigating the AI Future

Adopting an anti-prediction lens requires a balanced view of AI’s potential. It calls for caution against overhyped short-term gains while maintaining an open mind about the transformative possibilities in the long run. This approach also emphasizes the importance of ongoing ethical and regulatory discussions to ensure that AI develops in a way that benefits humanity as a whole.

In conclusion, I hope this perspective offers another and more realistic approach to understanding AI’s future. AI’s true impact might be different and far more profound than our current predictions suggest. Yet the way this develops might be different than many predictions want you to believe. 

Welcome to another year of transformative change and surprising stabilization and consolidation

Following the pattern of massive change in 2022 and the linear progression of these changes in 2023, the year 2024 is likely to be characterized by both transformative change and stabilization and consolidation. Here’s what this could entail:

On transformative change:

The U.S. political landscape: Post-Trump (yes he’ll be a man of the past in 2024; and that sooner than many think), the U.S. political scene moves towards a new era of leadership. The Republican Party, seeking to redefine itself, may embrace more moderate, reality-based policies, distancing itself from past extremist tendencies. Meanwhile, Democrats will face their own challenges in maintaining unity and addressing pressing domestic issues.

AI & the tech industry’s continued evolution: AI will continue its massive impact trajectory. I’ve seen enough of the economic impact to see entire industries turned upside down. A few things will go wrong, actually badly wrong. More things will go right. The tech industry, will hopefully see a shift towards more sustainable and socially responsible behavior, with a focus on data privacy and ethical AI. Europe’s stringent regulations start to influence global tech norms, emphasizing accountability and consumer protection.

China’s Global Role: China is a volatile and capricious actor. The economy is – if you look under the hood – in dire straits. Difficult to foresee on what level the act rational (and adventures across a straight may be ‘rational’ depending on the local situation in Beijing)

Russia’s Ongoing Challenges: The Ukraine conflict continues to weigh heavily on Russia (see below). The country, still grappling with internal strife, will seek to redefine its global position, but with a more pragmatic and internally-focused approach. This period of introspection might lead to unexpected reforms and a gradual shift away from aggressive foreign policies.

Stabilisation:

Consolidation of Changes: The changes initiated in 2022 and developed in 2023 will begin to bear fruit. This means that policies, innovations, and societal shifts started in the previous years will start to become more entrenched and widely accepted, leading to a more predictable and secure global environment.

Economic Adjustment: Economies around the world will continue to adapt to the new realities post-2022 changes. This will involve shifts in global trade patterns, a stronger focus on sustainable business practices, and the continued evolution of the job market in response to technological advancements. Though a bad economic set up is not excluded (see US repo and credit markets…)

Ukraine’s Rebuilding and European Integration: Following its victory, Ukraine embarks on a massive rebuilding effort, gaining significant support from European nations. This cooperation accelerates Ukraine’s integration into European frameworks, further solidifying the continent’s unity and resilience.

For us:

Our company saw an inflection point end 2022. Now we start to see the momentum of this change accelerating. 2024 marks a period of growth, innovation, and increased market presence, reflecting the successful navigation of recent challenges.

2024is a year where cautious optimism blends with practical action, setting the stage for a brighter future.

In previous discussions, we extensively explored the multifaceted world of Retrieval-Augmented Generation (RAG) – a paradigm that synergistically combines the prowess of information retrieval with natural language generation to produce more informed and contextually rich responses. The series delved deep into the theoretical aspects and inner workings of this compelling technology, demystifying how it harnesses the confluence of knowledge retrieval and text generation to elevate the capabilities of language models.

Today, we pivot our focus towards the real-world, unfolding the practical use cases of Retrieval-Augmented Generation. In this article, we will elucidate how RAG is being deployed across various domains, providing solutions, enhancing efficiencies, and creating value by solving intricate problems and generating high-quality, contextually-relevant content.

For any Website

First up: “Chatify” your website. With a setup time of just 30 minutes, SquirroGPT empowers you to elevate your website’s user experience, providing an interactive, engaging, and responsive chat interface. Whether it’s addressing user queries, offering support, or facilitating seamless navigation, SquirroGPT is equipped to handle it all, ensuring your visitors find exactly what they’re looking for with ease and convenience.

For your Company Data

In today’s data-driven business environment, having seamless access to company data is paramount. Chatifying your company data means integrating conversational AI and chat functionalities into your data management systems, allowing users to interact with, analyze, and understand complex data through simple conversational queries. This not only democratizes data access across various departments but also empowers team members to derive insights swiftly and make informed decisions. By adopting a chatified approach to company data, businesses can unlock unparalleled efficiencies, reduce the time spent on data analysis, and foster a more informed and agile organizational culture.

Automate RFP/RFI Responses

Responding to Requests for Proposals (RFPs) or Requests for Information (RFI) can be a daunting and time-consuming task, requiring meticulous attention to detail and extensive knowledge of your company’s offerings. Enter SquirroGPT designed to revolutionize the way you handle RFPs/RFIs. SquirroGPT can quickly and accurately generate comprehensive response sheets to even the most complex RFPs/RFIs, ensuring your proposals are coherent, compelling, and to the point. By leveraging SquirroGPT, companies can not only expedite the RFP/RFI answering process but also significantly enhance the quality and precision of their responses, thereby increasing the chances of securing valuable contracts.

GPT-Enabled Fashion Recommendations

Dive into the future of style with GPT-Enabled Fashion Recommendations, a sophisticated blend of fashion sense and SquirroGPT designed to change your shopping experience. By interpreting user preferences, browsing history, and current fashion trends, SquirroGPT generates personalized fashion advice and outfit recommendations. Whether you’re in search of a new look for a marriage in the south of France or the perfect accessory to complete your ensemble, SquirroGPT provides to the point recommendations directly linked into the relevant eCommerce outlets.

Conclusion

This is just the start. Over the next few months a number of additional use cases will show up transforming online habits for good. And all of it is available today with our SquirroGPT solution bringing the use cases introduced above to life. Try it for yourself.

It has been claimed, that to create the ultimate Retrieval Augmented Generation (RAG) Stack, all one needs is a plain vector database, a touch of LangChain, and a smattering of OpenAI. Here are the 10 essential considerations when delving into constructing a RAG from a business perspective*.

1. Data Access and Life Cycle Management:

• How will you effectively manage the entire lifecycle of the information, from acquisition to deletion. This includes connecting to different enterprise data sources and collecting extensive, varied data swiftly and accurately.
• The next step is ensuring that every piece of data is efficiently processed, enriched, readily available, and eventually archived or deleted when no longer needed. This involves constant monitoring, updates, and maintenance, to meet data integrity, security, business needs and compliance standards.

2. Data Indexing & Hybrid Search

• Data indexing and the operation of hybrid search within a large-scale enterprise context are complex, ongoing endeavors that extend beyond initial setup. It involves creating structured, searchable representations of vast datasets, allowing efficient and precise retrieval of information. A hybrid search amplifies this complexity by combining different search methodologies to deliver more comprehensive and relevant results.
• Maintaining such a large-scale index over time is non-trivial; it necessitates continuous updates and refinement to ensure the accuracy and relevance of the retrieved data, reflecting the most current state of information.

3. Enterprise Security & Access Control at Scale

• Enterprise security, especially respecting complex Access Control Lists (ACL), is a crucial aspect of data management and system interactions in any organization. Proper implementation of ACLs is paramount to ensure that every user gains access only to the resources they are permitted to use, aligning with their role and responsibilities within the organization.
• The task is not just about setting up strict access controls but maintaining and updating them in a dynamically changing index (see previous point) to adapt to evolving organizational structures and roles.
• Any cloud system interaction with an enterprise opens attack vectors. The setup of such a system needs to be well thought through. We’ll deal with that in a separate post (and keep it short by pointing to our ISO27001 certification)

4. Chat User Interface

• Building an adaptable chat interface is relatively straight forward. Integrating it with value-add services / agents is where the real challenge lies.
• Recommendations, next-best-action tasks, (semi) autonomous automation are more difficult to implement and integrate as they require a whole lot more of scaffolding behind the scenes.

5. Comprehensive System Interaction

• Developing a system that integrates interactions with indices, Large Language Models (LLM), and performs entailment checks of answers is a multidimensional challenge.
• Building a comprehensive Information Retrieval (IR) Stack is an intricate endeavor. It demands meticulous consideration of the types and sources of data to be incorporated, aiming to achieve a good understanding of the information involved. By accurately accounting for the diversity and nature of data, the system can significantly enhance the quality and relevance of the generated results, providing more precise and contextualized responses.
• In essence, the initial simplicity masks the underlying complexity and sophistication required to orchestrate coherent interactions among various components effectively.

6. Prompt Engineering

• Creating an effective prompt service to facilitate interaction with a Large Language Model (LLM) requires a nuanced approach. It involves crafting prompts that are concise, clear, and contextually rich, to elicit accurate and relevant responses from the LLM.
• The prompts should be designed considering the model’s capabilities and limitations, focusing on clarity and specificity to avoid ambiguous or generalized queries that might result in imprecise or irrelevant answers.
• Additionally, integrating adaptive mechanisms can help refine prompts based on real-time interactions and feedback, enhancing the quality of the dialogue between the user and the LLM. Balancing specificity with adaptability is key in optimizing the efficacy of prompt services interacting with LLMs.

7. Chain of Reasoning

• The implementation of a chain of reasoning represents a sophisticated progression in the journey of developing intelligent systems. It transcends the rudimentary interaction levels, enabling systems to engage in continuous, meaningful dialogue by logically connecting multiple pieces of information.
• This involves not just processing individual queries but understanding and relating pieces of information in a coherent, logical sequence, allowing the system to provide more nuanced, contextual, and insightful responses. It represents a shift from isolated retrieval and response mechanisms to a more integrated, coherent interaction model, where the system can comprehend, relate, and extrapolate information across multiple interaction points, paving the way for more advanced, context-aware conversational experiences.

8. Enterprise Integration

• Integrating a RAG into an existing enterprise setup is the next step, and often more intricate than anticipated, especially when striving to avoid inducing ‘yet another dashboard’ fatigue once the initial novelty diminishes.
• Such integration is not about just plugging in a new component; it demands comprehensive Software Development Kits (SDKs) and thorough interoperability assessments to ensure seamless interaction within the existing technological ecosystem.
• While APIs can offer pathways for integration, relying solely on them is insufficient. They are part of the solution, not the complete answer, serving as components. Achieving seamless integration is about harmonizing new capabilities with established systems, requiring meticulous planning, execution, and ongoing refinement.

9. Continuous Operation

• The continuous operation of advanced systems demands attention to updates, upgrades, and enhancements to sustain optimal performance and adapt to evolving needs. This ongoing endeavor is not only about maintaining the system but also about refining and advancing it continuously.
• A notable point is that the talents who develop such systems are often not the ones who manage them in the long run. The industry is dynamic, and the risk of skilled developers being recruited away is ever-present.

10. Cost Considerations

• Cost considerations are paramount when scaling technologies like LLMs within a company. Early trials, while revealing, often expose vast amounts of data to LLMs, and when these are scaled, the costs escalate significantly. LLM operations tend to be 10-20x more expensive than classic retrievals. Key is the setup a system with a good balance between both
• Operating sophisticated technologies at scale over time, especially with little prior experience, can lead to painful lessons learned in navigating diverse environments and addressing unforeseen challenges. Furthermore, the financial implications extend beyond operational costs to include maintenance, updates, employee training, and support.

Conclusion

Building a perfect RAG stack is not as simple as mixing a few ingredients. It is a meticulous process, riddled with complexities and steep learning curves. It involves considering aspects from data management to continuous operation, enterprise integration, costs, and beyond.

For readers I put the 10 points into an easy-to-use checklist (pdf download).

* We will delve into a technical discussion on the challenges of building RAG at scale in one of the forthcoming posts.

** We’ve been in this business for very long. Gartner thinks of us as the Visionary in the space. Not convinced about the build or buy case? Here’s a paper. And our SquirroGPT solution brings the points made above to life. Try it for yourself.

Last Sunday, I delved into the often misconceived notion of a seemingly easy recipe for a Retrieval Augmented Generation (RAG) solution. Many seem to believe it’s as straightforward as combining a vector database, a sprinkle of LangChain, and a dash of OpenAI. But the reality is far more intricate. Today, we shall investigate a more effective approach that goes beyond these oversimplified ideas.

To illustrate this, let’s revisit the example I mentioned about the bank. This takes us back to the turn of the century when a renowned bank enthusiastically jumped on the Internet bandwagon and started building their own Content Management System, imagining it would be a transformative asset.

However, it soon dawned on them that multi-language content rendering isn’t really a unique selling proposition (USP) of their bank, nor actually of any bank. In the dynamic world of banking and finance, being able to render content in multiple languages is undoubtedly valuable, but it’s hardly a standout feature. And with its own intricacies it isn’t easy either. A multi-million investment, they stopped the ill-fated venture a couple of years later.

What’s the Alternative?

Instead of fixating on what every other organization might be doing or adopting, the mantra should be to channel energy into one’s unique strengths. The answer lies in a two-pronged strategy:

• Leverage External Expertise: Partner with external professionals and vendors who specialize in the core platform. This should be their domain of expertise. Instead of reinventing the wheel, it makes business sense to let the these folks handle what they’re best at – they are hopefully doing it for a long time. Not sure if they are experts? Organizations like Gartner and Forrester are a good starting point. With respect to the RAG dichotomy I suggest to consult e.g. the Gartner Magic Quadrant for Insights Engines. The vendors mentioned know what information retrieval is – better than most upstarts. We’ve been at this for a long time*.
• Hone in on Internal Specialties: Focus internal resources on what isn’t easily available in the market. For our aforementioned bank, this meant developing a cutting-edge portal that provided users with a 15-minute delayed feed of stock market prices and quotes. This might seem commonplace now, but at the time, around 2000, it was groundbreaking. Here’s the kicker: This was a service no content management vendor offered. Yet, the bank possessed the necessary expertise. They understood the user requirements for such a service, had access to the crucial data, and went beyond to make this vision a reality. Notably, this portal still stands today, testament to its utility and the bank’s foresight.

Modern-Day Application

Drawing a parallel to the present day, the principle remains unaltered. Businesses should consider the offerings of platform vendors* and should focus their actual efforts on what sets them apart. A modern-day setup could for instance be:

• Imagine a company selling transportation tickets. Many customer requests for a simple ticket from A to B are already handled well. Their online purchasing process is reasonably straight forward and easy. But here and there a customer needs a more detailed answer or simply wants a trip recommendation. Obvious that a GPT based solution might come in handy.
• Let a vendor like us handle the platform / GPT side of things. Ingestion of data, handling of the full data life cycle process, user authentication and authorization, the actual user interface, and more.
• The company should focus its efforts to “GPT enable” the ticket vending process. This is not something any vendor will provide. For starters because none has access to said system.
• The result: Users can interact with the chatbot to quickly find events, compare ticket prices, and get recommendations. It can answer complex queries, handle multiple event searches, and even assist in transaction processes. This not only improves user experience but also streamlines ticket purchases. Additionally, utilizing the AI’s capabilities, it can predict user preferences based on past interactions, ensuring personalized event suggestions, making ticket buying more intuitive and user-centric. And yes, the same recipe may be applied to any business.

Concluding Thoughts

The lure of the latest technological innovations is strong, and it’s easy to get swept away by the tide. Yet, while integrating solutions like RAG can be valuable, it’s imperative for businesses to recognize what truly sets them apart. Sometimes, the real differentiator might not be a flashy new AI tool but a service or solution that caters to the unique needs of their clientele.

In essence, while the world rushes towards adopting every new technological innovation, it might be more prudent to take a step back, evaluate what truly matters, and invest in those areas. After all, in the vast sea of technology, it’s the unique solutions that stand out and stand the test of time.

* It’s vital to recognize that it’s more intricate than simply merging some Azure Services with Microsoft-provisioned OpenAI access A glaring challenge that comes to mind is the aspect of data integration and re-integration into the existing enterprise landscape. Why this is important? No company runs exclusively on Microsoft products. Thus, relying solely on their offerings can be limiting, especially when data integration extends beyond their product range.

** We’ve been in this business for very long. Gartner thinks of us as the Visionary in the space. Not convinced about the build or buy case? Here’s a paper. And our SquirroGPT solution brings the points made above to life. Try it for yourself.

*** Wondering which bank it is? Ping me. 😉

In the grand tradition of the tech industry, I currently see a number of folks jumping onto the latest bandwagon without a seatbelt. And a number of riders are about to fall into the trap of believing that the latest and greatest tools are the perfect fit for their enterprise needs.

Some are now claiming that a simple vector database, a sprinkle of LangChain, and a dash of OpenAI are the magic recipe for a perfect Retrieval Augmented Generation (RAG) Stack.

Sure, let’s throw in a unicorn and a pot of gold at the end of the rainbow while we’re at it. I’ve seen this movie. Back in the late nineties many self-respecting CIOs and CTOs of Forbes 1’000 companies started to build their own Content Management Systems.

I remember a very large bank to join the bandwagon, too, only to realize that multi-language content rendering is not a bank USP. A decade later we were back at the same game: The eCommerce wave produced a number of large scale fails when retailer CIOs discovered that for a successful web shop you need more than a digital shopping basket.

So here we go again: Just because it’s new and glitzy doesn’t mean it’s golden. A vector database? Sure, it’s a robust system for certain tasks. LangChain? Stellar at simplyfing the cobbling together of such solutions. OpenAI? Undeniably, a giant in text generation. But slap them together and expect enterprise magic? That’s like expecting a random mix of haute couture pieces to make you the next fashion icon. Good luck with that. 

The Misconception of Synergy

Just because individual components excel in their domains doesn’t necessarily mean their combined utility will take you ever beyond 80% of the road. And it’s the last 20% of the road that are the hardest. Outlining just a few of those issues:

• While vector databases excel at retrieving relevant data based on similarities, much of the retrieval has to do with keywords. Not something an out-of-the-box vector database is good at. The way forward will be hybrid search.
• Additionally, the accuracy depends largely on the quality of embeddings and the algorithm’s ability to discern subtle differences. Retrieving a slightly incorrect piece of information can drastically affect the output’s quality. To deal with that you need a lot of experience in classic information retrieval (IR).
• Users might pose queries with multiple valid interpretations. Catering to such ambiguity and ensuring the retrieved knowledge aligns with the user’s intended context can be a significant challenge. You need an entire query parsing and user profiling / scoring setup.
• On top of that most foundational models are trained on vast swaths of internet text, making them susceptible to inherent biases present in the data. Merging retrieved information with such models can sometimes inadvertently perpetuate or even amplify biases.
• In addition, no foundational model will ever understand the finer points of an enterprise’s specific language (e.g. non-public product catalogue). You need a refined IR and eventually graph approach to get to the required precision levels for enterprise use.
• The capability to connect disparate databases can be both a boon and a bane. While it allows for broad knowledge access, ensuring the cohesiveness and consistency of data from different sources can be challenging.
• Which leads to a few other challenges such as data lifecycle, enterprise security, data access control, scaling of such solutions, user interfaces and integration into the existing enterprise landscape, total cost of operation aspects, etc. E.g., as the system evolves and the underlying databases grow, ensuring efficient retrieval without compromising on speed and accuracy becomes challenging.

Conclusion

A generic approach may work and get you 80% of the way. The last 20% to perfection encapsulates the complexities outlined above. It involves iterative refinement, extensive validation, and often, domain-specific adaptations. It’s the nuanced challenges that make the journey to 100% a demanding endeavor.

PS:

We’ve been in this business for very long. Gartner thinks of us as the Visionary in the space. So we got a few insights about build versus buy, shared here. And our SquirroGPT solution brings the points made above to life. Try it for yourself.

In today’s saturated marketplace, there is a cacophony of voices and solutions. Navigating this noise demands differentiation that delivers value. SquirroGPT offers exactly that. Here’s what sets the solution apart:

Three Pillars of Excellence:

At the core of the offering is the concept of Retrieval Augmented LLMs or Retrieval Augmented Generation (RAG) embedded in the solution:

• Evidence-based Results: SquirroGPT is unique in its promise of zero hallucinations. Every piece of information it generates is traceable to a source, ensuring credibility.
• Personalization with Your Own Data: The ability to connect proprietary data sets ensures that the insights you receive are tailored to your business needs. It’s not just an answer; it’s your answer.
• Uncompromising Security: ISO certified, with a fully built out enterprise search stack, SquirroGPT prioritizes the security of enterprise data, which includes fine-grained access level control.

Diving Deeper: The Enterprise Advantage

While many may echo similar sentiments, here’s what sets SquirroGPT apart in the enterprise context:

• Highlighting: Rapidly pinpoint critical data with highlighted passages within the original sources.
• Expand to Enterprise Search: This feature transcends mere text generation, offering powerful search capabilities across enterprise data. Coupled with personalized content recommendations, it transforms how businesses access and utilize information.
• Enterprise-Grade Integration: SquirroGPT’s versatile connectors allow easy integration with existing enterprise tools and platforms.
• Stringent Access Control: Through ACLs, access to data is meticulously managed, reinforcing data security.
• Mastering Data Life Cycle**: It’s not just about using data; it’s about managing it. SquirroGPT champions the complete data life cycle, ensuring that every piece of information is current, accurate, and auditable.
• Seamless Integration with Existing Workbenches: Whether it’s Salesforce, SharePoint, or any other platform, SquirroGPT augments them without the need for overhauls.
• Cost-Effective Excellence: With the innovative Retrieval Augmented LLMs / RAG approach, SquirroGPT optimizes both performance and cost. This means better outcomes without stretching budgets. (As a side note: LLMs are expensive and on most search based operations going solo >10x to 20x more expensive than the RAG approach)
• Graph-Enabled Capabilities: Navigating data becomes an enhanced experience with SquirroGPT’s graph-enabled features, enabling more contextually rich and swift responses.
• Diverse Applications on a Unified Platform: Beyond text generation, SquirroGPT empowers businesses with features like Summarization and Automation, making it a versatile solution for varied challenges.
• Promoting Collaboration: By understanding user profiles and patterns, Squirro streamlines information sharing and discovery, enhancing collaboration across teams.

In summary, SquirroGPT’s unique blend of features, integration capabilities, and cost-effectiveness makes it the go-to choice for enterprises seeking a superior GPT solution. So, if you’re in the market for a solution that punctures the hype balloon with genuine value, talk to us.

Oh and you can try yourself: https://start.squirro.com