Introduction

In the ever-evolving landscape of AI and machine learning, Google’s Gen AI Toolbox for Databases stands out. This open-source server enables developers to connect generative AI applications to enterprise databases, facilitating prompt-based querying and natural language processing (NLP). Whether you’re setting up your LLM on-premises using OLLAMA or leveraging providers like Gemini, Claude, or OpenAI, this toolbox offers a versatile and powerful solution. Lets explore it in detail.

Background

There was a requirement to perform series of tasks, involving generation of output files, such that the required throughput is achieved. These tasks involve database read operation, external API invocation and file i/o. Generally, benchmarking showed that executing them in sequential way was not helpful. What if asynchronous programming be used to perform this task.

So Lets Start.

Approach

Lets assume that this typical use case requires,

• fetching data from database for the purpose of merging placeholders in a Template and perform mail merge

Introduction

Artificial Intelligence, especially Large language models (LLMs) are all in high demand. Since OpenAI released ChatGPT, interest has gone up multi-fold. Since 2023, Powerful LLMs can be run on local machines. Local Large Language Models offer advantages in terms of data privacy and security and can be enriched using enterprise-specific data using Retrieval augmentation generation (RAG).Several tools exist that make it relatively easy to obtain, run and manage such models locally on our machines. Few examples are Ollama, Langchain, LocalAI.

Background

Ever encountered a scenario where REST API consumption works from tools like curl, Web Browser but not from Application. Lets dive in.

The requirement is as simple as consuming REST API from a Application over TLS.

Problem Statement

The REST API, to be consumed, is standard API interface which requires access over TLS. The client in this case is Windows 2016 server.

During Development, Windows 10 is used to develop and test the code. Later, the same is tested on a Windows 2016 Server. It is at this stage, it fails with cryptic Error “The request was aborted: Could not create SSL/TLS secure channel”. But it works fine with other tools like curl, PostMan or even from a Web Browser.

Background

A Client has E-commerce Application consisting of services aimed at specific domains of business functionality it serves. One of these services is responsible for accepting the order, authenticating it and forwarding it for further processing in terms of inventory checks, payment and so on. For Authentication, this service sends SMS to Customer’s Mobile number (and e-mail id) and customer is supposed to confirm this order placement by means of entering Code received in it. This code is valid for a short duration.

Introduction

While embarking on building any new server application, one of the key requirement is whether it needs durable, persistent storage of data (and in most cases, it does). This is followed by evaluating suitable data store. Likely evaluation criteria is Application’s Requirement (Tolerance for eventual consistency, High Availability etc.), Team’s familiarity, Costs, Tech. support availability and so on. In case of choices in relational databases, typical go to options are MySQL, PostgreSQL or even proprietary databases like Oracle , SQL Server. Seldom one considers SQLite for this purpose.

Introduction

We develop a piece of software with aim to fulfil specific business requirements in terms of resource usage, throughput, availability among others. Profiling and benchmarking are approaches that developer has in his/her arsenal to gain continuous feedback on whether a piece of code is behaving optimally and adhering to it’s objectives.

Lets look at what they mean,

• Profiling is defined as process aimed at understanding the behavior of a program. A profile result might be a table of time taken per function, as per this and this)
• Benchmarking measures the time for some whole operation. e.g. I/O operations per second under some workload. So the result is typically a single number, in either seconds or operations per second. Or a data set with results for different parameters, so you can graph it.. Refer this for more information. Also do check Benchmarking correctly is hard by Julia Evans.

Typically, Profiling is supported by most of the environments (either via IDEs like Visual Studio or through language itself [Like Go] has buil-in provision for the same while Benchmarking is typically performed on dedicated testing infrastructure.

Introduction

I have been reading excellent Database Reliability Engineering book and below are my notes from it.

• Key Incentive(s) for Automation

  • Elimination of Toil - Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows.

• Important System Characteristics

  • Latency, also known as response time, is a time-based measurement indicating how long it takes to receive a response from a request. It is best to measure this for end-to-end response from the customer rather than breaking it down component by component. This is customer-centric design and is crucial for any system that has customers, which is any system

Introduction

Suppose you have a distributed application running in production and it is based on Micro services/Service Oriented Architecture and have SLA of being “always on” (be available 24*7, barring deployments of course !!). In such cases, having proper monitoring of Application health in place is absolutely essential.

What if Monitoring is an afterthought (i.e. application is already in production) ? and that there is little apetite for additional components like (Visualization tools, specialized storage for logs/metrics/traces) for monitoring?

Introduction

One of the design considerations stressed upon by Jeffrey richter about APIs (Read more here) is that “API is expected to be stable over long period of time”. Recently,for a .NET based project, we decided to upgrade some of the ASMX (legacy SOAP based approach) based APIs and were immediately reminded by Customer(s) to avoid any kind of impact on existing users.

This means that upgrade must be done keeping in mind,