Thursday, October 2, 2025

What is Google AgentSpace?

 AgentSpace is a dedicated, enterprise-grade platform designed by Google (often integrated within Vertex AI) for the complete lifecycle management of complex, autonomous AI Agents.


It moves AI Agents—which are programs built on Large Language Models (LLMs) like Gemini that can reason, plan, and use external tools/APIs—from research prototypes into reliable, scalable, and governed business solutions.


Think of AgentSpace as the operating system or orchestration layer for your organization's fleet of AI assistants. It provides the tooling necessary to manage the complexity that comes from agents making decisions and taking actions autonomously.


What is AgentSpace?

AgentSpace provides a centralized environment for four core functions related to AI Agents:


Building and Iteration: It offers frameworks and templates to define an agent's reasoning capabilities, its permitted external tools (APIs, databases), and its core mission (e.g., "The Customer Service Agent").


Deployment: It handles the transition from a development environment to a production environment, ensuring the agent is containerized, secure, and ready to handle high traffic.


Governance and Safety: It allows developers to define guardrails and constraints to ensure the agent's actions are safe, ethical, and comply with corporate policy.


Monitoring and Evaluation: It continuously tracks the agent's performance, latency, failure rates, and reasoning paths, allowing for rapid debugging and improvement.


How AgentSpace Benefits Enterprises

The value of AgentSpace lies in solving the specific challenges that arise when autonomous AI agents are integrated into critical business operations:


1. Robust Governance and Auditability

In an enterprise, every system action must be traceable. Since an AI agent makes its own decisions (e.g., calling an internal API or creating a ticket), strict control is necessary.


Benefit: AgentSpace provides detailed logging and audit trails for every action an agent takes, every tool it calls, and every internal reasoning step. This ensures regulatory compliance and provides a clear chain of accountability.


Safety Guards: It allows the enterprise to define security parameters—what APIs the agent is allowed to call, what data tables it is prohibited from accessing—thereby mitigating security and compliance risks.


2. Scalability and Reliability (Observability)

An agent that works well in testing must scale to handle thousands or millions of user interactions.


Benefit: AgentSpace is built on cloud infrastructure designed for massive scale. It handles load balancing and resource allocation automatically. More importantly, it provides deep observability tools (dashboards, metrics) that track agent performance in real-time. This helps enterprises quickly identify and fix issues like agents getting stuck in loops, using outdated information, or generating high-latency responses.


3. Accelerated Time-to-Value

Building a complex, custom agent often involves stitching together multiple tools, models, and data sources.


Benefit: The platform provides pre-integrated tools and frameworks that simplify the creation of complex agents. By managing the underlying infrastructure, versioning, and deployment logic, AgentSpace dramatically reduces the time required for developers to move an agent from a concept to a reliable production service. This means faster delivery of capabilities like automated triage, complex data analysis assistants, and autonomous execution of workflows.

What is Gemini Gems ?

A "Gem" is essentially a dedicated, personalized workspace powered by the Gemini model. You can think of it as your own private, tailored AI assistant created for a specific purpose or project.


The core idea behind Gems is to give users control over the scope and focus of their conversations, offering a middle ground between a general public chat and a highly customized application.


Key Characteristics of Gems:

Specialization: You can create a Gem with a specific persona and instructions. For example:


A "Coding Coach" Gem focused only on Python and Docker.


A "Travel Planner" Gem focused only on itinerary creation and logistics.


A "Creative Writer" Gem focused on fiction and storytelling.


Isolated Context: A Gem maintains its own history and context, separate from your main Gemini chat history. This isolation helps keep conversations focused and prevents context from bleeding across unrelated topics.


Efficiency: Because the Gem has a defined role, it is often more efficient and accurate in responding to specialized prompts within that domain.


What is "Saved Info in Gems"?

"Saved Info" is the feature that allows you to provide a Gem with long-term, persistent context and preference data that it uses across all your future interactions with that specific Gem.


This is fundamentally different from standard chat history, where the model only remembers what was discussed in the current thread.


The Purpose of Saved Info:

Personalized Grounding: You can input explicit, private data that the Gem should always reference.


Consistent Persona: The Gem can use this information to maintain consistency and relevance over time.



In short, Gems are the personalized chat environments, and Saved Info is the specific, long-term memory that makes each Gem uniquely useful to you by eliminating the need to repeat your preferences in every new conversation.



Wednesday, October 1, 2025

Google Cloud Learning - GenMedia MCP server

You can use the Firebase MCP server to give AI-powered development tools the ability to work with your Firebase projects. The Firebase MCP server works with any tool that can act as an MCP client, including Claude Desktop, Cline, Cursor, Visual Studio Code Copilot, Windsurf Editor, and more.

An editor configured to use the Firebase MCP server can use its AI capabilities to help you:


Create and manage Firebase projects

Manage your Firebase Authentication users

Work with data in Cloud Firestore and Firebase Data Connect

Retrieve Firebase Data Connect schemas

Understand your security rules for Firestore and Cloud Storage for Firebase

Send messages with Firebase Cloud Messaging



MCP Servers for Genmedia x Gemini CLI


What is the "Genmedia x Gemini CLI" Context?

Before defining MCP, let's look at the components:


Gemini CLI: The command-line interface used to interact with the Gemini model family, allowing developers and users to trigger GenAI tasks, deploy models, and manage input/output data.


Genmedia: This is a term likely referring to a suite of Google Cloud Media Services or applications focused on Generative Media (handling, processing, and generating video, audio, and high-resolution images). These workloads are extremely resource-intensive.


The MCP Servers are the dedicated backbone for the "Genmedia" part of the equation.


The Role of MCP Servers (Media-Optimized Compute)

While "MCP" can have various meanings, in this high-performance context, it is inferred to stand for a specialized compute platform, potentially Media Compute Platform or similar proprietary internal terminology.


These servers are designed to address the unique challenges of generative media:


1. High-Performance Hardware

These are not general-purpose virtual machines. MCP Servers would be provisioned with specialized hardware necessary to run state-of-the-art media and AI models efficiently:


GPUs/TPUs: They are powered by massive arrays of Graphics Processing Units (GPUs) or Tensor Processing Units (TPUs), which are essential for the parallel computations required by large transformer models like Gemini.


Large Memory and VRAM: Generative media tasks (especially video) require large amounts of Video RAM (VRAM) and system memory to hold both the large models and the massive input/output files.


2. High Throughput & Low Latency

Processing a 4K video or generating several minutes of complex animation requires moving terabytes of data quickly.


High-Speed Networking: MCP Servers are equipped with extremely high-bandwidth networking (often 100Gbps or higher) to minimize the latency involved in reading media from storage, running it through the model, and writing the result back.


Optimized Storage: They often interface directly with low-latency, high-throughput storage systems tailored for media workloads.


3. Dedicated Workloads for Genmedia

When you use the Gemini CLI to initiate a video generation task (a Genmedia workload), the system transparently routes that request to these specialized MCP Servers because they are the only infrastructure capable of completing the task economically and quickly


Tuesday, September 30, 2025

Google Cloud Learning - Context7 MCP Server

Context7 provides up-to-date documentation for LLMs and AI code editors. If you are looking to provide the context to the LLM with the latest documentation for the framework of your choice, the Context7 MCP server is a good one to configure.

Make sure that you do have your library listed at the Context7 home page.

Here is the MCP Server that you need to add in the settings.json file.

"context7": {

      "httpUrl": "https://mcp.context7.com/mcp"

    }

Once the MCP Server is configured and Gemini CLI loaded successfully with it, you should be able to view the Context7 tools as shown below:

You can now be specific in your prompt and ask Gemini CLI to use Context7 for the latest documentation, while generating your application or code snippet, using a specific XYZ framework.

Here is an example prompt, where I want to write an Agent using the Agent Development Kit (ADK) from Google. I am specifying in my prompt to look up the documentation for the same via the Context7 MCP Server.

I am working on coding an Agent using the Agent Development Kit (ADK) from Google. I would like to know how to create the LLMAgent in Python. Use Context7 for the latest documentation on ADK and specifically use /google/adk-python, /google/adk-docs and adk.wiki 

Google Slides MCP Server

The Github project at https://github.com/matteoantoci/google-slides-mcp provides a MCP server for interacting with the Google Slides API. It allows you to create, read, and modify Google Slides presentations programmatically.

The steps to configure the MCP server are given in the project. You will need to have a Node.js environment where you build out the server, configure a Google Cloud Project and OAuth 2.0 tokens and then configure the MCP Server in the settings.json file.

Once setup, you can run prompts like:

Extract the latest information from "web_url", summarize it into key points and create a presentation named "my_presentation".

Give it a try!


Google Cloud Learning - Github MCP Server

The Github official MCP Server provides sufficient documentation on the tools that it exposes along with how to configure the same. You can pick your choice in terms of running it locally or remotely, since Gemini CLI supports remote MCP Servers too.


Once you have PAT, you will need to add the MCP Server object in the settings.json file. The complete settings.json file on my system is shown below. You might have additional settings, but the mcpServers object should be as given below:


{

  "theme": "Default",

  "selectedAuthType": "oauth-personal",

  "mcpServers": {

       "github": {

            "httpUrl": "https://api.githubcopilot.com/mcp/",

            "headers": {

                "Authorization": "GITHUB_PAT"

            },

            "timeout": 5000

       }

  }

}



You can either start Gemini CLI again or do a /mcp refresh command, once you have updated the settings.json with the Github MCP Server configuration. The screenshot below highlights the Github MCP Server that is configured on my machine and the various tools that are now available to the Gemini CLI to work with MCP.


An example question like this below 


"Who am on GitHub?"


Notice that it will pick the correct tool from the Github MCP Server but as with other in-built Tools, this will also require that you provide explicit permission to invoke the tool. Go ahead and see what output you get.


You should now work with one of your Github projects. Give your queries in natural language like:


Describe the <repo-name> to me?

Clone the <repo-name> on my local machine.

Describe @<file-name> or @<directory-name>/

What are the different components of this repository?

I have made necessary changes. Can you push the changes to Github and use the Github MCP Server tools to do that.



Google Cloud Learning - Configuring MCP Servers

An MCP server is an application that exposes tools and resources to the Gemini CLI through the Model Context Protocol, allowing it to interact with external systems and data sources. MCP servers act as a bridge between the Gemini model and your local environment or other services like APIs.


An MCP server enables the Gemini CLI to discover and execute tools thereby extending Gemini CLI's capabilities to perform actions beyond its built-in features, such as interacting with databases, APIs, custom scripts, or specialized workflows.


You can configure MCP servers at the global level in the ~/.gemini/settings.json file or in your project's root directory. Create or open the .gemini/settings.json file. Within the file, you will need to add the mcpServers configuration block, as shown below:


"mcpServers": {

    "server_name_1": {},

    "server_name_2": {},

    "server_name_n": {}

 }


Each server configuration supports the following properties


Required (one of the following)


command (string): Path to the executable for Stdio transport

url (string): SSE endpoint URL (e.g., "http://localhost:8080/sse")

httpUrl (string): HTTP streaming endpoint URL



Optional


args (string[]): Command-line arguments for Stdio transport

headers (object): Custom HTTP headers when using url or httpUrl

env (object): Environment variables for the server process. Values can reference environment variables using $VAR_NAME or ${VAR_NAME} syntax

cwd (string): Working directory for Stdio transport

timeout (number): Request timeout in milliseconds (default: 600,000ms = 10 minutes)

trust (boolean): When true, bypasses all tool call confirmations for this server (default: false)

includeTools (string[]): List of tool names to include from this MCP server. When specified, only the tools listed here will be available from this server (whitelist behavior). If not specified, all tools from the server are enabled by default.

excludeTools (string[]): List of tool names to exclude from this MCP server. Tools listed here will not be available to the model, even if they are exposed by the server. Note: excludeTools takes precedence over includeTools - if a tool is in both lists, it will be excluded.



Google Cloud Learning - Gemini CLI Shell mode

Gemini Shell mode (sometimes referred to simply as "Shell" or "Command Line Mode") is a specialized environment where the Gemini model acts as a highly intelligent terminal assistant. Instead of just generating explanatory text about how to perform a task, the model generates, executes, and debugs commands directly in a simulated command-line interface.


It's essentially a way to get help with scripting, debugging, configuration, and environment setup without leaving the command line.


You can go into shell mode by pressing ! in the message box. This will toggle to a Shell mode. You can get back by pressing ! again or by hitting the ESC key.


When in shell mode, you will see the ! at the start of the prompt as shown below:


You can directly check using standard commands like pwd and ls as shown below. Please note that the commands assume a Linux type OS. If you are on another operating system (Windows), please use equivalent commands like (cd, dir and type). Do note that the output of these calls is included in the model's context window.


That's a great question about one of the interactive and helpful modes of the Gemini family of models!


Gemini Shell mode (sometimes referred to simply as "Shell" or "Command Line Mode") is a specialized environment where the Gemini model acts as a highly intelligent terminal assistant. Instead of just generating explanatory text about how to perform a task, the model generates, executes, and debugs commands directly in a simulated command-line interface.


It's essentially a way to get help with scripting, debugging, configuration, and environment setup without leaving the command line.


How Gemini Shell Mode Works

The core idea is to bridge the gap between human language and complex command-line actions:


1. Context and Execution

When you enter Shell mode, you give Gemini a high-level task (e.g., "Find all Python files that haven't been modified in the last 30 days and list their sizes").


2. Command Generation

Gemini translates your request into the appropriate command-line syntax (e.g., a complex find command with arguments).


3. Verification and Execution

Crucially, Gemini typically asks you to verify the command before executing it in the simulated environment. Once executed, it displays the output, just as a real terminal would.


4. Iteration and Debugging

If the command fails or if the output isn't what you expected, you can tell Gemini: "That output is too verbose; just show me the file names." Gemini will then generate a new, refined command, often involving piping the output to other tools like grep or awk.


Main Benefits of Using Shell Mode

Learning and Exploration: It helps users quickly learn complex shell commands (bash, zsh, etc.) by showing the exact syntax and explaining the function of each flag.


Time Savings: It eliminates the need to look up documentation for tricky commands (like awk or complex sed expressions).


Debugging Assistance: It helps diagnose issues with scripts, environment variables, and missing libraries quickly, especially in environments like Docker or remote servers.


Complex Task Automation: It can combine multiple steps (e.g., compress a directory, encrypt it, and upload it to a remote server) into a sequence of executable, verified commands.


In short, it transforms the command line from a strictly manual interface into a collaborative workspace with an AI expert.


Google Cloud Learning - Gemini CLI basics

# option 1: install Gemini CLI

npm install -g @google/gemini-cli

# .. and then run

gemini


# option 2: run without installing Gemini CLI


npx https://github.com/google-gemini/gemini-cli


gemini --version



Login via Google can be used for initial personal tests 


Settings are applied with the following precedence (Cloud Shell only makes User settings available):


System: /etc/gemini-cli/settings.json (applies to all users, overrides user and workspace settings).

Workspace: .gemini/settings.json (overrides user settings).

User: ~/.gemini/settings.json.



Typical JSON is as below 


{

  "theme": "Default",

  "selectedAuthType": "oauth-personal" or "cloud-shell"

}



Sample query as below 


Give me a famous quote on Artificial Intelligence and who said that?


You will notice that our query resulted in a GoogleSearch tool (an in-built tool in Gemini CLI) that got invoked. In other words, you have already exercised one of Gemini CLIs powerful in-built tools namely GoogleSearch that grounds its responses based on information that it gets from the web. You will learn more about Tools in the next section.


A quick way to understand Gemini CLI and the various commands that it supports is to type /help



gemini --help => For help 



Currently, at the time of writing this writing, these are the only two models supported. By default the Gemini 2.5 Pro model is used, but if you would like to use the Flash Model, you can do that at the time of starting Gemini CLI via the -m parameter as shown below:


gemini -m "gemini-2.5-flash"


Non-interactive mode

An interesting option is to run Gemini CLI in a non-interactive mode. This means that you directly provide it the prompt and it will go ahead and respond to it, without the Gemini CLI interactive terminal opening up. This is very useful if you plan to use Gemini CLI in an automated fashion as part of the script or any other automation process. You use the -p parameter to provide the prompt to Gemini CLI as shown below:


gemini -p "What is the gcloud command to deploy to Cloud Run"


To get a list of current Built-in Tools, invoke the /tools command as shown below:


While there is a --yolomode available when you start the CLI (not typically recommended), you will find that Gemini CLI will prompt you for permission to run the tool that it has chosen. You can refuse permission, or let it run once or give it blanket permission to always run. You are and should be in full control of things.



A typical example is like below 


Search for the latest headlines today in the world of finance and save them in a file named finance-news-today.txt



The first thing it does is that it invokes the GoogleSearch tool to search the web.

Once that is done, it is ready to write this to the file and it will use the WriteFile tool, but since that is a sensitive operation (write), it will ask for your permission. You can decide the permission type i.e. allow once , allow always, etc. Go ahead and select allow once for now.

This will then write the information to the file and a success message 


read the contents of @finance-news-today.txt => this can be used to read the contents of the file. 


Google Cloud Learning - Google cloud commands list

Once connected to Cloud Shell, you check that you're already authenticated and that the project is set to your project ID using the following command:


gcloud auth list



Run the following command in Cloud Shell to confirm that the gcloud command knows about your project.

gcloud config list project



If your project is not set, use the following command to set it:

gcloud config set project <YOUR_PROJECT_ID>



You can install Gemini CLI globally on your system first. You may need Administrator access to perform this step.


# option 1: install Gemini CLI


npm install -g @google/gemini-cli


# .. and then run

gemini



option 2: run without installing Gemini CLI


npx https://github.com/google-gemini/gemini-cli


gemini --version


If you installed Gemini CLI on your machine and launched it for the first time, you selected a theme and then an authentication method.


Now, on subsequent runs of Gemini CLI, you will not be asked to select a theme and authentication method again. This means that it is getting persisted somewhere and the file that it uses is called settings.json and it is the way to customize Gemini CLI.


Settings are applied with the following precedence (Cloud Shell only makes User settings available):


System: /etc/gemini-cli/settings.json (applies to all users, overrides user and workspace settings).

Workspace: .gemini/settings.json (overrides user settings).

User: ~/.gemini/settings.json.


Typical settings.json is like below 


{

  "theme": "Default",

  "selectedAuthType": "oauth-personal" or "cloud-shell"

}



Overview of LlamaParse Features

Supported file types: PDF, .pptx, .docx, .rtf, .pages, .epub, etc…

Transformed output type: Markdown, text

Extraction Capabilities: Text, tables, images, graphs, comic books, mathematics equations

Customized Parsing Instructions: Since LlamaParse is LLM enabled, you can pass it instructions just as if you were prompting an LLM. You could use this prompt to describe the document therefore adding more context for the LLM to use while parsing, indicate how you want the output to look, or ask the LLM to do preprocessing during parsing like sentiment analysis, language translation, summarization, etc…

JSON Mode: Outputs the complete structure of the document, extracts images with size and location metadata, extracts tables in JSON format for easy analysis. This is perfect for custom RAG applications where document structure and metadata are used to maximize informational value of documents and for citing where in a document retrieved nodes originate.


The Markdown Advantage

There are some unique advantages to LlamaParse transforming a PDF into markdown format. Markdown specifies the inherent structure of the document by identifying structural elements like titles, headers, subsections, tables, and images. This may seem trivial, but since markdown identifies these elements, we can easily split a document into smaller chunks based on structure using specialized parsers from LlamaIndex like the MarkdownElementNodeParser(). The result of representing a PDF file in markdown format is it enables us to extract each element of the PDF and ingest them into the RAG pipeline.



Below is a retrieval query for the same 


from openai import OpenAI

client = OpenAI()


def embed_query(query):

    query_embedding = client.embeddings.create(

            input=query,

            model="text-embedding-3-small"

        )

    return query_embedding.data[0].embedding


def retrieve_data(query):

    query_embedding = embed_query(query)

    results = table.search(vectors={'flat':[query_embedding]},n=5,filter=[('<>','document_id','4a9551df-5dec-4410-90bb-43d17d722918')])

    retrieved_data_for_RAG = []

    for index, row in results[0].iterrows():

      retrieved_data_for_RAG.append(row['text'])

    return retrieved_data_for_RAG


def RAG(query):

  question = "You will answer this question based on the provided reference material: " + query

  messages = "Here is the provided context: " + "\n"

  results = retrieve_data(query)

  if results:

    for data in results:

      messages += data + "\n"

  response = client.chat.completions.create(

      model="gpt-4o",

      messages=[

          {"role": "system", "content": question},

          {

          "role": "user",

          "content": [

              {"type": "text", "text": messages},

          ],

          }

      ],

      max_tokens=300,

  )

  content = response.choices[0].message.content

  return content


Saturday, September 27, 2025

What is Power BI

 Power BI is a suite of business intelligence (BI) tools and services developed by Microsoft. Its core purpose is to transform raw data into interactive, meaningful visualizations and actionable insights. It allows users to connect to hundreds of data sources, model the data, and create reports and dashboards that can be shared across an organization.

In essence, it helps businesses monitor performance, identify trends, and make data-driven decisions.

Main Applications of Power BI

Power BI is versatile and used across virtually every industry and business function:

Financial Analysis and Reporting: Creating dashboards to track key performance indicators (KPIs) like revenue, profit margins, operational costs, and budget vs. actuals in real-time.

Sales and Marketing Analytics: Visualizing sales pipeline health, campaign performance, lead conversion rates, and regional sales distribution. This helps teams identify top-performing products or sales regions.

Operational and Supply Chain Management: Monitoring inventory levels, tracking delivery performance, analyzing manufacturing defect rates, and visualizing supply chain bottlenecks.

Human Resources (HR) Analytics: Reporting on employee turnover rates, tracking recruitment efficiency (time-to-hire), and visualizing workforce demographics.

Customer Service and Experience: Creating reports to track customer satisfaction scores (CSAT), analyze ticket volumes, and identify trends in support issues.

Main Advantages of Power BI

Power BI has become a dominant tool in the BI space due to several key advantages:

Ease of Use and Accessibility:

It offers a highly intuitive, drag-and-drop interface, making it accessible to business analysts and non-technical users who want to create reports without relying heavily on IT staff.

Integration with Microsoft Ecosystem:

It offers seamless, native integration with other Microsoft products, including Excel, Azure, SQL Server, and Teams. This makes it an easy choice for organizations already invested in the Microsoft stack.

Robust Data Connectivity:

It can connect to hundreds of data sources, from local files (Excel, CSV) and cloud services (Google Analytics, Salesforce) to large-scale data warehouses (Azure Synapse, Snowflake).

Interactive and Dynamic Visualizations:

Reports are highly interactive; users can filter, drill down, and cross-highlight data points simply by clicking on charts or tables. This allows for deep, on-the-fly data exploration.

Scalability and Performance:

Built on Microsoft's robust cloud platform (Azure), it can handle massive datasets (Big Data) efficiently and scales easily as organizational data needs grow.

DAX (Data Analysis Expressions):

Power BI uses the DAX language for creating calculated columns and measures. DAX is very powerful for performing complex calculations like Year-over-Year growth, running totals, and time-intelligence functions, providing deep analytical capabilities.


R learning - What does these below statements do in R plotting ?

# Import ggplot2 package for graphs

install.packages("ggplot2")

library(ggplot2)

  

## Numeric Distribution

 

# Default: Histogram

qplot(GDP.per.capita.2017,data= Countries_Sub)

 

# Add Title and labels:

qplot(GDP.per.capita.2017,data= Countries_Sub,

      main = " GDP Per Capita Distribution in 2017",

      xlab = "GDP per Capita",

      ylab = "Frequency")

 

# Save plot in global environment

p = qplot(GDP.per.capita.2017,data= Countries_Sub,

      main = "SAVED - GDP Per Capita Distribution in 2017",

      xlab = "GDP per Capita",

      ylab = "Frequency")

#Call object

p

 

# Density

qplot(GDP.per.capita.2017,data= Countries_Sub, geom ="density",

      main = " GDP Per Capita Distribution in 2017",

      xlab = "GDP per Capita",

      ylab = "Density")

 

 

## Categorical distribution

 

## Default: Bar plot

qplot(Region,data= Countries_Sub,

      main = "Region Distribution",

      xlab = "Region",

      ylab = "Frequency")

 

 

## Numerical against Categorical

 

# Default: Dot plot

qplot(Region,Under.5.Mortality.Rate.2017,data=Countries_Sub,

      main = "Mortality Rate Distribution per Region",

      xlab = "Region",

      ylab = "Under 5 Mortality Rate")

 

 

# Box plot

qplot(Region, Under.5.Mortality.Rate.2017,data=Countries_Sub, geom= "boxplot",

      main = "Mortality Rate Distribution per Region",

      xlab = "Region",

      ylab = "Under 5 Mortality Rate")

 

# Colored Density plot

qplot( Under.5.Mortality.Rate.2017,fill = Region, data=Countries_Sub, geom= "density", alpha=I(.8),

      main = "Mortality Rate Distribution per Region",

      xlab = "Under 5 Mortality Rate",

      ylab = "Frequency")

 

 

## Categorical against categorical

 

# Colored bar plot

qplot(Region,fill=IncomeGroup,data=Countries_Sub, geom= "bar",

      main = "Income Group Distribution per Region",

      xlab = "Region",

      ylab = "Frequency")

 

 

## Numerical against numerical

 

# Default: Dot plot

qplot(GDP.per.capita.2017, Under.5.Mortality.Rate.2017,data=Countries_Sub,

      main = "Mortality Rate Distribution against Countries' GDP per Capita in 2017",

      xlab = "GDP per Capita",

      ylab = "Under 5 Mortality Rate")

 

 

 

## Three variables

 

#Colored Dot plot

qplot(GDP.per.capita.2017, Under.5.Mortality.Rate.2017, color = Region, data=Countries_Sub,

      main = "Mortality Rate Distribution against Countries' GDP per Capita in 2017",

      xlab = "GDP per Capita",

      ylab = "Under 5 Mortality Rate")

 

## Save last plot as an image

ggsave('Mortality_Rate_GDP_Region.jpeg', width = 9, height = 6)

 

## Save any plot from global environment as an image

ggsave('GDP_per_Capita_Distrib.jpeg',p, width = 9, height = 6)




Analysis is as below 



# Import ggplot2 package for graphs

install.packages("ggplot2")

library(ggplot2)



install.packages("ggplot2"): This is a one-time command that downloads and installs the ggplot2 package from the CRAN repository onto your computer. This package is essential for creating the plots that follow.


library(ggplot2): This command loads the installed ggplot2 package into your current R session, making all its functions (like qplot, ggsave, etc.) available for use.




2. Analyzing Numeric Distribution (One Variable)


This section focuses on visualizing the distribution of a single numerical variable: GDP.per.capita.2017. The primary function used is qplot() (Quick Plot), which is a simplified way to use ggplot2.


qplot(GDP.per.capita.2017, data= Countries_Sub)

Creates a histogram showing the frequency of different GDP per capita ranges. (The default for one numeric variable).


qplot(..., main = "...", xlab = "...", ylab = "...")

Adds a title (main) and custom labels for the X and Y axes (xlab, ylab) to the histogram for better clarity.


p = qplot(...) and p

Saves the plot to an object named p in the environment, rather than just displaying it. The command p then displays the saved plot.



qplot(..., geom ="density")

Creates a density plot, which is a smoothed-out histogram that shows the probability distribution of the data.



3. Analyzing Categorical Distribution (One Variable)


## Default: Bar plot

qplot(Region,data= Countries_Sub, ...)


This creates a bar plot that shows the count (Frequency) of countries belonging to each unique Region. (The default plot for one categorical variable).


4. Numerical Against Categorical (Two Variables)

This compares the distribution of a numeric variable (Under.5.Mortality.Rate.2017) across different categories (Region).


qplot(Region, Under.5.Mortality.Rate.2017, ...)

Creates a dot plot (or scatter plot) by default, showing individual mortality rate values grouped by region.


qplot(..., geom= "boxplot")

Creates a box plot (or box-and-whisker plot) for each region. This visually summarizes the minimum, maximum, median, and quartiles of the mortality rates for each region.


qplot(..., fill = Region, geom= "density", ...)

Creates a density plot of the mortality rate, but draws a separate, colored density line for each Region to visually compare their probability distributions.



5. Categorical Against Categorical (Two Variables)

# Colored bar plot

qplot(Region, fill=IncomeGroup, data=Countries_Sub, geom= "bar", ...)


This creates a stacked or grouped bar plot. It shows the distribution of Region, but the bars are colored (fill) by IncomeGroup. This visualizes the composition of income groups within each geographic region.



6. Numerical Against Numerical (Two or Three Variables)

This section explores the correlation between two numeric variables.


qplot(GDP.per.capita.2017, Under.5.Mortality.Rate.2017, ...)

Creates a scatter plot (dot plot) where each country is a point, allowing you to visually assess the relationship (or correlation) between GDP and Mortality Rate.


qplot(..., color = Region, ...)

Takes the scatter plot from above and adds a third variable (Region) by coloring each point according to its regional group. This helps identify if the relationship between GDP and Mortality Rate changes based on the region.


7. Saving the Plots

## Save last plot as an image

ggsave('Mortality_Rate_GDP_Region.jpeg', width = 9, height = 6)

 

## Save any plot from global environment as an image

ggsave('GDP_per_Capita_Distrib.jpeg',p, width = 9, height = 6)


ggsave(...): This function saves the specified plot as an image file.


The first ggsave saves the last plot displayed (the Colored Dot plot) to a file named Mortality_Rate_GDP_Region.jpeg.

The second ggsave explicitly saves the plot object p (which was the "GDP Per Capita Distribution" histogram saved earlier) to a file named GDP_per_Capita_Distrib.jpeg.

width and height: These parameters specify the size of the output image in inches.






Friday, September 26, 2025

What is Ollama WebSearch

A new web search API is now available in Ollama. Ollama provides a generous free tier of web searches for individuals to use, and higher rate limits are available via Ollama’s cloud.

This web search capability can augment models with the latest information from the web to reduce hallucinations and improve accuracy.

Web search is provided as a REST API with deeper tool integrations in Ollama’s Python and JavaScript libraries. This also enables models such as OpenAI’s gpt-oss models to conduct long-running research tasks.


curl https://ollama.com/api/web_search \

  --header "Authorization: Bearer $OLLAMA_API_KEY" \

  -d '{

    "query": "what is ollama?"

  }'


Example output


{

  "results": [

    {

      "title": "Ollama",

      "url": "https://ollama.com/",

      "content": "Cloud models are now available..."

    },

    {

      "title": "What is Ollama? Introduction to the AI model management tool",

      "url": "https://www.hostinger.com/tutorials/what-is-ollama",

      "content": "Ariffud M. 6min Read..."

    },

    {

      "title": "Ollama Explained: Transforming AI Accessibility and Language ...",

      "url": "https://www.geeksforgeeks.org/artificial-intelligence/ollama-explained-transforming-ai-accessibility-and-language-processing/",

      "content": "Data Science Data Science Projects Data Analysis..."

    }

  ]

}


To build an agent using web search, below is the process 



ollama pull qwen3:4b


from ollama import chat, web_fetch, web_search


available_tools = {'web_search': web_search, 'web_fetch': web_fetch}


messages = [{'role': 'user', 'content': "what is ollama's new engine"}]


while True:

  response = chat(

    model='qwen3:4b',

    messages=messages,

    tools=[web_search, web_fetch],

    think=True

    )

  if response.message.thinking:

    print('Thinking: ', response.message.thinking)

  if response.message.content:

    print('Content: ', response.message.content)

  messages.append(response.message)

  if response.message.tool_calls:

    print('Tool calls: ', response.message.tool_calls)

    for tool_call in response.message.tool_calls:

      function_to_call = available_tools.get(tool_call.function.name)

      if function_to_call:

        args = tool_call.function.arguments

        result = function_to_call(**args)

        print('Result: ', str(result)[:200]+'...')

        # Result is truncated for limited context lengths

        messages.append({'role': 'tool', 'content': str(result)[:2000 * 4], 'tool_name': tool_call.function.name})

      else:

        messages.append({'role': 'tool', 'content': f'Tool {tool_call.function.name} not found', 'tool_name': tool_call.function.name})

  else:

    break




R Learning - Part 1

What does the below statements do in R

# Set columns in right format

Countries$Country.Name = as.factor(Countries$Country.Name)

Countries$Country.Code = as.factor(Countries$Country.Code)


The as.factor() function is used to convert a column from one data type (likely a character string) into a factor.


Countries$Country.Name = as.factor(...): This line takes the values in the Country.Name column and converts them into an R factor. In R, factors are used to store categorical data (data that falls into distinct groups), such as country names. By converting it to a factor, R treats each unique country name as a distinct, ordered or unordered level.


Countries$Country.Code = as.factor(...): Similarly, this line converts the Country.Code column into a factor. This is appropriate because country codes (e.g., "USA," "CAN," "IND") are also categorical identifiers.


Why is this done?

Converting to a factor is crucial before generating summaries or running statistical models. It tells R that these columns represent distinct groups, not continuous numeric values or arbitrary text.



What does the below do in R ? Is First.Countries.Pop is assignment variable? what does . repersent in between? Space. ?



# Focus on first rows


First.Countries.Pop = head(Countries$Total.Population.2017,10)


First.Countries.Pop


 


# Create logical vector


Big.First.Countries.Flag = First.Countries.Pop>10000000 ## Condition: More than 10 million


Big.First.Countries.Flag



# Focus on first rows

First.Countries.Pop = head(Countries$Total.Population.2017, 10)

First.Countries.Pop


head(): This is a function that returns the first part of an object. In this case, head() is getting the first 10 values from the Countries$Total.Population.2017 column.


First.Countries.Pop is an assignment variable. The = sign is the assignment operator in R. It takes the result of the head() function (a vector of the first 10 population values) and stores it in a new variable named First.Countries.Pop. You can tell this is the case because the second line simply prints the contents of this new variable.


The dot (.) in the variable name: In this context, the dot has no special function in R's syntax. It is simply part of the variable name itself. Programmers often use dots or underscores to separate words in variable names for readability.



# Create logical vector

Big.First.Countries.Flag = First.Countries.Pop > 10000000

Big.First.Countries.Flag


What it does: The result of this test is a new vector made up of only TRUE and FALSE values. For every country with a population over 10 million, the vector will contain a TRUE. For those with a population less than or equal to 10 million, it will contain a FALSE. This type of vector is known as a logical vector.


Big.First.Countries.Flag: This is another assignment variable. The name is a common convention in programming to indicate that the variable's purpose is to act as a flag or a boolean indicator for a specific condition.


In summary, the code first isolates the population data for the first 10 countries and then creates a list of TRUE/FALSE values to easily identify which of those countries meet a specific population threshold.



In the below statement, from where does the read_csv coming from? and what is make ? What is colnames ?


Import dataset

library(readr)

Countries = read_csv("Countries Population.csv")

 


## Get info about functions

?colnames

?make.names() 

 


## Give data frame's columns proper names

colnames(Countries) = make.names(colnames(Countries))

colnames(Countries)



1. Where does read_csv come from?

The function read_csv() is provided by the readr package.


library(readr): This line is necessary because it loads the readr package into your current R session. Once loaded, all the functions within that package—including read_csv(), read_tsv(), etc.—become available for use.


What it does: read_csv("Countries Population.csv") is designed to read Comma Separated Values (CSV) files quickly and reliably. It's often preferred over R's base function read.csv() because it's faster and less likely to automatically convert character columns into factors, which gives the user more control over data types.


Countries = ...: This assigns the resulting data frame (the contents of the CSV file) to the variable named Countries.



2. What is make.names()?

The make.names() function is a base R function, meaning it is always available in your session without needing to load a package. It is used to automatically clean up character strings so they are valid and safe to use as variable or column names in R.


Syntax Correction: R has rules for what constitutes a valid variable name. It cannot start with a number, contain spaces, or include most punctuation (like hyphens, parentheses, etc.).


How it works:


It converts spaces and other illegal characters into dots (.).


If a name starts with a number, it will prepend an X.


It ensures all names are unique by appending a number (e.g., Column.1, Column.2) if duplicates exist.


In your code:


colnames(Countries): This function first returns the current names of all columns in the Countries data frame (e.g., it might return a vector like c("Country Name", "Total Population (2017)")).


make.names(colnames(Countries)): make.names() takes that vector of names and converts them into safe names (e.g., "Country Name" becomes "Country.Name", and "Total Population (2017)" might become "Total.Population..2017.").


colnames(Countries) = ...: Finally, the assignment operator (=) replaces the original column names with the new, standardized, dot-separated names.



3. What is colnames()?

The colnames() function is another base R function used to interact with the column names of a data frame or matrix.


When used alone (as a getter): colnames(Countries) returns a character vector listing all the names of the columns in the Countries data frame.


When used with assignment (as a setter): colnames(Countries) = new_names is used to replace the existing column names with a new vector of names (new_names), as demonstrated when it is combined with make.names().


The last line of your code, colnames(Countries), simply prints the final, cleaned column names to the console to confirm the make.names() operation was successful.


Thursday, September 25, 2025

How to have SSE based respones from FAstAPI to react front end ?

Below is the python script to do this

==========================

from fastapi import FastAPI

from fastapi.middleware.cors import CORSMiddleware

from starlette.responses import StreamingResponse

import asyncio


app = FastAPI()


# Configure CORS to allow the React app to connect

origins = [

    "http://localhost:3000",  # Default create-react-app port

]


app.add_middleware(

    CORSMiddleware,

    allow_origins=origins,

    allow_credentials=True,

    allow_methods=["*"],

    allow_headers=["*"],

)


async def stream_generator():

    """Generates a stream of data for SSE."""

    for i in range(1, 11):

        yield f"data: This is message number {i}.\n\n"

        await asyncio.sleep(1)

    

@app.get("/stream")

async def get_stream():

    """Endpoint that returns a Server-Sent Event stream."""

    return StreamingResponse(stream_generator(), media_type="text/event-stream")


Below is the app jsx file 

=================

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>SSE React App</title>
<script src="https://unpkg.com/react@18/umd/react.production.min.js" crossorigin></script>
<script src="https://unpkg.com/react-dom@18/umd/react-dom.production.min.js" crossorigin></script>
<script src="https://unpkg.com/@babel/standalone/babel.min.js"></script>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif;
margin: 0;
padding: 0;
display: flex;
justify-content: center;
align-items: flex-start;
min-height: 100vh;
background-color: #f0f2f5;
}
.container {
background: #fff;
padding: 40px;
border-radius: 12px;
box-shadow: 0 4px 15px rgba(0, 0, 0, 0.1);
max-width: 600px;
width: 90%;
margin-top: 40px;
}
h1 {
color: #333;
font-size: 2rem;
text-align: center;
margin-bottom: 1rem;
}
.info-text {
color: #666;
text-align: center;
margin-bottom: 2rem;
}
.output-box {
background: #fafafa;
border: 1px solid #ddd;
border-radius: 8px;
padding: 15px;
min-height: 150px;
margin-top: 20px;
overflow-y: auto;
font-size: 0.9rem;
line-height: 1.6;
color: #444;
white-space: pre-wrap;
word-wrap: break-word;
}
.placeholder-text {
color: #999;
text-align: center;
font-style: italic;
padding-top: 20px;
}
.error-text {
color: #d9534f;
text-align: center;
font-weight: bold;
}
.loading-text {
color: #5bc0de;
text-align: center;
font-style: italic;
}
.action-button {
display: block;
width: 100%;
padding: 12px 24px;
margin-top: 25px;
font-size: 1.1rem;
font-weight: bold;
color: #fff;
background-color: #007bff;
border: none;
border-radius: 8px;
cursor: pointer;
transition: background-color 0.3s, transform 0.1s;
}
.action-button:hover:not(:disabled) {
background-color: #0056b3;
}
.action-button:active:not(:disabled) {
transform: translateY(1px);
}
.action-button:disabled {
background-color: #ccc;
cursor: not-allowed;
}
</style>
</head>
<body>
<div id="root"></div>

<script type="text/babel">
const { useState, useEffect } = React;

function App() {
const [streamedText, setStreamedText] = useState("");
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState(null);

const startStreaming = () => {
setIsLoading(true);
setError(null);
setStreamedText("");
const eventSource = new EventSource("http://localhost:8000/stream");
eventSource.onmessage = (event) => {
setStreamedText(prevText => prevText + event.data);
};
eventSource.onerror = (err) => {
console.error("EventSource failed:", err);
setError("Failed to connect to the streaming service. Please check the server.");
setIsLoading(false);
eventSource.close();
};
eventSource.onopen = () => {
console.log("Connection opened.");
// This will automatically stop the loading state when the stream closes
eventSource.addEventListener('close', () => {
setIsLoading(false);
eventSource.close();
});
};
};

return (
<div className="container">
<h1>SSE Streaming Demo</h1>
<p className="info-text">
This is a simple demo showing a React client consuming a Server-Sent Event stream from a FastAPI backend.
<br />
Make sure your backend server is running on <a href="http://localhost:8000" target="_blank" rel="noopener noreferrer">http://localhost:8000</a> before you click the button.
</p>
<div className="output-box">
{isLoading && !error && (
<p className="loading-text">Streaming response...</p>
)}
{error && <p className="error-text">{error}</p>}
{!isLoading && streamedText.length === 0 && !error && (
<p className="placeholder-text">Click the button below to start the stream.</p>
)}
<pre>{streamedText}</pre>
</div>
<button onClick={startStreaming} disabled={isLoading} className="action-button">
{isLoading ? 'Streaming...' : 'Start Streaming'}
</button>
</div>
);
}

ReactDOM.render(<App />, document.getElementById('root'));
</script>
</body>
</html>


 

Monday, September 22, 2025

How does the persistent volume get created?

 The flow chart is somewhat like this below 

flowchart LR

  P1["Pod"]

  PVC1["PersistentVolumeClaim (PVC)"]

  SC1["StorageClass"]

  PV1["PersistentVolume (PV)"]


  P1 -->|mounts| PVC1

  PVC1 -->|requests storage| SC1

  SC1 -->|dynamically provisions| PV1

  PVC1 -->|bound to| PV1

  PV1 -->|actualstorage| P1


  1. Pod mounts a PVC.

  2. The PVC uses a StorageClass.

  3. The StorageClass provisions a PV dynamically.

  4. The PVC becomes bound to that PV.

  5. The Pod now uses that PV as actual storage.




Wednesday, September 17, 2025

What is Kubernetes namespace?

🔹 What is a Kubernetes Namespace?

A namespace in Kubernetes is a way to divide cluster resources into isolated, virtual clusters.

Think of it as a way to group resources logically (like projects, teams, or environments) within the same physical cluster.

Default namespaces:

default → where resources go if you don’t specify a namespace.

kube-system → system resources like CoreDNS, kube-proxy.

kube-public → public resources accessible cluster-wide.

kube-node-lease → for node heartbeats.

Why namespaces?

1. Separation of concerns (team A vs team B apps).

2. Resource quotas and limits per namespace.

3. Easier RBAC (role-based access control).

4. Prevent naming collisions (two pods with same name but in different namespaces).

🔹 How it Works

1. Every resource (Pod, Service, Deployment, etc.) belongs to a namespace.

2. Names are unique per namespace, not across the whole cluster.

3. You can control resources in one namespace without affecting others.

4. Some cluster-scoped resources (like Nodes, PersistentVolumes, StorageClasses) are not namespaced.

🔹 Verify Namespace Features (Hands-On in Minikube)

Here’s a set of YAML files you can try.

1. Create a Namespace

namespace.yaml

apiVersion: v1

kind: Namespace

metadata:

  name: dev-team

Apply it:

kubectl apply -f namespace.yaml

kubectl get namespaces

2. Create a Pod in that Namespace

pod-dev.yaml

apiVersion: v1

kind: Pod

metadata:

  name: nginx-dev

  namespace: dev-team

labels:

  app: nginx

spec:

  containers:

  - name: nginx

    image: nginx:latest

    ports:

    - containerPort: 80

Apply it:

kubectl apply -f pod-dev.yaml

kubectl get pods -n dev-team

3. Same Pod Name in Different Namespace

Let’s create another pod with the same name nginx-dev but in default.

pod-default.yaml

apiVersion: v1

kind: Pod

metadata:

  name: nginx-dev

  namespace: default

spec:

  containers:

  - name: nginx

    image: nginx:latest

Apply:

kubectl apply -f pod-default.yaml

kubectl get pods -n default

👉 Now you’ll see two pods with the same name, one in dev-team, one in default. This shows name isolation.

4. Access Pods by Namespace

kubectl get pods -A          # list pods across all namespaces

kubectl describe pod nginx-dev -n dev-team

5. (Optional) Set Default Namespace for kubectl

Instead of -n dev-team every time:

kubectl config set-context --current --namespace=dev-team

Now all commands default to dev-team.

6. Clean Up

kubectl delete namespace dev-team

kubectl delete pod nginx-dev -n default

✅ With this flow, you’ll see how namespaces isolate resources, allow duplicate names, and provide flexibility.

Monday, September 15, 2025

What does pip install -e .

 

Great question 👌 — pip install -e . is a very common command in Python development, especially when working on local projects or libraries.


🔍 What it means

  • pip install → tells pip to install a package.

  • -e (short for --editable) → tells pip to install the package in editable/development mode.

  • . → means “install the current directory” (where your setup.py or pyproject.toml file lives).


So, pip install -e . installs your project in editable mode from the current folder.


⚡ What actually happens

  1. Normal install (without -e)

    • Pip builds your package (from setup.py / pyproject.toml).

    • Copies all files into your Python site-packages directory.

    • Any change in your local source files won’t affect the installed version — you’d need to reinstall.

  2. Editable install (-e)

    • Instead of copying, pip creates a link (symlink or .egg-link) from site-packages back to your source directory.

    • Your project’s metadata (.egg-info or .dist-info) is still created in site-packages.

    • Now, when you import the package, Python loads directly from your source folder.

    ✅ This means any changes you make to the code locally are immediately reflected without reinstalling.


🛠 Typical use cases

  • Developing your own Python package and testing changes quickly.

  • Contributing to an open-source library and wanting to run it locally.

  • Avoiding repeated re-installation during active development.


📌 Example


Suppose your project looks like this:

myproject/
├── setup.py
├── pyproject.toml
└── mypackage/
    └── __init__.py

Run:

pip install -e .

  • Your package mypackage is now importable anywhere (e.g., import mypackage).

  • Any edits you make to mypackage/__init__.py will be reflected immediately when you re-run Python.


👉 Would you like me to also explain how pip install -e . works with pyproject.toml-based projects (PEP 517/518) since many modern projects no longer use setup.py?