Thursday, January 16, 2025

What is Langchain Agent-Inbox?


Allows to view and respond to interrupts generated from Langgraph. 




class HumanInterruptConfig(TypedDict):

    allow_ignore: bool

    allow_respond: bool

    allow_edit: bool

    allow_accept: bool


class ActionRequest(TypedDict):

    action: str

    args: dict


class HumanInterrupt(TypedDict):

    action_request: ActionRequest

    config: HumanInterruptConfig

    description: Optional[str]


class HumanResponse(TypedDict):

    type: Literal['accept', 'ignore', 'response', 'edit']

    args: Union[None, str, ActionRequest]


The human interrupt schema is used to define the types of interrupts, and what actions can be taken in response to each interrupt. We've landed on four types of actions:


accept: Accept the interrupt's arguments, or action. Will send an ActionRequest in the args field on HumanResponse. This ActionRequest will be the exact same as the action_request field on HumanInterrupt, but with all keys of the args field on ActionRequest converted to strings.

edit: Edit the interrupt's arguments. Sends an instance of ActionRequest in the args field on HumanResponse. The args field on ActionRequest will have the same structure as the args field on HumanInterrupt, but the values of the keys will be strings, and will contain any edits the user has made.

response: Send a response to the interrupt. Does not require any arguments. Will always send back a single string in the args field on HumanResponse.

ignore: Ignore the interrupt's arguments, or action. Returns null for the args field on HumanResponse.

You can set any combination of these actions in the config field on HumanInterrupt.


At the moment, you're required to pass a list of HumanInterrupt objects in the interrupt function, however the UI is currently limited to rendering only the first object in the list. (We are open to suggestions for how to improve this schema, so if you have feedback, please reach out to discuss!). The same goes for the HumanResponse, which the Agent Inbox will always send back as a list with a single HumanResponse object in it.


Tuesday, January 14, 2025

OpenSPG in AI

OpenSPG is a knowledge graph engine developed by Ant Group in collaboration with OpenKG, based on the SPG (Semantic-enhanced Programmable Graph) framework, which is a summary of Ant Group's years of experience in constructing and applying diverse domain knowledge graphs in the financial scenarios.


SPG (Semantic-enhanced Programmable Graph): semantic-enhanced programmable framework is a set of semantic representation framework based on property graph precipitated by Ant Knowledge Graph platform after years of supporting business in the financial field. It creatively integrates LPG structural and RDF semantic, which overcomes the problem that RDF/OWL semantic complexity cannot be industrially landed, and fully inherits the advantages of LPG structural simplicity and compatibility with big data system. The framework defines and represents knowledge semantics from three aspects. First, SPG explicitly defines the formal representation and programmable framework of "knowledge", so that it can be defined, programmed, understood and processed by machines. Secondly, SPG achieves compatibility and progressive advancement between knowledge levels, supporting the construction of knowledge graphs and the continuous iterative evolution of incomplete data states in industrial-level scenarios. Finally, SPG serves as an effective bridge between big data and AI technology systems, facilitating the efficient transformation of massive data into knowledge-based insights. By doing so, it enhances the value and application potential of the data. With the SPG framework, we can construct and manage graph data more efficiently, and at the same time, we can better support business requirements and application scenarios. Since SPG framework has good scalability and flexibility, new business scenarios can quickly build their domain models and solutions by extending the domain knowledge model and developing new operators.


references:

https://github.com/OpenSPG/openspg?tab=readme-ov-file

https://spg.openkg.cn/en-US


OpenSPG in Physics!

 What is OpenSPG


OpenSPG refers to Open Source Physics Graphics, a specialized software library or framework designed for high-performance physics simulations and visualizations. Its primary goal is to combine computational efficiency with advanced graphical rendering, enabling developers, researchers, and engineers to model and visualize complex physics phenomena.


Although there is limited information about a specific project named "OpenSPG" in mainstream open-source communities as of my knowledge cutoff, the name may refer to specialized or niche software, potentially in the following contexts:


Possible Interpretations of OpenSPG

Physics Simulation Framework:


Designed for simulating dynamic systems in areas such as mechanics, thermodynamics, electromagnetism, or fluid dynamics.

May integrate physics engines like Bullet or PyBullet for robust and accurate simulations.

Graphics-Focused Platform:


Provides APIs for rendering high-quality 2D/3D physics models, often used in educational tools, research, or entertainment.

Integrates with graphics libraries like OpenGL, Vulkan, or DirectX.

Educational or Research Tool:


May be tailored for academic purposes, offering pre-built modules to study concepts of classical or quantum physics.

Open Source Collaboration:


Likely an open-source initiative aimed at democratizing access to tools for physics modeling and visualization, fostering innovation in the field.

Core Features (Hypothetical)

If OpenSPG is similar to other physics simulation and graphics platforms, its features might include:


Cross-Platform Support: Works on various operating systems like Windows, macOS, and Linux.

Integration with Machine Learning: Used for AI-driven optimization of simulations or training reinforcement learning models in simulated environments.

High Performance: Exploits modern GPU acceleration for real-time rendering and computation.

Ease of Use: Includes user-friendly APIs and GUI tools for creating simulations without needing extensive programming expertise.


references:

OpenAI 

Monday, January 13, 2025

What is difference between ray[air] and ray[default] package

The difference between ray[air] and ray[default] lies in the set of dependencies they install when you set up the Ray library. These options allow users to customize their installation to suit their specific needs, minimizing unnecessary dependencies.

1. ray[default]

What it includes:

This is the default installation, which contains the core Ray functionality and basic dependencies required to use Ray for distributed computation.

Use case:

Ideal for general-purpose distributed applications, task orchestration, and workflows.

Provides the base components needed for Ray core features like task scheduling, actor management, and Ray's Python API.

Dependencies:

Core Ray libraries only (e.g., task scheduling and execution).

Install Command:

pip install ray[default]

ray[air]

What it includes:

The ray[air] installation includes all dependencies required for Ray AI Runtime (AIR), which is Ray's unified framework for machine learning workflows.

Use case:

Best suited for machine learning and AI-related tasks, such as training, hyperparameter tuning, and serving models.

Adds support for Ray Train, Ray Tune, Ray Serve, and Ray Data, which are specialized components of Ray for ML/AI workflows.

Dependencies:


Includes additional libraries such as:

torch, tensorflow, and other ML frameworks for training models.

scikit-learn for machine learning utilities.

Libraries for handling large datasets and streaming data efficiently.

Ray Serve dependencies for model deployment and serving.

Install Command:

pip install ray[air]

When to Use Which?

Use Case Recommended Installation

Distributed task orchestration ray[default]

General distributed computing ray[default]

Machine learning workflows ray[air]

Model training and hyperparameter tuning ray[air]

Model serving and deployment ray[air]

Handling large-scale data processing ray[air]


Summary

ray[default]: Lightweight, general-purpose distributed computing.

ray[air]: Focused on ML/AI workflows with added dependencies for training, tuning, and serving models.


Friday, January 10, 2025

What is SimCSE Embeeding model?

 Enhanced Semantic Search with Sentence Transformers / Clustering  ( or any other depending on the data) 

A Data specific embedding function that can improve the similarity search ( Instead of using OpenAIEmbedding) 

SimCSE (Simple Contrastive Sentence Embeddings) is a powerful framework for creating high-quality sentence embeddings. It is based on a contrastive learning approach and trains embeddings by minimizing the distance between embeddings of augmented sentence pairs and maximizing the distance between embeddings of different sentences. There are multiple SimCSE-based models, both unsupervised and supervised, developed using pre-trained language models as a base.


Common SimCSE-Based Embedding Models:

SimCSE-Unsupervised Models:


These models generate sentence embeddings using contrastive learning on unlabeled data.

They rely on dropout noise as the augmentation mechanism, meaning the same sentence is passed through the model twice with different dropout masks.

Examples:

bert-base-uncased with SimCSE

roberta-base with SimCSE

distilbert-base-uncased with SimCSE

SimCSE-Supervised Models:


These models are trained on datasets with labeled pairs (e.g., sentence pairs with known similarity scores) such as the Natural Language Inference (NLI) dataset.

They use contrastive learning to align embeddings of semantically similar sentence pairs and separate dissimilar ones.

Examples:

bert-large-nli-stsb trained with SimCSE

roberta-large-nli-stsb trained with SimCSE

xlnet-base-nli-stsb with SimCSE

Multilingual SimCSE Models:


Adapt SimCSE for multilingual text by using multilingual pre-trained models like mBERT or XLM-R.

Examples:

xlm-roberta-base with SimCSE

bert-multilingual with SimCSE

Domain-Specific SimCSE Models:


SimCSE fine-tuned on domain-specific corpora to create embeddings specialized for particular tasks or fields.

Examples:

Legal text embeddings

Biomedical text embeddings (e.g., using BioBERT or SciBERT)

Key Features of SimCSE Models:

They leverage existing pre-trained language models like BERT, RoBERTa, or DistilBERT as the backbone.

SimCSE embeddings are highly effective for tasks like semantic similarity, text clustering, and information retrieval.

Supervised SimCSE models generally outperform unsupervised ones due to task-specific labeled data.

Impact of SimCSE on Retrieval and Similarity Search:

High-Quality Embeddings: SimCSE embeddings capture fine-grained semantic nuances, improving retrieval accuracy.

Efficient Similarity Search: Enhanced embeddings lead to better clustering and ranking in similarity-based searches.

Domain Adaptability: By fine-tuning SimCSE models on specific domains, the performance for domain-specific retrieval improves significantly.

For implementations, you can find pre-trained SimCSE models on platforms like Hugging Face, or you can train your own model using the SimCSE framework available in the official GitHub repository.



references:

OpenAI 

Thursday, January 9, 2025

What are the transports in MCP?

MCP currently defines two standard transport mechanisms for client-server communication:

stdio, communication over standard in and standard out

HTTP with Server-Sent Events (SSE)

Clients SHOULD support stdio whenever possible.

In the stdio transport:

The client launches the MCP server as a subprocess.

The server receives JSON-RPC messages on its standard input (stdin) and writes responses to its standard output (stdout).

Messages are delimited by newlines, and MUST NOT contain embedded newlines.

The server MAY write UTF-8 strings to its standard error (stderr) for logging purposes. Clients MAY capture, forward, or ignore this logging.

The server MUST NOT write anything to its stdout that is not a valid MCP message.

The client MUST NOT write anything to the server’s stdin that is not a valid MCP message.

HTTP with SSE 

In the SSE transport, the server operates as an independent process that can handle multiple client connections.

The server MUST provide two endpoints:

An SSE endpoint, for clients to establish a connection and receive messages from the server

A regular HTTP POST endpoint for clients to send messages to the server

When a client connects, the server MUST send an endpoint event containing a URI for the client to use for sending messages. All subsequent client messages MUST be sent as HTTP POST requests to this endpoint.

Server messages are sent as SSE message events, with the message content encoded as JSON in the event data.

Custom Transports 

Clients and servers MAY implement additional custom transport mechanisms to suit their specific needs. The protocol is transport-agnostic and can be implemented over any communication channel that supports bidirectional message exchange.

Implementers who choose to support custom transports MUST ensure they preserve the JSON-RPC message format and lifecycle requirements defined by MCP. Custom transports SHOULD document their specific connection establishment and message exchange patterns to aid interoperability.

references:

https://spec.modelcontextprotocol.io/specification/basic/transports/

Base protocol for MCP

 Base Protocol: Core JSON-RPC message types

Lifecycle Management: Connection initialization, capability negotiation, and session control

Server Features: Resources, prompts, and tools exposed by servers

Client Features: Sampling and root directory lists provided by clients

Utilities: Cross-cutting concerns like logging and argument completion


Auth 

Authentication and authorization are not currently part of the core MCP specification, but we are considering ways to introduce them in future. 


Lifecycle Phases 

Initialization 

The initialization phase MUST be the first interaction between client and server. During this phase, the client and server:


Establish protocol version compatibility

Exchange and negotiate capabilities

Share implementation details

The client MUST initiate this phase by sending an initialize request containing:


Protocol version supported

Client capabilities

Client implementation information


The server MUST respond with its own capabilities and information:

Sample client info 

{

  "jsonrpc": "2.0",

  "id": 1,

  "method": "initialize",

  "params": {

    "protocolVersion": "2024-11-05",

    "capabilities": {

      "roots": {

        "listChanged": true

      },

      "sampling": {}

    },

    "clientInfo": {

      "name": "ExampleClient",

      "version": "1.0.0"

    }

  }

}


Sample server info 

{

  "jsonrpc": "2.0",

  "id": 1,

  "result": {

    "protocolVersion": "2024-11-05",

    "capabilities": {

      "logging": {},

      "prompts": {

        "listChanged": true

      },

      "resources": {

        "subscribe": true,

        "listChanged": true

      },

      "tools": {

        "listChanged": true

      }

    },

    "serverInfo": {

      "name": "ExampleServer",

      "version": "1.0.0"

    }

  }

}


The client SHOULD NOT send requests other than pings before the server has responded to the initialize request.

The server SHOULD NOT send requests other than pings and logging before receiving the initialized notification.


Version Negotiation 

In the initialize request, the client MUST send a protocol version it supports. This SHOULD be the latest version supported by the client.


If the server supports the requested protocol version, it MUST respond with the same version. Otherwise, the server MUST respond with another protocol version it supports. This SHOULD be the latest version supported by the server.


If the client does not support the version in the server’s response, it SHOULD disconnect.




Error Handling 

Implementations SHOULD be prepared to handle these error cases:


Protocol version mismatch

Failure to negotiate required capabilities

Initialize request timeout

Shutdown timeout

Implementations SHOULD implement appropriate timeouts for all requests, to prevent hung connections and resource exhaustion.


Example initialization error:

{

  "jsonrpc": "2.0",

  "id": 1,

  "error": {

    "code": -32602,

    "message": "Unsupported protocol version",

    "data": {

      "supported": ["2024-11-05"],

      "requested": "1.0.0"

    }

  }

}

references:

https://spec.modelcontextprotocol.io/specification/basic/


Wednesday, January 8, 2025

What is capabiltiy Negotiation in MCP

Capability Negotiation 

The Model Context Protocol uses a capability-based negotiation system where clients and servers explicitly declare their supported features during initialization. Capabilities determine which protocol features and primitives are available during a session.

Servers declare capabilities like resource subscriptions, tool support, and prompt templates

Clients declare capabilities like sampling support and notification handling

Both parties must respect declared capabilities throughout the session

Additional capabilities can be negotiated through extensions to the protocol



Each capability unlocks specific protocol features for use during the session. For example:

Implemented server features must be advertised in the server’s capabilities

Emitting resource subscription notifications requires the server to declare subscription support

Tool invocation requires the server to declare tool capabilities

Sampling requires the client to declare support in its capabilities

This capability negotiation ensures clients and servers have a clear understanding of supported functionality while maintaining protocol extensibility.

references:

https://spec.modelcontextprotocol.io/specification/architecture/


MCP What are Security and Trust & Safety principles in specification?

The Model Context Protocol enables powerful capabilities through arbitrary data access and code execution paths. With this power comes important security and trust considerations that all implementors must carefully address.


Key Principles 

User Consent and Control


Users must explicitly consent to and understand all data access and operations

Users must retain control over what data is shared and what actions are taken

Implementors should provide clear UIs for reviewing and authorizing activities

Data Privacy


Hosts must obtain explicit user consent before exposing user data to servers

Hosts must not transmit resource data elsewhere without user consent

User data should be protected with appropriate access controls

Tool Safety


Tools represent arbitrary code execution and must be treated with appropriate caution

Hosts must obtain explicit user consent before invoking any tool

Users should understand what each tool does before authorizing its use

LLM Sampling Controls


Users must explicitly approve any LLM sampling requests

Users should control:

Whether sampling occurs at all

The actual prompt that will be sent

What results the server can see

The protocol intentionally limits server visibility into prompts

Implementation Guidelines 

While MCP itself cannot enforce these security principles at the protocol level, implementors SHOULD:


Build robust consent and authorization flows into their applications

Provide clear documentation of security implications

Implement appropriate access controls and data protections

Follow security best practices in their integrations

Consider privacy implications in their feature designs

Monday, January 6, 2025

MCP Standards in a nutshell

 1. Request and Response Formats

MCP aims to standardize the structure of requests sent to models and the responses they return. This includes:

Input Formats: Ensuring models can process queries in a common format, regardless of the vendor.

Output Formats: Defining a consistent structure for model responses, including metadata like confidence scores, provenance information, and structured data (e.g., JSON).

Error Handling: Standardized error codes and messages for better debugging and reliability.

2. Context Sharing and State Management

MCP proposes mechanisms to manage and share context between models or sessions, such as:

Memory Persistence: How context is maintained across multiple queries.

Session Management: Allowing continuity in conversations or tasks by persisting user-defined context.

Global Context: Enabling multiple models or tools to access shared context seamlessly.

3. Compatibility Across Tools and APIs

The protocol aims to bridge different vendor ecosystems by:

Unified API Interfaces: A single API specification that can be implemented by all participating models.

Interoperability Standards: Enabling models, vector databases, and tools to work together in workflows like retrieval-augmented generation (RAG) without vendor lock-in.

4. Metadata and Provenance Standards

MCP emphasizes the importance of detailed metadata in model responses, including:

Source Attribution: Where information comes from, especially in multi-source systems.

Confidence Scores: How certain the model is about its outputs.

Execution Logs: Tracing the steps taken to generate a response.

5. Tool Interactions and Plugin Standards

MCP proposes standards for how models interact with external tools, databases, and APIs, including:

Plugin Interfaces: Defining a unified way to integrate tools (e.g., calculators, retrieval systems).

Execution Standards: How models should invoke tools and handle tool responses.

6. Security and Privacy

Establishing protocols to ensure:

Secure Data Transmission: Encrypting queries and responses.

Access Control: Defining who can interact with the model or tools.

Compliance: Adhering to legal and ethical standards for data handling.

7. Evaluation and Logging Standards

Proposals for how to:

Benchmark Models: Using standardized datasets or metrics.

Log Interactions: Tracking user-model interactions for auditing or improving system behavior.

Summary

MCP is essentially proposing a holistic standard that covers:


Request/Response Formats

Context and State Management

Interoperability Across Vendors and Tools

Metadata and Provenance

Security and Compliance

Tool and Plugin Interactions

By addressing these areas, MCP aims to create a more unified, efficient, and user-friendly ecosystem for working with AI models. However, its adoption depends on industry-wide collaboration and agreement.


What are GLM ( Generalized Linear Models)

GLMs can handle a wider range of data distributions.

Explanation:

Generalized Linear Models (GLMs) are an extension of traditional linear regression models that can handle response variables with distributions other than the normal distribution. For example:

Binary outcomes (using logistic regression).

Count data (using Poisson regression).

Proportions (using binomial regression).

Key Advantages of GLMs:

Wider range of distributions: GLMs use a link function to relate the mean of the response variable to the linear predictor, allowing flexibility in modeling different types of data distributions.

Relaxation of linearity assumptions: GLMs allow for non-linear relationships between predictors and the response variable through the link function.

Misconceptions about the other options:

"Simpler to interpret than regression models": Not always true; the interpretation of coefficients in GLMs depends on the link function, which can make them less intuitive than traditional regression.

"Less computationally intensive": GLMs can be more computationally intensive due to their iterative fitting procedures (e.g., maximum likelihood estimation).

"Do not require the assumption of linearity": This is somewhat true in the sense that the relationship between the predictors and the response variable is modeled through a link function, but the linearity assumption still applies to the predictors in the linear predictor.


How to display tables and images using mark downs?

 We can display tables in a markdown document using the below examples 

Basic Table Structure

Header Row:

Start with a row of column headers separated by pipes (|).

Include a pipe at the beginning and end of the row.

Separator Row:

Add a row of dashes (-) below the header row.

Optionally, add colons (:):

: at the start of a row of dashes aligns text to the left.

: at the end of a row of dashes aligns text to the right.

: at both the start and end of a row of dashes centers the text.

Data Rows:

Add subsequent rows with data for each cell, also separated by pipes.   

Example


Markdown


| Header 1 | Header 2 | Header 3 |

| -------- | :-------: | --------: | 

| Row 1, Cell 1 | Row 1, Cell 2 | Row 1, Cell 3 |

| Row 2, Cell 1 | Row 2, Cell 2 | Row 2, Cell 3 | 

This will render as:


Header 1 Header 2 Header 3

Row 1, Cell 1 Row 1, Cell 2 Row 1, Cell 3

Row 2, Cell 1 Row 2, Cell 2 Row 2, Cell 3


Export to Sheets

Key Points:


Consistency: Ensure consistent use of pipes throughout the table.

Alignment: Use colons (:), as shown above, to control text alignment within cells.

Whitespace: Whitespace within cells is generally preserved.

Line Breaks: To add line breaks within a cell, use the <br> HTML tag.

By following these guidelines, you can effectively create and format tables within your Markdown documents.



We can embed images in Markdown using the following syntax:


![Alt text](image_url)


![](): This is the basic syntax for embedding an image in Markdown.

Alt text: This is the alternative text that will be displayed if the image cannot be loaded. It's crucial for accessibility (screen readers, search engines).   

image_url: This is the URL of the image file. It can be:

A local file path: If the image is stored in the same directory as your Markdown file (e.g., images/my_image.jpg).

A URL to an image online: (e.g., https://example.com/image.png)

  

Example:


Markdown


![A beautiful cat](images/cat.jpg)

This will display an image named "cat.jpg" located in the "images" folder within the same directory as your Markdown file. The alt text will be "A beautiful cat".


Important Notes:


Image Size: You can control the size of the image using HTML within the Markdown.

For example: ![Alt text](image_url "width=200 height=100")

Image Hosting: If you're sharing your Markdown file online, ensure the images are hosted on a server or use a service like GitHub Pages or Imgur to host the images.

By using this syntax, you can easily incorporate images into your Markdown documents, making them more visually appealing and informative.

Sources and related content


Docker Image Layer Analysis

Lets examine various layers using below Dockerfile 

FROM python:3.12-slim

# Set the working directory

WORKDIR /app

# Copy the requirements and install dependencies

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

# Copy the application code

COPY ./app ./app

# Expose the application port

EXPOSE 8000

# Start the FastAPI app

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

1. Base Image Layer

What Happens During Image Build?

Read the Dockerfile:

Docker sequentially reads each instruction in the Dockerfile.

Base Image:

If the Dockerfile starts with a FROM instruction, Docker pulls the specified base image (e.g., python:3.12) from the registry (e.g., Docker Hub) if it's not already available locally.

Layer Creation:

For each instruction (e.g., RUN, COPY, ADD), Docker creates a layer.

Each layer represents changes made to the filesystem (e.g., installed packages, copied files).

Layer Caching:


Docker caches each layer. If you rebuild the image and the instructions haven't changed, Docker reuses the cached layer instead of executing the instruction again.

This caching improves build performance.

Final Image:


After processing all instructions, Docker combines the layers into a single image.

The resulting image contains all the instructions applied in sequence, ready for distribution or execution.



How Layers Improve Docker Build Efficiency

Caching: Docker reuses unchanged layers from the cache when rebuilding an image, speeding up builds.

Shared Layers: Containers based on the same image share layers, reducing storage usage.



Best Practices for Layer Efficiency

Minimize Layer Count: Combine commands where possible (e.g., RUN apt-get update && apt-get install).

Order Matters: Place the most frequently changing instructions last in the Dockerfile.

Leverage .dockerignore: Exclude unnecessary files to reduce context size and prevent cache invalidation.

Group related instructions to minimize layers.

Install dependencies in one step to avoid redundant layers.

Use a lightweight base image (e.g., slim).



Build using below command 


docker build -t mrrathish/docker-layer-analysis:latest .




The output is displayed like the below 



Now instead of displaying the base image name as python:3.12-slim which is specified in the Dockerfile, it displays Blob . This is because Dive analyzes and displays all layers in the resulting image, including those inherited from the base image. Docker stores image layers as content-addressable blobs. Each blob corresponds to a change in the filesystem or metadata. 

dive doesn't show the image tag (python:3.12-slim) because the tag is metadata used for convenience. Instead, it shows the underlying layers that make up the image. This is the reason the  FROM python:3.12-slim is translated into the 4 different layers that make up the base image. 



If notice, EXPOSE does not appear as a separate layer. This is because Layers in Docker images are typically created only by instructions that alter the file system, such as RUN, ADD, or COPY. The EXPOSE goes as part of Metadata of the docker image. The port specified by EXPOSE is included in the image metadata which we can see as ExposedPorts in the docker inspect <image-name> output. 


Some of he layers are 0B size, this is because These layers exist as placeholders in the image's metadata but do not contribute to the overall size.


Now let's examine how instruction change in Dockerfile affects layer and image size. Before this below steps, the size of the image created is 172MB.

Change the Dockerfile to look like below 


FROM python:3.12-slim


# Set the working directory

WORKDIR /app


# Copy the requirements and install dependencies

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt


RUN apt-get update && \

    apt-get install -y curl && \

    apt-get install -y vim


# Copy the application code

COPY ./app ./app


# Expose the application port

EXPOSE 8000


# Start the FastAPI app

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]



Now if we see the Layes, it looks like below 


Additional three layers got added.


The total images size has grown to 237MB 


Now let's combine the RUN statements into single like using && like this below. 


Now the layers looks like the below. 




The image size is reduced slightly to 236MB and the layers got reduced. 


Also in the docker file frequently changing instructions has to be placed towards bottom of the Dockerfile


If frequently changing layers are placed at the top, every subsequent layer (even rarely changing ones) must be rebuilt during a change.

By placing rarely changing instructions at the top, Docker can reuse cached layers, rebuilding only the final layers that are affected by changes.




Sunday, January 5, 2025

MCP Architecture

 Why MCP?

MCP helps you build agents and complex workflows on top of LLMs. LLMs frequently need to integrate with data and tools, and MCP provides:


A growing list of pre-built integrations that your LLM can directly plug into

The flexibility to switch between LLM providers and vendors

Best practices for securing your data within your infrastructure


General architecture

At its core, MCP follows a client-server architecture where a host application can connect to multiple servers:





MCP Hosts: Programs like Claude Desktop, IDEs, or AI tools that want to access data through MCP

MCP Clients: Protocol clients that maintain 1:1 connections with servers

MCP Servers: Lightweight programs that each expose specific capabilities through the standardized Model Context Protocol

Local Data Sources: Your computer’s files, databases, and services that MCP servers can securely access

Remote Services: External systems available over the internet (e.g., through APIs) that MCP servers can connect to


What is Model Context Protocol?

Model Control Protocol" (MCP) refers to the "Model Context Protocol," an open-source standard developed by Anthropic that allows developers to easily connect AI models (like their "Claude" assistant) to various data sources and external tools, essentially providing a standardized way for AI systems to access contextual information from different applications and databases; making integration between AI and other systems smoother and more efficient. 

Key points about MCP:

Function:

It acts as a bridge between AI models and external data sources, allowing them to access relevant information without requiring custom integrations for each new data source. 

Open-source nature:

The protocol is open-source, meaning developers can freely access and utilize it to build their own AI applications that can connect to diverse data sources. 

Benefits:

MCP simplifies the process of integrating AI models into existing systems, enabling more robust and contextually aware AI applications. 

Developed by Anthropic:

This protocol is primarily associated with Anthropic, the AI company known for their "Claude" language model. 

references:

Gemini 

Enabling prometheus monitoring in docker component in Anchore

Optional: Enabling Prometheus Monitoring

Uncomment the following section at the bottom of the docker-compose.yaml file:

#  # Uncomment this section to add a prometheus instance to gather metrics. This is mostly for quickstart to demonstrate prometheus metrics exported

#  prometheus:

#    image: docker.io/prom/prometheus:latest

#    depends_on:

#      - api

#    volumes:

#      - ./anchore-prometheus.yml:/etc/prometheus/prometheus.yml:z

#    logging:

#      driver: "json-file"

#      options:

#        max-size: 100m

#    ports:

#      - "9090:9090"

#

For each service entry in the docker-compose.yaml, change the following to enable metrics in the API for each service

ANCHORE_ENABLE_METRICS=false

to

ANCHORE_ENABLE_METRICS=true

Download the example prometheus configuration into the same directory as the docker-compose.yaml file, with name anchore-prometheus.yml

curl -O https://engine.anchore.io/docs/quickstart/anchore-prometheus.yml

docker-compose up -d

You should see a new container started and can access prometheus via your browser on http://localhost:9090

references:

https://engine.anchore.io/docs/quickstart/


Saturday, January 4, 2025

What is Anchore, docker image analysis tool?

The Anchore Engine is an open source project that provides a centralized service for inspection, analysis and certification of container images. The Anchore Engine is provided as a Docker container image that can be run standalone or on an orchestration platform such as Kubernetes, Docker Swarm, Rancher or Amazon ECS.


Using the Anchore Engine, container images can be downloaded from Docker V2 compatible container registries, analyzed and evaluated against user-defined policies. The Anchore Engine can integrate with Anchore's Navigator service allowing you to define policies and whitelists using a graphical editor that is automatically synchronized to the Anchore Engine.


The Anchore Engine can be integrated into CI/CD pipelines such as Jenkins to secure your CI/CD pipeline by adding image scanning including not just CVE based security scans but policy-based scans that can include checks around security, compliance and operational best practices.


The Anchore Engine can be accessed directly through a RESTful API or via the Anchore CLI. Adding an image to be analyzed is a simple one-line command:


anchore-cli image add docker.io/library/nginx:latest


The Anchore Engine will now download the image from the registry and perform deep inspection collecting data on packages, files, software artifacts and image metadata.


Once analyzed we can retrieve information about the image. For example, retrieving a list of packages:


anchore-cli image content docker.io/library/nginx:latest os


Will return a list of operating system (os) packages found in the image. In addition to operating system packages, we can retrieve details about files, Ruby GEMs and Node.JS NPMs.


anchore-cli image content docker.io/library/rails:latest gem



Package Version Location

actioncable 5.0.1 /usr/local/bundle/specifications/actioncable-5.0.1.gemspec

actionmailer 5.0.1 /usr/local/bundle/specifications/actionmailer-5.0.1.gemspec

actionpack 5.0.1 /usr/local/bundle/specifications/actionpack-5.0.1.gemspec

actionview 5.0.1 /usr/local/bundle/specifications/actionview-5.0.1.gemspec

activejob 5.0.1 /usr/local/bundle/specifications/activejob-5.0.1.gemspec

activemodel 5.0.1 /usr/local/bundle/specifications/activemodel-5.0.1.gemspec

activerecord 5.0.1 /usr/local/bundle/specifications/activerecord-5.0.1.gemspec

activesupport 5.0.1 /usr/local/bundle/specifications/activesupport-5.0.1.gemspec

arel 7.1.4 /usr/local/bundle/specifications/arel-7.1.4.gemspec


And if we wanted to see how many security vulnerabilities in an image you can run the following command:



anchore-cli image vuln docker.io/library/ubuntu:latest os



Vulnerability ID Package Severity Fix Vulnerability URL

CVE-2013-4235 login-1:4.2-3.1ubuntu5.3 Low None http://people.ubuntu.com/~ubuntu-security/cve/CVE-2013-4235

CVE-2013-4235 passwd-1:4.2-3.1ubuntu5.3 Low None http://people.ubuntu.com/~ubuntu-security/cve/CVE-2013-4235

CVE-2015-5180 libc-bin-2.23-0ubuntu9 Low None http://people.ubuntu.com/~ubuntu-security/cve/CVE-2015-5180

CVE-2015-5180 libc6-2.23-0ubuntu9 Low None http://people.ubuntu.com/~ubuntu-security/cve/CVE-2015-5180

CVE-2015-5180 multiarch-support-2.23-0ubuntu9 Low None http://people.ubuntu.com/~ubuntu-security/cve/CVE-2015-5180



As with the content sub-command we pass a parameter for the type of content we want to analyze - in this case, OS for operating system packages. Future releases will add support for non-package vulnerability data.


Next, we can evaluate the image against a policy that was defined either manually on the command line or using the Anchore Navigator


anchore-cli evaluate check registry.example.com/webapps/frontend:latest


Image Digest: sha256:86774cefad82967f97f3eeeef88c1b6262f9b42bc96f2ad61d6f3fdf54475ac3

Full Tag: registry.example.com/webapps/frontend:latest

Status: pass

Last Eval: 2017-09-09T18:30:22

Policy ID: 715a6056-87ab-49fb-abef-f4b4198c67bf


Here we can see that the image passed. To see the details of the evaluation you can add the --detail parameter. For example:


anchore-cli evaluate check registry.example.com/webapps/broker:latest --detail


Image Digest: sha256:7f97f3eeeef88c1b6262f9b42bc96f2ad61d6f3fdf54475ac354475ac

Full Tag: registry.example.com/webapps/broker:latest

Status: fail

Last Eval: 2017-09-09T17:30:22

Policy ID: 715a6056-87ab-49fb-abef-f4b4198c67bf


Gate                   Trigger              Detail                                                          Status        

DOCKERFILECHECK        NOHEALTHCHECK        Dockerfile does not contain any HEALTHCHECK instructions        warn

ANCHORESEC             VULNHIGH             HIGH Vulnerability found in package - libmount1 (CVE-2016-2779 - https://security-tracker.debian.org/tracker/CVE-2016-2779)                    stop          

ANCHORESEC             VULNHIGH             HIGH Vulnerability found in package - libncurses5 (CVE-2017-10684 - https://security-tracker.debian.org/tracker/CVE-2017-10684)                stop          

ANCHORESEC             VULNHIGH             HIGH Vulnerability found in package - libncurses5 (CVE-2017-10685 - https://security-tracker.debian.org/tracker/CVE-2017-10685)                stop



References:

https://anchore.com/blog/anchore-engine/


Wednesday, January 1, 2025

Docker: Why You See Unhealthy Without Specifying a Health Check

There are two possible scenarios when you see Up (unhealthy) without explicitly defining HEALTHCHECK:

Inherited Health Checks from Base Image

Some official or custom base images may already define a HEALTHCHECK directive in their Dockerfile. If your Docker image is built on such a base image, the health check will propagate to your final container unless explicitly overridden or disabled.

For example:

dockerfile

FROM mysql:latest  # The MySQL base image includes a default HEALTHCHECK

To inspect if a base image has a health check:


bash

Copy code

docker inspect <base-image-id> | grep -A 5 '"Healthcheck"'

Orchestrator or Monitoring Tool Integration

If you're running Docker containers with orchestration tools like Docker Swarm, Kubernetes, or using Docker with third-party monitoring systems, the orchestrator or tool might simulate health checks (e.g., by probing network endpoints). This can indirectly cause Up (unhealthy) statuses.


How Docker Handles Default Status Without Health Checks

Without Health Check:

If no health check is specified, Docker only tracks the basic running state of the container.

Status examples:


Up — The container process is running.

Exited — The container has stopped (with an exit code).

With Health Check (Inherited or Defined):

If a health check is defined or inherited:


Docker periodically runs the health check command.

The container status becomes:

healthy — If the health check succeeds.

unhealthy — If the health check fails multiple times consecutively.




How to Remove or Disable Inherited Health Checks

If the HEALTHCHECK directive is coming from a base image, you can disable it by adding the following in your Dockerfile:


dockerfile

Copy code


HEALTHCHECK NONE


This will completely remove any inherited health check from the base image.


Next Steps for Debugging

Check for Inherited Health Checks: Inspect the image you’re using to see if it has a health check:


docker inspect <image-name>


Override or Remove Health Checks: Use the HEALTHCHECK NONE directive in your Dockerfile or configure a custom health check to suit your app.


Log Analysis: Examine container logs (docker logs <container-id>) to understand why the runtime might appear "unhealthy."

What is Snyk

Snyk is a cloud-native security tool that helps developers build secure applications: 

Find and fix vulnerabilities: Snyk can automatically find and fix vulnerabilities in code, open-source dependencies, containers, and infrastructure as code. 

Prioritize security efforts: Snyk uses a risk-based approach to focus on issues that matter. 

Manage security programs: Snyk provides visibility into coverage and business context, smart policies, and analytics and reporting. 

Integrate with IDEs: Snyk has plugins for IDEs like Eclipse, VS Code, and Jetbrains. 

Enforce best practices: Snyk's IaC security platform uses built-in rulesets for Terraform, CloudFormation, ARM, and Kubernetes. 

Snyk was founded in 2015 in London and Tel Aviv, and is headquartered in Boston.

references: