What is Langchain Agent-Inbox?

Allows to view and respond to interrupts generated from Langgraph. 

class HumanInterruptConfig(TypedDict):

    allow_ignore: bool

    allow_respond: bool

    allow_edit: bool

    allow_accept: bool

class ActionRequest(TypedDict):

    action: str

    args: dict

class HumanInterrupt(TypedDict):

    action_request: ActionRequest

    config: HumanInterruptConfig

    description: Optional[str]

class HumanResponse(TypedDict):

    type: Literal['accept', 'ignore', 'response', 'edit']

    args: Union[None, str, ActionRequest]

The human interrupt schema is used to define the types of interrupts, and what actions can be taken in response to each interrupt. We've landed on four types of actions:

accept: Accept the interrupt's arguments, or action. Will send an ActionRequest in the args field on HumanResponse. This ActionRequest will be the exact same as the action_request field on HumanInterrupt, but with all keys of the args field on ActionRequest converted to strings.

edit: Edit the interrupt's arguments. Sends an instance of ActionRequest in the args field on HumanResponse. The args field on ActionRequest will have the same structure as the args field on HumanInterrupt, but the values of the keys will be strings, and will contain any edits the user has made.

response: Send a response to the interrupt. Does not require any arguments. Will always send back a single string in the args field on HumanResponse.

ignore: Ignore the interrupt's arguments, or action. Returns null for the args field on HumanResponse.

You can set any combination of these actions in the config field on HumanInterrupt.

At the moment, you're required to pass a list of HumanInterrupt objects in the interrupt function, however the UI is currently limited to rendering only the first object in the list. (We are open to suggestions for how to improve this schema, so if you have feedback, please reach out to discuss!). The same goes for the HumanResponse, which the Agent Inbox will always send back as a list with a single HumanResponse object in it.

OpenSPG in AI

OpenSPG is a knowledge graph engine developed by Ant Group in collaboration with OpenKG, based on the SPG (Semantic-enhanced Programmable Graph) framework, which is a summary of Ant Group's years of experience in constructing and applying diverse domain knowledge graphs in the financial scenarios.

SPG (Semantic-enhanced Programmable Graph): semantic-enhanced programmable framework is a set of semantic representation framework based on property graph precipitated by Ant Knowledge Graph platform after years of supporting business in the financial field. It creatively integrates LPG structural and RDF semantic, which overcomes the problem that RDF/OWL semantic complexity cannot be industrially landed, and fully inherits the advantages of LPG structural simplicity and compatibility with big data system. The framework defines and represents knowledge semantics from three aspects. First, SPG explicitly defines the formal representation and programmable framework of "knowledge", so that it can be defined, programmed, understood and processed by machines. Secondly, SPG achieves compatibility and progressive advancement between knowledge levels, supporting the construction of knowledge graphs and the continuous iterative evolution of incomplete data states in industrial-level scenarios. Finally, SPG serves as an effective bridge between big data and AI technology systems, facilitating the efficient transformation of massive data into knowledge-based insights. By doing so, it enhances the value and application potential of the data. With the SPG framework, we can construct and manage graph data more efficiently, and at the same time, we can better support business requirements and application scenarios. Since SPG framework has good scalability and flexibility, new business scenarios can quickly build their domain models and solutions by extending the domain knowledge model and developing new operators.

references:

https://github.com/OpenSPG/openspg?tab=readme-ov-file

https://spg.openkg.cn/en-US

OpenSPG in Physics!

 What is OpenSPG

OpenSPG refers to Open Source Physics Graphics, a specialized software library or framework designed for high-performance physics simulations and visualizations. Its primary goal is to combine computational efficiency with advanced graphical rendering, enabling developers, researchers, and engineers to model and visualize complex physics phenomena.

Although there is limited information about a specific project named "OpenSPG" in mainstream open-source communities as of my knowledge cutoff, the name may refer to specialized or niche software, potentially in the following contexts:

Possible Interpretations of OpenSPG

Physics Simulation Framework:

Designed for simulating dynamic systems in areas such as mechanics, thermodynamics, electromagnetism, or fluid dynamics.

May integrate physics engines like Bullet or PyBullet for robust and accurate simulations.

Graphics-Focused Platform:

Provides APIs for rendering high-quality 2D/3D physics models, often used in educational tools, research, or entertainment.

Integrates with graphics libraries like OpenGL, Vulkan, or DirectX.

Educational or Research Tool:

May be tailored for academic purposes, offering pre-built modules to study concepts of classical or quantum physics.

Open Source Collaboration:

Likely an open-source initiative aimed at democratizing access to tools for physics modeling and visualization, fostering innovation in the field.

Core Features (Hypothetical)

If OpenSPG is similar to other physics simulation and graphics platforms, its features might include:

Cross-Platform Support: Works on various operating systems like Windows, macOS, and Linux.

Integration with Machine Learning: Used for AI-driven optimization of simulations or training reinforcement learning models in simulated environments.

High Performance: Exploits modern GPU acceleration for real-time rendering and computation.

Ease of Use: Includes user-friendly APIs and GUI tools for creating simulations without needing extensive programming expertise.

references:

OpenAI 

What is difference between ray[air] and ray[default] package

The difference between ray[air] and ray[default] lies in the set of dependencies they install when you set up the Ray library. These options allow users to customize their installation to suit their specific needs, minimizing unnecessary dependencies.

1. ray[default]

What it includes:

This is the default installation, which contains the core Ray functionality and basic dependencies required to use Ray for distributed computation.

Use case:

Ideal for general-purpose distributed applications, task orchestration, and workflows.

Provides the base components needed for Ray core features like task scheduling, actor management, and Ray's Python API.

Dependencies:

Core Ray libraries only (e.g., task scheduling and execution).

Install Command:

pip install ray[default]

ray[air]

What it includes:

The ray[air] installation includes all dependencies required for Ray AI Runtime (AIR), which is Ray's unified framework for machine learning workflows.

Use case:

Best suited for machine learning and AI-related tasks, such as training, hyperparameter tuning, and serving models.

Adds support for Ray Train, Ray Tune, Ray Serve, and Ray Data, which are specialized components of Ray for ML/AI workflows.

Dependencies:

Includes additional libraries such as:

torch, tensorflow, and other ML frameworks for training models.

scikit-learn for machine learning utilities.

Libraries for handling large datasets and streaming data efficiently.

Ray Serve dependencies for model deployment and serving.

Install Command:

pip install ray[air]

When to Use Which?

Use Case Recommended Installation

Distributed task orchestration ray[default]

General distributed computing ray[default]

Machine learning workflows ray[air]

Model training and hyperparameter tuning ray[air]

Model serving and deployment ray[air]

Handling large-scale data processing ray[air]

Summary

ray[default]: Lightweight, general-purpose distributed computing.

ray[air]: Focused on ML/AI workflows with added dependencies for training, tuning, and serving models.

What is SimCSE Embeeding model?

 Enhanced Semantic Search with Sentence Transformers / Clustering  ( or any other depending on the data) 

A Data specific embedding function that can improve the similarity search ( Instead of using OpenAIEmbedding) 

SimCSE (Simple Contrastive Sentence Embeddings) is a powerful framework for creating high-quality sentence embeddings. It is based on a contrastive learning approach and trains embeddings by minimizing the distance between embeddings of augmented sentence pairs and maximizing the distance between embeddings of different sentences. There are multiple SimCSE-based models, both unsupervised and supervised, developed using pre-trained language models as a base.

Common SimCSE-Based Embedding Models:

SimCSE-Unsupervised Models:

These models generate sentence embeddings using contrastive learning on unlabeled data.

They rely on dropout noise as the augmentation mechanism, meaning the same sentence is passed through the model twice with different dropout masks.

Examples:

bert-base-uncased with SimCSE

roberta-base with SimCSE

distilbert-base-uncased with SimCSE

SimCSE-Supervised Models:

These models are trained on datasets with labeled pairs (e.g., sentence pairs with known similarity scores) such as the Natural Language Inference (NLI) dataset.

They use contrastive learning to align embeddings of semantically similar sentence pairs and separate dissimilar ones.

Examples:

bert-large-nli-stsb trained with SimCSE

roberta-large-nli-stsb trained with SimCSE

xlnet-base-nli-stsb with SimCSE

Multilingual SimCSE Models:

Adapt SimCSE for multilingual text by using multilingual pre-trained models like mBERT or XLM-R.

Examples:

xlm-roberta-base with SimCSE

bert-multilingual with SimCSE

Domain-Specific SimCSE Models:

SimCSE fine-tuned on domain-specific corpora to create embeddings specialized for particular tasks or fields.

Examples:

Legal text embeddings

Biomedical text embeddings (e.g., using BioBERT or SciBERT)

Key Features of SimCSE Models:

They leverage existing pre-trained language models like BERT, RoBERTa, or DistilBERT as the backbone.

SimCSE embeddings are highly effective for tasks like semantic similarity, text clustering, and information retrieval.

Supervised SimCSE models generally outperform unsupervised ones due to task-specific labeled data.

Impact of SimCSE on Retrieval and Similarity Search:

High-Quality Embeddings: SimCSE embeddings capture fine-grained semantic nuances, improving retrieval accuracy.

Efficient Similarity Search: Enhanced embeddings lead to better clustering and ranking in similarity-based searches.

Domain Adaptability: By fine-tuning SimCSE models on specific domains, the performance for domain-specific retrieval improves significantly.

For implementations, you can find pre-trained SimCSE models on platforms like Hugging Face, or you can train your own model using the SimCSE framework available in the official GitHub repository.

references:

OpenAI 

What are the transports in MCP?

MCP currently defines two standard transport mechanisms for client-server communication:

stdio, communication over standard in and standard out

HTTP with Server-Sent Events (SSE)

Clients SHOULD support stdio whenever possible.

In the stdio transport:

The client launches the MCP server as a subprocess.

The server receives JSON-RPC messages on its standard input (stdin) and writes responses to its standard output (stdout).

Messages are delimited by newlines, and MUST NOT contain embedded newlines.

The server MAY write UTF-8 strings to its standard error (stderr) for logging purposes. Clients MAY capture, forward, or ignore this logging.

The server MUST NOT write anything to its stdout that is not a valid MCP message.

The client MUST NOT write anything to the server’s stdin that is not a valid MCP message.

HTTP with SSE 

In the SSE transport, the server operates as an independent process that can handle multiple client connections.

The server MUST provide two endpoints:

An SSE endpoint, for clients to establish a connection and receive messages from the server

A regular HTTP POST endpoint for clients to send messages to the server

When a client connects, the server MUST send an endpoint event containing a URI for the client to use for sending messages. All subsequent client messages MUST be sent as HTTP POST requests to this endpoint.

Server messages are sent as SSE message events, with the message content encoded as JSON in the event data.

Custom Transports 

Clients and servers MAY implement additional custom transport mechanisms to suit their specific needs. The protocol is transport-agnostic and can be implemented over any communication channel that supports bidirectional message exchange.

Implementers who choose to support custom transports MUST ensure they preserve the JSON-RPC message format and lifecycle requirements defined by MCP. Custom transports SHOULD document their specific connection establishment and message exchange patterns to aid interoperability.

references:

https://spec.modelcontextprotocol.io/specification/basic/transports/

Base protocol for MCP

 Base Protocol: Core JSON-RPC message types

Lifecycle Management: Connection initialization, capability negotiation, and session control

Server Features: Resources, prompts, and tools exposed by servers

Client Features: Sampling and root directory lists provided by clients

Utilities: Cross-cutting concerns like logging and argument completion

Auth 

Authentication and authorization are not currently part of the core MCP specification, but we are considering ways to introduce them in future. 

Lifecycle Phases 

Initialization 

The initialization phase MUST be the first interaction between client and server. During this phase, the client and server:

Establish protocol version compatibility

Exchange and negotiate capabilities

Share implementation details

The client MUST initiate this phase by sending an initialize request containing:

Protocol version supported

Client capabilities

Client implementation information

The server MUST respond with its own capabilities and information:

Sample client info 

{

  "jsonrpc": "2.0",

  "id": 1,

  "method": "initialize",

  "params": {

    "protocolVersion": "2024-11-05",

    "capabilities": {

      "roots": {

        "listChanged": true

      },

      "sampling": {}

    },

    "clientInfo": {

      "name": "ExampleClient",

      "version": "1.0.0"

    }

  }

}

Sample server info 

{

  "jsonrpc": "2.0",

  "id": 1,

  "result": {

    "protocolVersion": "2024-11-05",

    "capabilities": {

      "logging": {},

      "prompts": {

        "listChanged": true

      },

      "resources": {

        "subscribe": true,

        "listChanged": true

      },

      "tools": {

        "listChanged": true

      }

    },

    "serverInfo": {

      "name": "ExampleServer",

      "version": "1.0.0"

    }

  }

}

The client SHOULD NOT send requests other than pings before the server has responded to the initialize request.

The server SHOULD NOT send requests other than pings and logging before receiving the initialized notification.

Version Negotiation 

In the initialize request, the client MUST send a protocol version it supports. This SHOULD be the latest version supported by the client.

If the server supports the requested protocol version, it MUST respond with the same version. Otherwise, the server MUST respond with another protocol version it supports. This SHOULD be the latest version supported by the server.

If the client does not support the version in the server’s response, it SHOULD disconnect.

Error Handling 

Implementations SHOULD be prepared to handle these error cases:

Protocol version mismatch

Failure to negotiate required capabilities

Initialize request timeout

Shutdown timeout

Implementations SHOULD implement appropriate timeouts for all requests, to prevent hung connections and resource exhaustion.

Example initialization error:

{

  "jsonrpc": "2.0",

  "id": 1,

  "error": {

    "code": -32602,

    "message": "Unsupported protocol version",

    "data": {

      "supported": ["2024-11-05"],

      "requested": "1.0.0"

    }

  }

}

references:

https://spec.modelcontextprotocol.io/specification/basic/