mllm_shap.connectors.openai_compat package

Submodules

mllm_shap.connectors.openai_compat.model module

Text-only causal LM via OpenAI-compatible HTTP API + local HF embeddings.

mllm_shap.connectors.openai_compat.model.ChatCompletionsTransport

POST body to parsed JSON dict (OpenAI-style chat.completions response).

alias of Callable[[str, Mapping[str, str], Mapping[str, Any]], Mapping[str, Any]]

class mllm_shap.connectors.openai_compat.model.OpenAICompatCausalText(device: device, base_url: str | None = None, chat_model: str | None = None, api_key: str | None = None, hf_repo_id: str | None = None, hf_revision: str | None = None, history_tracking_mode: ModelHistoryTrackingMode = ModelHistoryTrackingMode.TEXT, transport: Callable[[str, Mapping[str, str], Mapping[str, Any]], Mapping[str, Any]] | None = None, **kwargs: Any)

Bases: BaseMllmModel

Generation uses an OpenAI-compatible /chat/completions endpoint (LM Studio).

Static/contextual embeddings use a local Hugging Face causal LM (same tokenizer vocabulary as the served model) so SHAP similarity stages keep per-token vectors.

config: HuggingFaceModelConfig

The model configuration.

device: device

The device to run the model on.

generate(chat: BaseMllmChat, max_new_tokens: int = 128, model_config: ModelConfig = ModelConfig(text_temperature=0.0, text_top_k=1, audio_temperature=0.0, audio_top_k=1), keep_history: bool = False) → ModelResponse

Generate audio based on the current chat state.

Parameters:

• chat – The current chat state.

• max_new_tokens – The maximum number of new tokens to generate (default is 20).

• model_config – Additional model configuration parameters.

• keep_history – Whether to return chat state with full history or only generated content.

Returns:

The updated chat state after generation.

Return type:

ModelResponse

generate_batch(chats: list[BaseMllmChat], max_new_tokens: int = 128, model_config: ModelConfig = ModelConfig(text_temperature=0.0, text_top_k=1, audio_temperature=0.0, audio_top_k=1), keep_history: bool = False) → list[ModelResponse]

Generate responses for multiple chats in a single batched forward pass.

The default implementation loops over individual generate calls. Connectors that support true batched inference (e.g. HuggingFace Transformers) should override this for higher GPU throughput.

Parameters:

• chats – List of chat objects to generate from.

• max_new_tokens – Maximum new tokens per generation.

• model_config – Generation configuration.

• keep_history – Whether to keep full chat history in responses.

Returns:

List of ModelResponse objects, one per input chat.

get_contextual_embeddings(*args: Any, static_embeddings: list[Tensor] | None = None, **kwargs: Any) → list[Tensor]

Get contextual embeddings for the current chat state.

Parameters:

• static_embeddings – Precomputed static embeddings (if any).

• *args – Additional positional arguments for get_static_embeddings(). Used if static_embeddings is None.

• **kwargs – Additional keyword arguments for get_static_embeddings(). Used if static_embeddings is None.

Returns:

The context embeddings for the text and audio tokens, same format as in

get_static_embeddings().

Raises:

ValueError – If static_embeddings is not an instance of Tensor.

get_new_chat(**kwargs: Any) → TransformersTextChat

Get a new chat state for the model.

get_static_embeddings(responses: list[ModelResponse]) → list[Tensor]

Get static embeddings for the current chat state.

Parameters:

responses – The model responses to get embeddings for.

Returns:

The static embeddings for the text and audio tokens.

Raises:

ValueError – If responses is not a list of ModelResponse.

history_tracking_mode: ModelHistoryTrackingMode

The mode for tracking chat history.

model: PreTrainedModel

The model instance.

processor: Any

The model processor (tokenizer).

Module contents

OpenAI-compatible HTTP chat (LM Studio, vLLM, etc.) with local HF embeddings.

class mllm_shap.connectors.openai_compat.OpenAICompatCausalText(device: device, base_url: str | None = None, chat_model: str | None = None, api_key: str | None = None, hf_repo_id: str | None = None, hf_revision: str | None = None, history_tracking_mode: ModelHistoryTrackingMode = ModelHistoryTrackingMode.TEXT, transport: Callable[[str, Mapping[str, str], Mapping[str, Any]], Mapping[str, Any]] | None = None, **kwargs: Any)

Bases: BaseMllmModel

Generation uses an OpenAI-compatible /chat/completions endpoint (LM Studio).

Static/contextual embeddings use a local Hugging Face causal LM (same tokenizer vocabulary as the served model) so SHAP similarity stages keep per-token vectors.

config: HuggingFaceModelConfig

The model configuration.

device: device

The device to run the model on.

generate(chat: BaseMllmChat, max_new_tokens: int = 128, model_config: ModelConfig = ModelConfig(text_temperature=0.0, text_top_k=1, audio_temperature=0.0, audio_top_k=1), keep_history: bool = False) → ModelResponse

Generate audio based on the current chat state.

Parameters:

• chat – The current chat state.

• max_new_tokens – The maximum number of new tokens to generate (default is 20).

• model_config – Additional model configuration parameters.

• keep_history – Whether to return chat state with full history or only generated content.

Returns:

The updated chat state after generation.

Return type:

ModelResponse

generate_batch(chats: list[BaseMllmChat], max_new_tokens: int = 128, model_config: ModelConfig = ModelConfig(text_temperature=0.0, text_top_k=1, audio_temperature=0.0, audio_top_k=1), keep_history: bool = False) → list[ModelResponse]

Generate responses for multiple chats in a single batched forward pass.

The default implementation loops over individual generate calls. Connectors that support true batched inference (e.g. HuggingFace Transformers) should override this for higher GPU throughput.

Parameters:

• chats – List of chat objects to generate from.

• max_new_tokens – Maximum new tokens per generation.

• model_config – Generation configuration.

• keep_history – Whether to keep full chat history in responses.

Returns:

List of ModelResponse objects, one per input chat.

get_contextual_embeddings(*args: Any, static_embeddings: list[Tensor] | None = None, **kwargs: Any) → list[Tensor]

Get contextual embeddings for the current chat state.

Parameters:

• static_embeddings – Precomputed static embeddings (if any).

• *args – Additional positional arguments for get_static_embeddings(). Used if static_embeddings is None.

• **kwargs – Additional keyword arguments for get_static_embeddings(). Used if static_embeddings is None.

Returns:

The context embeddings for the text and audio tokens, same format as in

get_static_embeddings().

Raises:

ValueError – If static_embeddings is not an instance of Tensor.

get_new_chat(**kwargs: Any) → TransformersTextChat

Get a new chat state for the model.

get_static_embeddings(responses: list[ModelResponse]) → list[Tensor]

Get static embeddings for the current chat state.

Parameters:

responses – The model responses to get embeddings for.

Returns:

The static embeddings for the text and audio tokens.

Raises:

ValueError – If responses is not a list of ModelResponse.

history_tracking_mode: ModelHistoryTrackingMode

The mode for tracking chat history.

model: PreTrainedModel

The model instance.

processor: Any

The model processor (tokenizer).