Model

infra.llm.model

Summary

Dataclass describing an LLM model and its retry policy.

---

Carry the model identifier, provider name, and the retry, rate-limit, token, and timeout settings used by the pool retry loop.

Classes

ModelPort dataclass

ModelPort(
    name: str,
    provider: str,
    max_retries: int = 3,
    retry_delay: float = 2.0,
    rate_limit_delay: float = 1.0,
    max_tokens: int = 4096,
    timeout: float = 120.0,
)

Describe an LLM model and its request policy.

Attributes:

Name	Type	Description
name	str	Provider-specific model identifier.
provider	str	Provider key (for example groq).
max_retries	int	Number of retry attempts per request.
retry_delay	float	Seconds to wait between attempts.
rate_limit_delay	float	Seconds to wait before each call.
max_tokens	int	Maximum tokens to request from the model.
timeout	float	Request timeout in seconds.