Pool

infra.llm.providers.groq.pool

Summary

Groq pool adaptors.

---

Wire Groq configuration and model ports into the synchronous and asynchronous LLM pool adaptors.

Classes

AsyncGroqPool

AsyncGroqPool(**kwargs)

Bases: AsyncPoolAdaptor

Asynchronous LLM pool backed by Groq.

Initialize the pool with Groq defaults.

Parameters:

Name	Type	Description	Default
**kwargs		Overrides for the client and model ports.	{}

Functions

aclose async

aclose()

Close the underlying asynchronous HTTP client.

generate async

generate(
    prompt: str,
    system_prompt: Optional[str] = None,
    max_tokens: Optional[int] = None,
    models: Optional[list] = None,
) -> str

Generate text for a prompt asynchronously.

Parameters:

Name	Type	Description	Default
prompt	str	User prompt text.	required
system_prompt	Optional[str]	Optional system instruction.	None
max_tokens	Optional[int]	Token cap overriding model defaults.	None
models	Optional[list]	Model ports overriding the pool defaults.	None

Returns:

Name	Type	Description
str	str	Generated content.

SyncGroqPool

SyncGroqPool(**kwargs)

Bases: PoolAdaptor

Synchronous LLM pool backed by Groq.

Initialize the pool with Groq defaults.

Parameters:

Name	Type	Description	Default
**kwargs		Overrides for the client and model ports.	{}

Functions

close

close()

Close the underlying HTTP client.

generate

generate(
    prompt: str,
    system_prompt: Optional[str] = None,
    max_tokens: Optional[int] = None,
    models: Optional[list] = None,
) -> str

Generate text for a prompt.

Parameters:

Name	Type	Description	Default
prompt	str	User prompt text.	required
system_prompt	Optional[str]	Optional system instruction.	None
max_tokens	Optional[int]	Token cap overriding model defaults.	None
models	Optional[list]	Model ports overriding the pool defaults.	None

Returns:

Name	Type	Description
str	str	Generated content.

Functions