diff --git a/docs/content/_index.md b/docs/content/_index.md index 2adfcb4389..eeb5dbcb4f 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -32,13 +32,13 @@ It provides a step by step introduction on how to install Flink Agents and build ### Quickstart -* [Workflow Agent]({{< ref "docs/get-started/quickstart/workflow_agent" >}}) - Define agent logic as an explicit event-driven workflow -* [ReAct Agent]({{< ref "docs/get-started/quickstart/react_agent" >}}) - Let the model plan and call tools in a reason-act loop -* [YAML Agent]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) - Configure an agent declaratively in YAML +- [Build an Agent]({{< ref "docs/get-started/quickstart/create-agent" >}}) +- [Use the Built-in ReAct Agent]({{< ref "docs/get-started/quickstart/react_agent" >}}) +- [Define an Agent with YAML]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) ### Build and Operate -* [Chat Models]({{< ref "docs/development/chat_models" >}}), [Tools]({{< ref "docs/development/tool_use" >}}) and [Memory]({{< ref "docs/development/memory/overview" >}}) - The building blocks of an agent +* [Calling LLMs]({{< ref "docs/development/calling-llms" >}}), [Tool Use]({{< ref "docs/development/calling-llms/tool_use" >}}) and [Memory]({{< ref "docs/development/memory/overview" >}}) - The building blocks of an agent * [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}) - Run an agent inside a Flink job * [Deployment]({{< ref "docs/operations/deployment" >}}) and [Monitoring]({{< ref "docs/operations/monitoring" >}}) - Take an agent to production diff --git a/docs/content/docs/development/calling-llms/_index.md b/docs/content/docs/development/calling-llms/_index.md new file mode 100644 index 0000000000..91c77e7f49 --- /dev/null +++ b/docs/content/docs/development/calling-llms/_index.md @@ -0,0 +1,32 @@ +--- +title: "Calling LLMs" +weight: 2 +type: docs +bookCollapseSection: true +--- + + +- [Chat Models]({{< ref "docs/development/calling-llms/chat-models" >}}) +- [Prompts]({{< ref "docs/development/calling-llms/prompts" >}}) +- [Chat with Models]({{< ref "docs/development/calling-llms/chat" >}}) +- [Model Routing]({{< ref "docs/development/calling-llms/model_routing" >}}) +- [Tool Use]({{< ref "docs/development/calling-llms/tool_use" >}}) +- [MCP]({{< ref "docs/development/calling-llms/mcp" >}}) +- [Skills]({{< ref "docs/development/calling-llms/skills" >}}) diff --git a/docs/content/docs/development/calling-llms/chat-models/_index.md b/docs/content/docs/development/calling-llms/chat-models/_index.md new file mode 100644 index 0000000000..b25dd78038 --- /dev/null +++ b/docs/content/docs/development/calling-llms/chat-models/_index.md @@ -0,0 +1,298 @@ +--- +title: "Chat Models" +weight: 1 +type: docs +aliases: + - /docs/development/calling-llms/declare-resources/ + - /docs/development/chat_models/ +--- + + +# Chat Models + +## Overview + +Chat models enable agents to communicate with Large Language Models (LLMs) for natural language understanding, reasoning, and generation. In Flink Agents, chat models act as the "brain" of your agents, processing input messages and generating intelligent responses based on context, prompts, and available tools. + +## Getting Started + +To use chat models in your agents, you need to define both a connection and setup using decorators/annotations, then interact with the model through events. + +### Resource Declaration + +Flink Agents provides decorators in python and annotations in java to simplify chat model setup within agents: + +#### @chat_model_connection/@ChatModelConnection + +The `@chat_model_connection` decorator or `@ChatModelConnection` annotation marks a method that creates a chat model connection. This is typically defined once and shared across multiple chat model setups. + +#### @chat_model_setup/@ChatModelSetup + +The `@chat_model_setup` decorator or `@ChatModelSetup` annotation marks a method that creates a chat model setup. This references a connection and adds chat-specific configuration like prompts and tools. + +## Using Cross-Language Providers + +Flink Agents supports cross-language chat model integration, allowing you to use chat models implemented in one language (Java or Python) from agents written in the other language. This is particularly useful when a chat model provider is only available in one language (e.g., the DashScope integration for Alibaba Cloud Model Studio is currently Python-only). + +{{< hint warning >}} +**Limitations:** + +- Cross-language resources are currently supported only when [running in Flink]({{< ref "docs/operations/deployment#run-in-flink" >}}), not in local development mode +- Complex object serialization between languages may have limitations +{{< /hint >}} + +### How To Use + +To use a resource implemented in another language, declare its implementation class with the corresponding cross-language resource descriptor: + +- **Using Java chat models in Python**: Use `JavaResourceDescriptor(clazz=ResourceName.ChatModel.Java.OLLAMA_CONNECTION, ...)` (or the corresponding setup class). +- **Using Python chat models in Java**: Use `PythonResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.Python.OLLAMA_CONNECTION)` (or the corresponding setup class). + + + +### Usage Example + +{{< tabs "Cross-Language Chat Model Usage Example" >}} + +{{< tab "Using Java Chat Model in Python" >}} +```python +class MyAgent(Agent): + + @chat_model_connection + @staticmethod + def java_chat_model_connection() -> ResourceDescriptor: + # In pure Java, the equivalent ResourceDescriptor would be: + # ResourceDescriptor.Builder + # .newBuilder(ResourceName.ChatModel.OLLAMA_CONNECTION) + # .addInitialArgument("endpoint", "http://localhost:11434") + # .addInitialArgument("requestTimeout", 120) + # .build(); + return JavaResourceDescriptor( + clazz=ResourceName.ChatModel.Java.OLLAMA_CONNECTION, + endpoint="http://localhost:11434", + requestTimeout=120, + ) + + + @chat_model_setup + @staticmethod + def java_chat_model() -> ResourceDescriptor: + # In pure Java, the equivalent ResourceDescriptor would be: + # ResourceDescriptor.Builder + # .newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) + # .addInitialArgument("connection", "java_chat_model_connection") + # .addInitialArgument("model", "qwen3:8b") + # .addInitialArgument("prompt", "my_prompt") + # .addInitialArgument("tools", List.of("my_tool1", "my_tool2")) + # .addInitialArgument("extractReasoning", true) + # .build(); + return JavaResourceDescriptor( + clazz=ResourceName.ChatModel.Java.OLLAMA_SETUP, + connection="java_chat_model_connection", + model="qwen3:8b", + prompt="my_prompt", + tools=["my_tool1", "my_tool2"], + extract_reasoning=True, + ) + + @action(EventType.InputEvent) + @staticmethod + def process_input(event: Event, ctx: RunnerContext) -> None: + input_event = InputEvent.from_event(event) + # Create a chat request with user message + user_message = ChatMessage.user(f"input: {input_event.input}") + ctx.send_event( + ChatRequestEvent(model="java_chat_model", messages=[user_message]) + ) + + @action(EventType.ChatResponseEvent) + @staticmethod + def process_response(event: Event, ctx: RunnerContext) -> None: + chat_response = ChatResponseEvent.from_event(event) + response_content = chat_response.response.text + # Handle the LLM's response + # Process the response as needed for your use case +``` +{{< /tab >}} + +{{< tab "Using Python Chat Model in Java" >}} +```java +public class MyAgent extends Agent { + + @ChatModelConnection + public static ResourceDescriptor pythonChatModelConnection() { + // In pure Python, the equivalent ResourceDescriptor would be: + // ResourceDescriptor( + // clazz=ResourceName.ChatModel.OLLAMA_CONNECTION, + // request_timeout=120.0 + // ) + return PythonResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.Python.OLLAMA_CONNECTION) + .addInitialArgument("request_timeout", 120.0) + .build(); + } + + @ChatModelSetup + public static ResourceDescriptor pythonChatModel() { + // In pure Python, the equivalent ResourceDescriptor would be: + // ResourceDescriptor( + // clazz=ResourceName.ChatModel.OLLAMA_SETUP, + // connection="pythonChatModelConnection", + // model="qwen3:8b", + // tools=["tool1", "tool2"], + // extract_reasoning=True + // ) + return PythonResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.Python.OLLAMA_SETUP) + .addInitialArgument("connection", "pythonChatModelConnection") + .addInitialArgument("model", "qwen3:8b") + .addInitialArgument("tools", List.of("tool1", "tool2")) + .addInitialArgument("extract_reasoning", true) + .build(); + } + + @Action(EventType.InputEvent) + public static void processInput(Event event, RunnerContext ctx) throws Exception { + InputEvent inputEvent = InputEvent.fromEvent(event); + ChatMessage userMessage = + new ChatMessage(MessageRole.USER, String.format("input: {%s}", inputEvent.getInput())); + ctx.sendEvent(new ChatRequestEvent("pythonChatModel", List.of(userMessage))); + } + + @Action(EventType.ChatResponseEvent) + public static void processResponse(Event event, RunnerContext ctx) + throws Exception { + ChatResponseEvent chatResponse = ChatResponseEvent.fromEvent(event); + String response = chatResponse.getResponse().getText(); + // Handle the LLM's response + // Process the response as needed for your use case + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +## Custom Providers + +{{< hint warning >}} +The custom provider APIs are experimental and unstable, subject to incompatible changes in future releases. +{{< /hint >}} + +If you want to use chat models not offered by the built-in providers, you can extend the base chat classes and implement your own! The chat model system is built around two main abstract classes: + +### BaseChatModelConnection + +Handles the connection to chat model services and provides the core chat functionality. + +{{< tabs "Custom BaseChatModelConnection" >}} + +{{< tab "Python" >}} +```python +class MyChatModelConnection(BaseChatModelConnection): + + def chat( + self, + messages: Sequence[ChatMessage], + tools: List[Tool] | None = None, + **kwargs: Any, + ) -> ChatResult: + # Core method: send messages to LLM and return response + # - messages: Input message sequence + # - tools: Optional list of tools available to the model + # - kwargs: Additional parameters from model_kwargs + # - Returns: ChatResult with the assistant message and available usage information + pass +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class MyChatModelConnection extends BaseChatModelConnection { + + /** + * Creates a new chat model connection. + * + * @param descriptor a resource descriptor contains the initial parameters + * @param resourceContext context for resolving resources (e.g., tools) by name and type + */ + public MyChatModelConnection( + ResourceDescriptor descriptor, ResourceContext resourceContext) { + super(descriptor, resourceContext); + // get custom arguments from descriptor + String endpoint = descriptor.getArgument("endpoint"); + ... + } + + + @Override + public ChatResult chat( + List messages, List tools, Map arguments) { + // Core method: send messages to LLM and return response + // - messages: Input message sequence + // - tools: Optional list of tools available to the model + // - arguments: Additional parameters from ChatModelSetup + // - Returns: ChatResult with the assistant message and available usage information + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + + +### BaseChatModelSetup + +The setup class acts as a high-level configuration interface that defines which connection to use and how to configure the chat model. + +{{< tabs "Prepare Agents Execution Environment" >}} + +{{< tab "Python" >}} +```python +class MyChatModelSetup(BaseChatModelSetup): + # Add your custom configuration fields here + + @property + def model_kwargs(self) -> Dict[str, Any]: + # Return model-specific configuration passed to chat() + # This dictionary is passed as **kwargs to the chat() method + return {"model": self.model, "temperature": 0.7, ...} +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class MyChatModelSetup extends BaseChatModelSetup { + // Add your custom configuration fields here + + @Override + public Map getParameters() { + Map params = new HashMap<>(); + params.put("model", model); + ... + // Return model-specific configuration passed to chat() + // This dictionary is passed as arguments to the chat() method + return params; + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +[Chat with Models]({{< ref "docs/development/calling-llms/chat" >}}) · [Built-in Chat Model Providers]({{< ref "docs/development/calling-llms/chat-models/providers" >}}) · [Model Routing]({{< ref "docs/development/calling-llms/model_routing" >}}) diff --git a/docs/content/docs/development/chat_models.md b/docs/content/docs/development/calling-llms/chat-models/providers.md similarity index 72% rename from docs/content/docs/development/chat_models.md rename to docs/content/docs/development/calling-llms/chat-models/providers.md index 5dcb6b0b12..525a44bda2 100644 --- a/docs/content/docs/development/chat_models.md +++ b/docs/content/docs/development/calling-llms/chat-models/providers.md @@ -1,7 +1,9 @@ --- -title: Chat Models -weight: 4 +title: "Built-in Chat Model Providers" +weight: 1 type: docs +aliases: + - /docs/development/calling-llms/declare-resources/providers/ --- -# Chat Models - -## Overview - -Chat models enable agents to communicate with Large Language Models (LLMs) for natural language understanding, reasoning, and generation. In Flink Agents, chat models act as the "brain" of your agents, processing input messages and generating intelligent responses based on context, prompts, and available tools. - -## Getting Started - -To use chat models in your agents, you need to define both a connection and setup using decorators/annotations, then interact with the model through events. - -### Resource Declaration - -Flink Agents provides decorators in python and annotations in java to simplify chat model setup within agents: - -#### @chat_model_connection/@ChatModelConnection - -The `@chat_model_connection` decorator or `@ChatModelConnection` annotation marks a method that creates a chat model connection. This is typically defined once and shared across multiple chat model setups. - -#### @chat_model_setup/@ChatModelSetup - -The `@chat_model_setup` decorator or `@ChatModelSetup` annotation marks a method that creates a chat model setup. This references a connection and adds chat-specific configuration like prompts and tools. - -### Chat Events - -Chat models communicate through built-in events: - -- **ChatRequestEvent**: Sent by actions to request a chat completion from the LLM -- **ChatResponseEvent**: Received by actions containing the LLM's response - -A `ChatRequestEvent` can name a model router instead of a chat model to pick the model per request; see [Model Routing]({{< ref "docs/development/model_routing" >}}). - -### Message content blocks - -Use `ChatMessage.blocks` to combine text and media in a single message, in the -order you want them presented. Supported block types are `TextBlock`, `ImageBlock`, -`AudioBlock`, `VideoBlock`, and `DocumentBlock`. Assistant messages can also contain -`ReasoningBlock` and `ToolCallBlock`; a tool message contains one `ToolResultBlock` -whose `call_id` matches the corresponding tool call and whose blocks hold the result. - -To read the complete content of a message or `ChatResult`, use `getBlocks()` (Java) -or `blocks` (Python). Use `getText()` (Java) or `text` (Python) when you only need -text: these accessors concatenate the message's text blocks and omit media, -reasoning, tool calls, and text inside tool results. - -For tools that return text or media, see -[Tool response content and metadata]({{< ref "docs/development/tool_use#tool-response-content-and-metadata" >}}). - -**Provider support:** The OpenAI Chat Completions integration, and the Azure OpenAI -and vLLM integrations built on it, send media blocks to the model; see -[Multimodal Input](#multimodal-input) for what each block becomes. Ollama sends -Base64 images; see its section. The Anthropic, Gemini, Amazon Bedrock, IBM -watsonx.ai, DashScope and OpenAI Responses integrations do not send media yet: a -message with a media block raises `UnsupportedContentBlockException` (Java) or -`UnsupportedContentBlockError` (Python). - -For example, a message can contain a question followed by an image: - -{{< tabs "Message content blocks" >}} - -{{< tab "Python" >}} -```python -from flink_agents.api.chat_message import ChatMessage, ImageBlock, TextBlock - -message = ChatMessage.user([ - TextBlock(text="Describe this image"), - ImageBlock.from_url("image/png", "https://example.org/image.png"), -]) -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -ChatMessage message = ChatMessage.user(List.of( - TextBlock.of("Describe this image"), - ImageBlock.fromUrl("image/png", "https://example.org/image.png"))); -``` -{{< /tab >}} - -{{< /tabs >}} - -Each media block requires a MIME type such as `image/png` and a media source. -Use `from_url` (Python) or `fromUrl` (Java) for an externally managed URL, as above. -For inline data, all four media block types provide two separate entry points: - -- `from_bytes(media_type, data)` (Python) or `fromBytes(mediaType, data)` (Java) - accepts non-empty raw `bytes` or `byte[]` and encodes them as standard Base64 - without line wrapping at construction. Java does not retain the caller's array. -- `from_base64(media_type, data)` or `fromBase64(mediaType, data)` accepts a - non-empty, already Base64-encoded string. It preserves the string without - encoding it again or validating its Base64 syntax. - -{{< tabs "Inline media" >}} -{{< tab "Python" >}} -```python -from pathlib import Path -from flink_agents.api.chat_message import ImageBlock - -image = ImageBlock.from_bytes("image/png", Path("image.png").read_bytes(), name="image.png") -# If a service already returned Base64, pass that string directly. -encoded_image = ImageBlock.from_base64("image/png", base64_from_service) -``` -{{< /tab >}} -{{< tab "Java" >}} -```java -import java.nio.file.Files; -import java.nio.file.Path; -import org.apache.flink.agents.api.chat.messages.ImageBlock; - -ImageBlock image = ImageBlock.fromBytes("image/png", Files.readAllBytes(Path.of("image.png"))); -// If a service already returned Base64, pass that string directly. -ImageBlock encodedImage = ImageBlock.fromBase64("image/png", base64FromService); -``` -{{< /tab >}} -{{< /tabs >}} - -The framework does not verify that the contents match the declared media type. -Choose a concrete block such as `ImageBlock`, rather than constructing the shared -`MediaBlock` base class. Both inline factories produce the same Base64 source -representation for serialization and provider conversion. - -Media blocks also accept optional metadata: `name` for a file name, `size_bytes` -for the media size in bytes, and `sha256` for a content checksum. - -To read only the text, use `.text` (Python) or `getText()` (Java). These concatenate -the text blocks and exclude media. - -### Usage Example - -Here's how to define and use chat models in a workflow agent: - -{{< tabs "Usage Example" >}} - -{{< tab "Python" >}} -```python -class MyAgent(Agent): - - @chat_model_connection - @staticmethod - def ollama_connection() -> ResourceDescriptor: - return ResourceDescriptor( - clazz=ResourceName.ChatModel.OLLAMA_CONNECTION, - base_url="http://localhost:11434", - request_timeout=30.0 - ) - - @chat_model_setup - @staticmethod - def ollama_chat_model() -> ResourceDescriptor: - return ResourceDescriptor( - clazz=ResourceName.ChatModel.OLLAMA_SETUP, - connection="ollama_connection", - model="qwen3:8b", - temperature=0.7 - ) - - @action(EventType.InputEvent) - @staticmethod - def process_input(event: Event, ctx: RunnerContext) -> None: - input_event = InputEvent.from_event(event) - # Create a chat request with user message - user_message = ChatMessage.user(f"input: {input_event.input}") - ctx.send_event( - ChatRequestEvent(model="ollama_chat_model", messages=[user_message]) - ) - - @action(EventType.ChatResponseEvent) - @staticmethod - def process_response(event: Event, ctx: RunnerContext) -> None: - chat_response = ChatResponseEvent.from_event(event) - response_content = chat_response.response.text - # Handle the LLM's response - # Process the response as needed for your use case -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyAgent extends Agent { - @ChatModelConnection - public static ResourceDescriptor ollamaConnection() { - return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_CONNECTION) - .addInitialArgument("endpoint", "http://localhost:11434") - .build(); - } - - @ChatModelSetup - public static ResourceDescriptor ollamaChatModel() { - return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) - .addInitialArgument("connection", "ollamaConnection") - .addInitialArgument("model", "qwen3:8b") - .build(); - } - - @Action(EventType.InputEvent) - public static void processInput(Event event, RunnerContext ctx) throws Exception { - InputEvent inputEvent = InputEvent.fromEvent(event); - ChatMessage userMessage = - new ChatMessage(MessageRole.USER, String.format("input: {%s}", inputEvent.getInput())); - ctx.sendEvent(new ChatRequestEvent("ollamaChatModel", List.of(userMessage))); - } - - @Action(EventType.ChatResponseEvent) - public static void processResponse(Event event, RunnerContext ctx) - throws Exception { - ChatResponseEvent chatResponse = ChatResponseEvent.fromEvent(event); - String response = chatResponse.getResponse().getText(); - // Handle the LLM's response - // Process the response as needed for your use case - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - - ## Built-in Providers ### Amazon Bedrock @@ -245,7 +31,7 @@ public class MyAgent extends Agent { Amazon Bedrock provides access to a wide range of foundation models from leading AI providers through a unified API. The Flink Agents Bedrock integration uses the [Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html), which provides a consistent interface across all supported models with native tool calling support. Authentication is handled via SigV4 using the AWS default credentials chain. No API keys are required. {{< hint info >}} -Amazon Bedrock is only supported in Java currently. To use Amazon Bedrock from Python agents, see [Using Cross-Language Providers](#using-cross-language-providers). +Amazon Bedrock is only supported in Java currently. To use Amazon Bedrock from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/calling-llms/chat-models#using-cross-language-providers" >}}). {{< /hint >}} #### Prerequisites @@ -667,7 +453,7 @@ Vertex AI support is experimental. The connection path has been smoke-tested dur {{< /hint >}} {{< hint info >}} -Gemini is only supported in Java currently. To use Gemini from Python agents, see [Using Cross-Language Providers](#using-cross-language-providers). +Gemini is only supported in Java currently. To use Gemini from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/calling-llms/chat-models#using-cross-language-providers" >}}). {{< /hint >}} #### Prerequisites @@ -881,7 +667,7 @@ public class MyAgent extends Agent { .addInitialArgument("extract_reasoning", true) .build(); } - + ... } ``` @@ -1074,7 +860,7 @@ The other built-in providers (Anthropic, Gemini, Amazon Bedrock, IBM watsonx.ai, #### Responses API {{< hint info >}} -Responses API is only supported in Java currently. To use OpenAI Responses API from Python agents, see [Using Cross-Language Providers](#using-cross-language-providers). +Responses API is only supported in Java currently. To use OpenAI Responses API from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/calling-llms/chat-models#using-cross-language-providers" >}}). {{< /hint >}} ##### OpenAIResponsesModelConnection Parameters @@ -1172,7 +958,7 @@ Model availability and specifications may change. Always check the official Open This integration connects to Alibaba Cloud Model Studio (Bailian) through the DashScope SDK to access chat models. {{< hint info >}} -The DashScope integration is currently supported in the Python API only. To use it from Java agents, see [Using Cross-Language Providers](#using-cross-language-providers). +The DashScope integration is currently supported in the Python API only. To use it from Java agents, see [Using Cross-Language Providers]({{< ref "docs/development/calling-llms/chat-models#using-cross-language-providers" >}}). {{< /hint >}} #### Prerequisites @@ -1511,279 +1297,3 @@ Some popular options include: {{< hint warning >}} Model availability and specifications may change. Always check the official IBM watsonx.ai documentation for the latest information before implementing in production. {{< /hint >}} - -## Using Cross-Language Providers - -Flink Agents supports cross-language chat model integration, allowing you to use chat models implemented in one language (Java or Python) from agents written in the other language. This is particularly useful when a chat model provider is only available in one language (e.g., the DashScope integration for Alibaba Cloud Model Studio is currently Python-only). - -{{< hint warning >}} -**Limitations:** - -- Cross-language resources are currently supported only when [running in Flink]({{< ref "docs/operations/deployment#run-in-flink" >}}), not in local development mode -- Complex object serialization between languages may have limitations -{{< /hint >}} - -### How To Use - -To use a resource implemented in another language, declare its implementation class with the corresponding cross-language resource descriptor: - -- **Using Java chat models in Python**: Use `JavaResourceDescriptor(clazz=ResourceName.ChatModel.Java.OLLAMA_CONNECTION, ...)` (or the corresponding setup class). -- **Using Python chat models in Java**: Use `PythonResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.Python.OLLAMA_CONNECTION)` (or the corresponding setup class). - - - -### Usage Example - -{{< tabs "Cross-Language Chat Model Usage Example" >}} - -{{< tab "Using Java Chat Model in Python" >}} -```python -class MyAgent(Agent): - - @chat_model_connection - @staticmethod - def java_chat_model_connection() -> ResourceDescriptor: - # In pure Java, the equivalent ResourceDescriptor would be: - # ResourceDescriptor.Builder - # .newBuilder(ResourceName.ChatModel.OLLAMA_CONNECTION) - # .addInitialArgument("endpoint", "http://localhost:11434") - # .addInitialArgument("requestTimeout", 120) - # .build(); - return JavaResourceDescriptor( - clazz=ResourceName.ChatModel.Java.OLLAMA_CONNECTION, - endpoint="http://localhost:11434", - requestTimeout=120, - ) - - - @chat_model_setup - @staticmethod - def java_chat_model() -> ResourceDescriptor: - # In pure Java, the equivalent ResourceDescriptor would be: - # ResourceDescriptor.Builder - # .newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) - # .addInitialArgument("connection", "java_chat_model_connection") - # .addInitialArgument("model", "qwen3:8b") - # .addInitialArgument("prompt", "my_prompt") - # .addInitialArgument("tools", List.of("my_tool1", "my_tool2")) - # .addInitialArgument("extractReasoning", true) - # .build(); - return JavaResourceDescriptor( - clazz=ResourceName.ChatModel.Java.OLLAMA_SETUP, - connection="java_chat_model_connection", - model="qwen3:8b", - prompt="my_prompt", - tools=["my_tool1", "my_tool2"], - extract_reasoning=True, - ) - - @action(EventType.InputEvent) - @staticmethod - def process_input(event: Event, ctx: RunnerContext) -> None: - input_event = InputEvent.from_event(event) - # Create a chat request with user message - user_message = ChatMessage.user(f"input: {input_event.input}") - ctx.send_event( - ChatRequestEvent(model="java_chat_model", messages=[user_message]) - ) - - @action(EventType.ChatResponseEvent) - @staticmethod - def process_response(event: Event, ctx: RunnerContext) -> None: - chat_response = ChatResponseEvent.from_event(event) - response_content = chat_response.response.text - # Handle the LLM's response - # Process the response as needed for your use case -``` -{{< /tab >}} - -{{< tab "Using Python Chat Model in Java" >}} -```java -public class MyAgent extends Agent { - - @ChatModelConnection - public static ResourceDescriptor pythonChatModelConnection() { - // In pure Python, the equivalent ResourceDescriptor would be: - // ResourceDescriptor( - // clazz=ResourceName.ChatModel.OLLAMA_CONNECTION, - // request_timeout=120.0 - // ) - return PythonResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.Python.OLLAMA_CONNECTION) - .addInitialArgument("request_timeout", 120.0) - .build(); - } - - @ChatModelSetup - public static ResourceDescriptor pythonChatModel() { - // In pure Python, the equivalent ResourceDescriptor would be: - // ResourceDescriptor( - // clazz=ResourceName.ChatModel.OLLAMA_SETUP, - // connection="pythonChatModelConnection", - // model="qwen3:8b", - // tools=["tool1", "tool2"], - // extract_reasoning=True - // ) - return PythonResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.Python.OLLAMA_SETUP) - .addInitialArgument("connection", "pythonChatModelConnection") - .addInitialArgument("model", "qwen3:8b") - .addInitialArgument("tools", List.of("tool1", "tool2")) - .addInitialArgument("extract_reasoning", true) - .build(); - } - - @Action(EventType.InputEvent) - public static void processInput(Event event, RunnerContext ctx) throws Exception { - InputEvent inputEvent = InputEvent.fromEvent(event); - ChatMessage userMessage = - new ChatMessage(MessageRole.USER, String.format("input: {%s}", inputEvent.getInput())); - ctx.sendEvent(new ChatRequestEvent("pythonChatModel", List.of(userMessage))); - } - - @Action(EventType.ChatResponseEvent) - public static void processResponse(Event event, RunnerContext ctx) - throws Exception { - ChatResponseEvent chatResponse = ChatResponseEvent.fromEvent(event); - String response = chatResponse.getResponse().getText(); - // Handle the LLM's response - // Process the response as needed for your use case - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - -## Custom Providers - -{{< hint warning >}} -The custom provider APIs are experimental and unstable, subject to incompatible changes in future releases. -{{< /hint >}} - -If you want to use chat models not offered by the built-in providers, you can extend the base chat classes and implement your own! The chat model system is built around two main abstract classes: - -### BaseChatModelConnection - -Handles the connection to chat model services and provides the core chat functionality. - -{{< tabs "Custom BaseChatModelConnection" >}} - -{{< tab "Python" >}} -```python -class MyChatModelConnection(BaseChatModelConnection): - - def chat( - self, - messages: Sequence[ChatMessage], - tools: List[Tool] | None = None, - **kwargs: Any, - ) -> ChatResult: - # Core method: send messages to LLM and return response - # - messages: Input message sequence - # - tools: Optional list of tools available to the model - # - kwargs: Additional parameters from model_kwargs - # - Returns: ChatResult with the assistant message and available usage information - pass -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyChatModelConnection extends BaseChatModelConnection { - - /** - * Creates a new chat model connection. - * - * @param descriptor a resource descriptor contains the initial parameters - * @param resourceContext context for resolving resources (e.g., tools) by name and type - */ - public MyChatModelConnection( - ResourceDescriptor descriptor, ResourceContext resourceContext) { - super(descriptor, resourceContext); - // get custom arguments from descriptor - String endpoint = descriptor.getArgument("endpoint"); - ... - } - - - @Override - public ChatResult chat( - List messages, List tools, Map arguments) { - // Core method: send messages to LLM and return response - // - messages: Input message sequence - // - tools: Optional list of tools available to the model - // - arguments: Additional parameters from ChatModelSetup - // - Returns: ChatResult with the assistant message and available usage information - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - - -### BaseChatModelSetup - -The setup class acts as a high-level configuration interface that defines which connection to use and how to configure the chat model. - -{{< tabs "Prepare Agents Execution Environment" >}} - -{{< tab "Python" >}} -```python -class MyChatModelSetup(BaseChatModelSetup): - # Add your custom configuration fields here - - @property - def model_kwargs(self) -> Dict[str, Any]: - # Return model-specific configuration passed to chat() - # This dictionary is passed as **kwargs to the chat() method - return {"model": self.model, "temperature": 0.7, ...} -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyChatModelSetup extends BaseChatModelSetup { - // Add your custom configuration fields here - - @Override - public Map getParameters() { - Map params = new HashMap<>(); - params.put("model", model); - ... - // Return model-specific configuration passed to chat() - // This dictionary is passed as arguments to the chat() method - return params; - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - -## Built-in Events and Actions - -`ChatResponseEvent` reports whether a chat request succeeded or failed: - -- On success, `event.getResponse()` (Java) or `event.response` (Python) returns a - `ChatResult`. It contains the assistant message and, when available, the model - name, response ID, token usage, and finish reason. Use `getText()` / `text` to - read the answer as text, or `getBlocks()` / `blocks` to include all content. -- On failure, `event.getError()` (Java) or `event.error` (Python) provides an error - description. Check `event.isFailed()` / `event.is_failed` before reading the - response: accessing the response of a failed event raises an exception. - -If you requested structured output, read the parsed result from -`event.getStructuredOutput()` (Java) or `event.structured_output` (Python). - -Applications that need to recover from a failed chat request should handle the -failed event explicitly. The built-in ReAct agent propagates the failure. - -Ordinary provider, resource-resolution, response-validation and structured-output errors produce a failed response after configured retries and routing fallbacks are exhausted. `max-retries` controls the number of additional attempts (default: 0), and `retry-wait-interval` controls exponential backoff. These settings also apply to routing judge calls. A judge call failure fails the request rather than selecting the default model; a normal abstaining verdict still selects the default. Intermediate attempts and Tool rounds do not emit terminal responses, and the final response always refers to the initial request ID. - -Cancellation, interruption and fatal errors propagate. Durable persistence/recovery, required Tool context, and event-delivery failures also propagate to Flink rather than becoming Chat failures. A response cannot be guaranteed during cancellation or an unrecoverable runtime failure. Recovery may replay physical events; consumers must correlate by request ID and complete idempotently. - -Create results with `ChatResponseEvent.success(...)` or `ChatResponseEvent.failed(...)`. This is a breaking event and durable-call format change: `status` is mandatory, and old events and old persisted Chat/routing call results are not supported. Update custom producers, consumers, and stored fixtures together. - -The built-in `chat_model_action` listens to `ChatRequestEvent` and `ToolResponseEvent`. To request a chat completion, send a `ChatRequestEvent`. If the model returns a final answer, the action sends a `ChatResponseEvent`. - -If the model asks to call tools, `chat_model_action` sends a `ToolRequestEvent` instead of a final `ChatResponseEvent`. After the tools finish, it receives the matching `ToolResponseEvent`, appends the tool results to the chat history, and calls the model again. This loop continues until the model returns a final response. For details on how tools are executed, see [Built-in Events and Actions in Tool Use]({{< ref "docs/development/tool_use#built-in-events-and-actions" >}}). When the request names a model router, `chat_model_action` selects the model first and emits a `ModelRoutingEvent`; see [Model Routing]({{< ref "docs/development/model_routing#observability" >}}). diff --git a/docs/content/docs/development/calling-llms/chat.md b/docs/content/docs/development/calling-llms/chat.md new file mode 100644 index 0000000000..f747c47307 --- /dev/null +++ b/docs/content/docs/development/calling-llms/chat.md @@ -0,0 +1,319 @@ +--- +title: "Chat with Models" +weight: 3 +type: docs +--- + + +## Overview + +Agents interact with chat models by sending messages and receiving responses. A request identifies a registered chat model and supplies the messages for it to process. The response contains the model's answer or reports a failure. + +This chapter explains how chat requests and responses flow through an Agent, then walks through a text-only example and handling successful and failed responses. It then covers multimodal messages and the tool-calling loop. + +## Chat Flow {#chat-events} + +Chat models communicate through built-in events: + +- **ChatRequestEvent**: Sent by actions to request a chat completion from the LLM +- **ChatResponseEvent**: Received by actions containing the LLM's response + +An Action in your Agent sends a `ChatRequestEvent`. The built-in `chat_model_action` invokes the named model and emits a `ChatResponseEvent` when the request completes. A response Action in your Agent handles that event; you do not need to implement the model invocation yourself. + +A basic text-answering Agent follows this flow: + +```text +InputEvent → request Action → ChatRequestEvent + → chat_model_action → ChatResponseEvent → response Action → OutputEvent +``` + +A `ChatRequestEvent` can name a model router instead of a chat model. The built-in action selects the model first and emits a `ModelRoutingEvent`; see [Model Routing]({{< ref "docs/development/calling-llms/model_routing" >}}). + +## Request and Response Example {#usage-example} + +The Agent below declares an Ollama connection and model, sends each input as a user message, and emits the model's text answer as an `OutputEvent`. The connection and model declarations are included to show which resource names the request references. + +The example uses Ollama at `http://localhost:11434` with `qwen3:8b`; adjust the endpoint and model to match your configuration. + +This example raises an exception when the chat response reports a failure, causing the Action to fail. Applications that need a fallback answer can instead handle the failed event explicitly before accessing its response. + +{{< tabs "Usage Example" >}} + +{{< tab "Python" >}} +```python +from flink_agents.api.agents.agent import Agent +from flink_agents.api.chat_message import ChatMessage +from flink_agents.api.decorators import action, chat_model_connection, chat_model_setup +from flink_agents.api.events.chat_event import ChatRequestEvent, ChatResponseEvent +from flink_agents.api.events.event import Event, InputEvent, OutputEvent +from flink_agents.api.events.event_type import EventType +from flink_agents.api.resource import ResourceDescriptor, ResourceName +from flink_agents.api.runner_context import RunnerContext + + +class MyAgent(Agent): + + @chat_model_connection + @staticmethod + def ollama_connection() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.ChatModel.OLLAMA_CONNECTION, + base_url="http://localhost:11434", + request_timeout=30.0 + ) + + @chat_model_setup + @staticmethod + def ollama_chat_model() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.ChatModel.OLLAMA_SETUP, + connection="ollama_connection", + model="qwen3:8b", + temperature=0.7 + ) + + @action(EventType.InputEvent) + @staticmethod + def process_input(event: Event, ctx: RunnerContext) -> None: + input_event = InputEvent.from_event(event) + # Create a chat request with user message + user_message = ChatMessage.user(f"input: {input_event.input}") + ctx.send_event( + ChatRequestEvent(model="ollama_chat_model", messages=[user_message]) + ) + + @action(EventType.ChatResponseEvent) + @staticmethod + def process_response(event: Event, ctx: RunnerContext) -> None: + chat_response = ChatResponseEvent.from_event(event) + if chat_response.is_failed: + raise RuntimeError(chat_response.error) + ctx.send_event(OutputEvent(output=chat_response.response.text)) +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +import java.util.List; + +import org.apache.flink.agents.api.Event; +import org.apache.flink.agents.api.EventType; +import org.apache.flink.agents.api.InputEvent; +import org.apache.flink.agents.api.OutputEvent; +import org.apache.flink.agents.api.agents.Agent; +import org.apache.flink.agents.api.annotation.Action; +import org.apache.flink.agents.api.annotation.ChatModelConnection; +import org.apache.flink.agents.api.annotation.ChatModelSetup; +import org.apache.flink.agents.api.chat.messages.ChatMessage; +import org.apache.flink.agents.api.chat.messages.MessageRole; +import org.apache.flink.agents.api.context.RunnerContext; +import org.apache.flink.agents.api.event.ChatRequestEvent; +import org.apache.flink.agents.api.event.ChatResponseEvent; +import org.apache.flink.agents.api.resource.ResourceDescriptor; +import org.apache.flink.agents.api.resource.ResourceName; + +public class MyAgent extends Agent { + @ChatModelConnection + public static ResourceDescriptor ollamaConnection() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_CONNECTION) + .addInitialArgument("endpoint", "http://localhost:11434") + .build(); + } + + @ChatModelSetup + public static ResourceDescriptor ollamaChatModel() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) + .addInitialArgument("connection", "ollamaConnection") + .addInitialArgument("model", "qwen3:8b") + .build(); + } + + @Action(EventType.InputEvent) + public static void processInput(Event event, RunnerContext ctx) throws Exception { + InputEvent inputEvent = InputEvent.fromEvent(event); + ChatMessage userMessage = + new ChatMessage(MessageRole.USER, String.format("input: %s", inputEvent.getInput())); + ctx.sendEvent(new ChatRequestEvent("ollamaChatModel", List.of(userMessage))); + } + + @Action(EventType.ChatResponseEvent) + public static void processResponse(Event event, RunnerContext ctx) + throws Exception { + ChatResponseEvent chatResponse = ChatResponseEvent.fromEvent(event); + if (chatResponse.isFailed()) { + throw new IllegalStateException(chatResponse.getError()); + } + ctx.sendEvent(new OutputEvent(chatResponse.getResponse().getText())); + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +In the request, `ollama_chat_model` (Python) or `ollamaChatModel` (Java) names the declared model setup. The provider model identifier, `qwen3:8b`, belongs in that setup. The response Action emits an `OutputEvent` so the Agent's answer can continue through the Flink pipeline. + +## Handle Success and Failure + +`ChatResponseEvent` reports whether a chat request succeeded or failed: + +- On success, `event.getResponse()` (Java) or `event.response` (Python) returns a + `ChatResult`. It contains the assistant message and, when available, the model + name, response ID, token usage, and finish reason. Use `getText()` / `text` to + read the answer as text, or `getBlocks()` / `blocks` to include all content. +- On failure, `event.getError()` (Java) or `event.error` (Python) provides an error + description. Check `event.isFailed()` / `event.is_failed` before reading the + response: accessing the response of a failed event raises an exception. + +If you requested structured output, read the parsed result from +`event.getStructuredOutput()` (Java) or `event.structured_output` (Python). + +Applications that need to recover from a failed chat request should handle the +failed event explicitly. The built-in ReAct agent propagates the failure. + +### Failures and Recovery + +Ordinary provider, resource-resolution, response-validation and structured-output errors produce a failed response after configured retries and routing fallbacks are exhausted. `max-retries` controls the number of additional attempts (default: 0), and `retry-wait-interval` controls exponential backoff. These settings also apply to routing judge calls. A judge call failure fails the request rather than selecting the default model; a normal abstaining verdict still selects the default. Intermediate attempts and Tool rounds do not emit terminal responses, and the final response always refers to the initial request ID. + +Cancellation, interruption and fatal errors propagate. Durable persistence/recovery, required Tool context, and event-delivery failures also propagate to Flink rather than becoming Chat failures. A response cannot be guaranteed during cancellation or an unrecoverable runtime failure. Recovery may replay physical events; consumers must correlate by request ID and complete idempotently. + +### Custom Response Producers + +Create results with `ChatResponseEvent.success(...)` or `ChatResponseEvent.failed(...)`. This is a breaking event and durable-call format change: `status` is mandatory, and old events and old persisted Chat/routing call results are not supported. Update custom producers, consumers, and stored fixtures together. + +## Multimodal Messages {#message-content-blocks} + +Multimodal messages combine text with images, audio, video, or documents. Use +`ChatMessage.blocks` to arrange these contents in the order you want them presented. Supported block types are `TextBlock`, `ImageBlock`, +`AudioBlock`, `VideoBlock`, and `DocumentBlock`. Assistant messages can also contain +`ReasoningBlock` and `ToolCallBlock`; a tool message contains one `ToolResultBlock` +whose `call_id` matches the corresponding tool call and whose blocks hold the result. + +To read the complete content of a message or `ChatResult`, use `getBlocks()` (Java) +or `blocks` (Python). Use `getText()` (Java) or `text` (Python) when you only need +text: these accessors concatenate the message's text blocks and omit media, +reasoning, tool calls, and text inside tool results. + +For tools that return text or media, see +[Tool response content and metadata]({{< ref "docs/development/calling-llms/tool_use#tool-response-content-and-metadata" >}}). + +**Provider support:** The OpenAI Chat Completions integration, and the Azure OpenAI +and vLLM integrations built on it, send media blocks to the model; see +[Multimodal Input]({{< ref "docs/development/calling-llms/chat-models/providers#multimodal-input" >}}) for what each block becomes. Ollama sends +Base64 images; see the [Ollama provider section]({{< ref "docs/development/calling-llms/chat-models/providers#ollama" >}}). The Anthropic, Gemini, Amazon Bedrock, IBM +watsonx.ai, DashScope and OpenAI Responses integrations do not send media yet: a +message with a media block raises `UnsupportedContentBlockException` (Java) or +`UnsupportedContentBlockError` (Python). + +For example, a message can contain a question followed by an image: + +{{< tabs "Multimodal Messages" >}} + +{{< tab "Python" >}} +```python +from flink_agents.api.chat_message import ChatMessage, ImageBlock, TextBlock + +message = ChatMessage.user([ + TextBlock(text="Describe this image"), + ImageBlock.from_url("image/png", "https://example.org/image.png"), +]) +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +ChatMessage message = ChatMessage.user(List.of( + TextBlock.of("Describe this image"), + ImageBlock.fromUrl("image/png", "https://example.org/image.png"))); +``` +{{< /tab >}} + +{{< /tabs >}} + +Each media block requires a MIME type such as `image/png` and a media source. +Use `from_url` (Python) or `fromUrl` (Java) for an externally managed URL, as above. +For inline data, all four media block types provide two separate entry points: + +- `from_bytes(media_type, data)` (Python) or `fromBytes(mediaType, data)` (Java) + accepts non-empty raw `bytes` or `byte[]` and encodes them as standard Base64 + without line wrapping at construction. Java does not retain the caller's array. +- `from_base64(media_type, data)` or `fromBase64(mediaType, data)` accepts a + non-empty, already Base64-encoded string. It preserves the string without + encoding it again or validating its Base64 syntax. + +{{< tabs "Inline media" >}} +{{< tab "Python" >}} +```python +from pathlib import Path +from flink_agents.api.chat_message import ImageBlock + +image = ImageBlock.from_bytes("image/png", Path("image.png").read_bytes(), name="image.png") +# If a service already returned Base64, pass that string directly. +encoded_image = ImageBlock.from_base64("image/png", base64_from_service) +``` +{{< /tab >}} +{{< tab "Java" >}} +```java +import java.nio.file.Files; +import java.nio.file.Path; +import org.apache.flink.agents.api.chat.messages.ImageBlock; + +ImageBlock image = ImageBlock.fromBytes("image/png", Files.readAllBytes(Path.of("image.png"))); +// If a service already returned Base64, pass that string directly. +ImageBlock encodedImage = ImageBlock.fromBase64("image/png", base64FromService); +``` +{{< /tab >}} +{{< /tabs >}} + +The framework does not verify that the contents match the declared media type. +Choose a concrete block such as `ImageBlock`, rather than constructing the shared +`MediaBlock` base class. Both inline factories produce the same Base64 source +representation for serialization and provider conversion. + +Media blocks also accept optional metadata: `name` for a file name, `size_bytes` +for the media size in bytes, and `sha256` for a content checksum. + +To read only the text, use `.text` (Python) or `getText()` (Java). These concatenate +the text blocks and exclude media. + +## Tool Calling {#built-in-events-and-actions} + +A chat model can use tools to perform operations while answering a request. To enable this, define or register the tools and reference their names in the model setup's `tools` list. See [Tool Use]({{< ref "docs/development/calling-llms/tool_use" >}}) for configuration and examples. + +Send a `ChatRequestEvent` as usual. The model decides whether to answer directly or request tool calls: + +```text +ChatRequestEvent → chat_model_action → ChatResponseEvent + │ ↑ + ToolRequestEvent │ │ ToolResponseEvent + ↓ │ + tool_call_action +``` + +When the model requests tools, the framework executes them, adds their results to the conversation, and calls the model again. This cycle continues until the model produces a final answer or the chat request fails. Your Agent handles the final `ChatResponseEvent` using the [success and failure handling](#handle-success-and-failure) described above; intermediate tool calls do not produce a final chat response. + +The built-in `chat_model_action` and `tool_call_action` coordinate this cycle. For this flow, you do not need to send or handle `ToolRequestEvent` and `ToolResponseEvent` yourself. For details on tool execution and results, see [Built-in Events and Actions in Tool Use]({{< ref "docs/development/calling-llms/tool_use#built-in-events-and-actions" >}}). + +## Next Steps + +- Use [Prompts]({{< ref "docs/development/calling-llms/prompts" >}}) to reuse message templates across requests. +- Add [Tool Use]({{< ref "docs/development/calling-llms/tool_use" >}}) or [Skills]({{< ref "docs/development/calling-llms/skills" >}}) when the model needs to perform operations beyond generating an answer. +- Use [Model Routing]({{< ref "docs/development/calling-llms/model_routing" >}}) to choose among registered models for each request. +- Consult [Built-in Chat Model Providers]({{< ref "docs/development/calling-llms/chat-models/providers" >}}) for provider-specific configuration and media support. diff --git a/docs/content/docs/development/mcp.md b/docs/content/docs/development/calling-llms/mcp.md similarity index 96% rename from docs/content/docs/development/mcp.md rename to docs/content/docs/development/calling-llms/mcp.md index 27e855e0b0..3636559072 100644 --- a/docs/content/docs/development/mcp.md +++ b/docs/content/docs/development/calling-llms/mcp.md @@ -1,7 +1,9 @@ --- -title: MCP -weight: 11 +title: "MCP" +weight: 6 type: docs +aliases: + - /docs/development/mcp/ --- + +- [Overview]({{< ref "docs/development/create-agent/overview" >}}) +- [Actions]({{< ref "docs/development/create-agent/actions" >}}) +- [Events]({{< ref "docs/development/create-agent/events" >}}) +- [YAML API]({{< ref "docs/development/create-agent/yaml" >}}) diff --git a/docs/content/docs/development/workflow_agent.md b/docs/content/docs/development/create-agent/actions.md similarity index 55% rename from docs/content/docs/development/workflow_agent.md rename to docs/content/docs/development/create-agent/actions.md index 1fdd05aa88..1c5b968da3 100644 --- a/docs/content/docs/development/workflow_agent.md +++ b/docs/content/docs/development/create-agent/actions.md @@ -1,6 +1,6 @@ --- -title: Workflow Agent -weight: 1 +title: "Actions" +weight: 2 type: docs --- -## Overview - -A workflow style agent in Flink-Agents is an agent whose reasoning and behavior are organized as a directed workflow of modular steps, called actions, connected by events. This design is inspired by the need to orchestrate complex, multi-stage tasks in a transparent, extensible, and data-centric way, leveraging Apache Flink's streaming architecture. - -In Flink-Agents, a workflow agent is a class that inherits from the `Agent` base class. Its workflow -is defined by actions: functions decorated with `@action(...)` in Python or methods annotated with -`@Action(...)` in Java. Each action declares one or more trigger conditions, which may be exact event -types or condition expressions. Actions consume events, perform reasoning or tool calls, and emit new -events that may trigger other actions. This event-driven workflow forms a directed graph that may -contain cycles. Each node is an action, and each edge represents an event flow between actions. - -A workflow agent is well-suited for scenarios where the solution requires explicit orchestration, branching, or multi-step reasoning, such as data enrichment, multi-tool pipelines, or complex business logic. - -{{< hint info >}} -For guidance on choosing Java or Python, see [Should I choose Java or Python?]({{< ref "docs/faq/faq#q3-should-i-choose-java-or-python" >}}). -{{< /hint >}} - -## Workflow Agent Example - -{{< tabs "Workflow Agent Example" >}} - -{{< tab "Python" >}} -```python -class ReviewAnalysisAgent(Agent): - """An agent that uses a large language model (LLM) to analyze product reviews - and generate a satisfaction score and potential reasons for dissatisfaction. - - This agent receives a product review and produces a satisfaction score and a list - of reasons for dissatisfaction. It handles prompt construction, LLM interaction, - and output parsing. - """ - - @prompt - @staticmethod - def review_analysis_prompt() -> Prompt: - """Prompt for review analysis.""" - return review_analysis_prompt - - @tool - @staticmethod - def notify_shipping_manager(id: str, review: str) -> None: - """Notify the shipping manager when product received a negative review due to - shipping damage. - - Parameters - ---------- - id : str - The id of the product that received a negative review due to shipping damage - review: str - The negative review content - """ - # reuse the declared function, but for parsing the tool metadata, we write doc - # string here again. - notify_shipping_manager(id=id, review=review) - - @chat_model_setup - @staticmethod - def review_analysis_model() -> ResourceDescriptor: - """ChatModel which focus on review analysis.""" - return ResourceDescriptor( - clazz=ResourceName.ChatModel.OLLAMA_SETUP, - connection="ollama_server", - model="qwen3:8b", - prompt="review_analysis_prompt", - tools=["notify_shipping_manager"], - extract_reasoning=True, - ) - - @action(EventType.InputEvent) - @staticmethod - def process_input(event: Event, ctx: RunnerContext) -> None: - """Process input event and send chat request for review analysis.""" - input_event = InputEvent.from_event(event) - input: ProductReview = input_event.input - ctx.short_term_memory.set("id", input.id) - - content = f""" - "id": {input.id}, - "review": {input.review} - """ - msg = ChatMessage(role=MessageRole.USER) - ctx.send_event( - ChatRequestEvent( - model="review_analysis_model", - messages=[msg], - prompt_args={"input": content}, - ) - ) - - @action(EventType.ChatResponseEvent) - @staticmethod - def process_chat_response(event: Event, ctx: RunnerContext) -> None: - """Process chat response event and send output event.""" - chat_response = ChatResponseEvent.from_event(event) - try: - json_content = json.loads(chat_response.response.text) - ctx.send_event( - OutputEvent( - output=ProductReviewAnalysisRes( - id=ctx.short_term_memory.get("id"), - score=json_content["score"], - reasons=json_content["reasons"], - ) - ) - ) - except Exception: - logging.exception( - f"Error processing chat response {chat_response.response.text}" - ) - - # To fail the agent, you can raise an exception here. -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -/** - * An agent that uses a large language model (LLM) to analyze product reviews and generate a - * satisfaction score and potential reasons for dissatisfaction. - * - *

This agent receives a product review and produces a satisfaction score and a list of reasons - * for dissatisfaction. It handles prompt construction, LLM interaction, and output parsing. - */ -public class ReviewAnalysisAgent extends Agent { - - private static final ObjectMapper MAPPER = new ObjectMapper(); - - @Prompt - public static org.apache.flink.agents.api.prompt.Prompt reviewAnalysisPrompt() { - return REVIEW_ANALYSIS_PROMPT; - } - - @ChatModelSetup - public static ResourceDescriptor reviewAnalysisModel() { - return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) - .addInitialArgument("connection", "ollamaChatModelConnection") - .addInitialArgument("model", "qwen3:8b") - .addInitialArgument("prompt", "reviewAnalysisPrompt") - .addInitialArgument("tools", Collections.singletonList("notifyShippingManager")) - .addInitialArgument("extract_reasoning", true) - .build(); - } - - /** - * Tool for notifying the shipping manager when product received a negative review due to - * shipping damage. - * - * @param id The id of the product that received a negative review due to shipping damage - * @param review The negative review content - */ - @Tool( - description = - "Notify the shipping manager when product received a negative review due to shipping damage.") - public static void notifyShippingManager( - @ToolParam(name = "id") String id, @ToolParam(name = "review") String review) { - CustomTypesAndResources.notifyShippingManager(id, review); - } - - /** Process input event and send chat request for review analysis. */ - @Action(EventType.InputEvent) - public static void processInput(Event event, RunnerContext ctx) throws Exception { - InputEvent inputEvent = InputEvent.fromEvent(event); - String input = (String) inputEvent.getInput(); - MAPPER.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); - CustomTypesAndResources.ProductReview inputObj = - MAPPER.readValue(input, CustomTypesAndResources.ProductReview.class); - - ctx.getShortTermMemory().set("id", inputObj.getId()); - - String content = - String.format( - "{\n" + "\"id\": %s,\n" + "\"review\": \"%s\"\n" + "}", - inputObj.getId(), inputObj.getReview()); - ChatMessage msg = new ChatMessage(MessageRole.USER, ""); - - ctx.sendEvent( - new ChatRequestEvent( - "reviewAnalysisModel", List.of(msg), Map.of("input", content), null)); - } - - @Action(EventType.ChatResponseEvent) - public static void processChatResponse(Event event, RunnerContext ctx) - throws Exception { - ChatResponseEvent chatResponse = ChatResponseEvent.fromEvent(event); - JsonNode jsonNode = MAPPER.readTree(chatResponse.getResponse().getText()); - JsonNode scoreNode = jsonNode.findValue("score"); - JsonNode reasonsNode = jsonNode.findValue("reasons"); - if (scoreNode == null || reasonsNode == null) { - throw new IllegalStateException( - "Invalid response from LLM: missing 'score' or 'reasons' field."); - } - List result = new ArrayList<>(); - if (reasonsNode.isArray()) { - for (JsonNode node : reasonsNode) { - result.add(node.asText()); - } - } - - ctx.sendEvent( - new OutputEvent( - new CustomTypesAndResources.ProductReviewAnalysisRes( - ctx.getShortTermMemory().get("id").getValue().toString(), - scoreNode.asInt(), - result))); - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - ## Action An action is a piece of code that can be executed. It declares one or more trigger conditions and is @@ -766,203 +555,3 @@ public class MyAgent extends Agent { - Cross-language actions are currently supported only when [running in Flink]({{< ref "docs/operations/deployment#run-in-flink" >}}), not in local development mode - Complex object serialization between languages may have limitations {{< /hint >}} - -## Event - -Events are JSON-serializable messages passed between actions. Every event has a `type` string and an -`attributes` map that carries its payload. Action routing can match the exact event type or evaluate -a condition expression against the event data. One event may trigger multiple actions. - -### Special Events - -* `InputEvent`: Generated by the framework when an input record arrives. The record is available in - the event's `input` attribute. Actions whose trigger conditions match an `InputEvent` are the - agent's entry points. -* `OutputEvent`: When an action sends this event, the framework emits its `output` attribute - downstream as an agent output. It bypasses action matching. - -### Unified Event - -For simple cases, users can pass data between actions directly using `Event` with a custom `type` and `attributes`, without needing to define a subclass. For more structured events, see [Custom Event Subclasses](#custom-event-subclasses) below. - -{{< tabs "Unified Event" >}} - -{{< tab "Python" >}} -```python -# Send a unified event from one action -@action(EventType.InputEvent) -@staticmethod -def create_my_event(event: Event, ctx: RunnerContext) -> None: - ctx.send_event( - Event(type="my_event", attributes={"field1": "test", "field2": 42}) - ) - -# Consume it in another action -@action("my_event") -@staticmethod -def handle_my_event(event: Event, ctx: RunnerContext) -> None: - field1: str = event.get_attr("field1") - field2: int = event.get_attr("field2") -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -// Send a unified event from one action -@Action(EventType.InputEvent) -public static void createMyEvent(Event event, RunnerContext ctx) { - ctx.sendEvent(new Event("my_event", Map.of("field1", "test", "field2", 42))); -} - -// Consume it in another action -@Action("my_event") -public static void handleMyEvent(Event event, RunnerContext ctx) { - String field1 = (String) event.getAttr("field1"); - int field2 = (int) event.getAttr("field2"); -} -``` -{{< /tab >}} - -{{< /tabs >}} - -{{< hint info >}} -`upstreamEventId` and `upstreamActionName` (`upstream_event_id` and -`upstream_action_name` in Python) are framework-managed lineage metadata, set when an Action -emits the Event. User code should keep user data in `attributes`. `sendEvent` / `send_event` -rejects an Event that already carries lineage, such as an Event received from another Action or -a typed reconstruction of one: emit a new Event instead. Deserialization and typed -reconstruction keep the lineage an Event already has; after recovery, restored outputs are bound -to the replayed trigger again. -{{< /hint >}} - -### JSON Serialization - -Events are serialized as JSON when passed between Python actions or across the Java-Python boundary. When an event crosses that boundary, its **type** and its **payload** are handled differently. - -**Built-in event types are restored automatically.** At the JSON boundary (`Event.fromJson` in Java, `Event.from_json` in Python) the framework restores every built-in event type to its concrete subclass and validates its fixed attribute schema. So inside an action, a dispatched `InputEvent`, `ChatRequestEvent`, `ChatResponseEvent`, or any other built-in event is already its concrete type: use it directly (with a cast in Java) instead of calling `from_event` / `fromEvent`. Those reconstruction methods stay available and idempotent, so code that still calls them keeps working. - -**Custom event types stay generic.** A user-defined event type is not known to the framework, so it arrives as a generic `Event`; call your subclass's `from_event` / `fromEvent` to reconstruct it (see [Custom Event Subclasses](#custom-event-subclasses)). - -**Arbitrary payload values are not reconstructed.** Restoration recovers the event's type and its typed built-in attributes (for example, `ChatRequestEvent.messages` become typed `ChatMessage` objects), but it does not rebuild arbitrary payload values. Non-trivial values nested in `attributes` — such as Pydantic models, or the record carried by `InputEvent.input` — lose their type information and arrive as plain `dict` / `Map` objects, so you must reconstruct them yourself: - -```python -# `event` is already a concrete InputEvent (auto-restored at the JSON boundary); -# only its opaque `input` payload needs manual reconstruction. -input_data = ItemData.model_validate(event.input) -``` - -### Custom Event Subclasses - -Users can also define custom event subclasses for reusable, structured events. Data should be stored in the `attributes` map, and the subclass must implement a `from_event` / `fromEvent` factory method that validates required attributes and reconstructs typed objects from the deserialized data. - -{{< tabs "Custom Event" >}} - -{{< tab "Python" >}} -```python -class MyEvent(Event): - EVENT_TYPE: ClassVar[str] = "my_event" - - def __init__(self, value: str) -> None: - super().__init__(type=MyEvent.EVENT_TYPE, attributes={"value": value}) - - @classmethod - @override - def from_event(cls, event: Event) -> "MyEvent": - assert "value" in event.attributes - result = cls(value=event.attributes["value"]) - return result.reconstruct_from(event) - - @property - def value(self) -> str: - return self.get_attr("value") -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyEvent extends Event { - public static final String EVENT_TYPE = "my_event"; - - public MyEvent(String value) { - super(EVENT_TYPE); - setAttr("value", value); - } - - @JsonCreator - public MyEvent( - @JsonProperty("id") UUID id, - @JsonProperty("attributes") Map attributes) { - super(id, EVENT_TYPE, attributes); - } - - public static MyEvent fromEvent(Event event) { - return reconstructFrom(event, MyEvent::new); - } - - public String getValue() { - return (String) getAttr("value"); - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - -{{< hint warning >}} -Python Event IDs are immutable. Custom `from_event` implementations written against earlier -versions may assign `result.id = event.id`; that assignment now raises a Pydantic -`ValidationError`. Replace `result.id = event.id` followed by `return result` with -`return result.reconstruct_from(event)`. - -Typed reconstruction represents the same Event occurrence, so it must preserve the base Event's -identity and framework-managed metadata: - -- **Python**: return `reconstruct_from(event)` after constructing the typed object. - It returns a new typed object with the Event's UUIDv4 `id`, `upstream_event_id`, and - `upstream_action_name`; the returned Event's `id` remains immutable. -- **Java**: implement the typed constructor that accepts `(UUID id, Map - attributes)`, then have `fromEvent` return - `reconstructFrom(event, MyEvent::new)`. The framework supplies the original ID and - attributes, validates that the ID is preserved, and carries over `sourceTimestamp`, - `upstreamEventId`, and `upstreamActionName`. -{{< /hint >}} - -{{< hint info >}} -All attribute values must be JSON-serializable. In Python, this means `BaseModel`-serializable or primitive types. In Java, values must be Jackson-serializable. -{{< /hint >}} - -{{< hint warning >}} -When defining a custom `Event` subclass in Java, annotate its `(UUID id, Map attributes)` constructor with `@JsonCreator`, and annotate both parameters with `@JsonProperty`. Durable execution stores events in `ActionState` and uses Jackson to restore concrete event subclasses during recovery. Without these annotations, recovery deserialization fails because the base `Event` constructor's `@JsonCreator` is not inherited by subclasses. - -If the subclass stores typed objects in `attributes`, convert them back to their typed forms inside this constructor too, as the built-in events do. For example, `ChatResponseEvent` restores its typed attributes like this: - -```java -@JsonCreator -public ChatResponseEvent( - @JsonProperty("id") UUID id, - @JsonProperty("attributes") Map attributes) { - super(id, EVENT_TYPE, normalizeAttributes(attributes)); -} - -/** Converts nested attributes back to their typed forms. */ -private static Map normalizeAttributes(Map attributes) { - Object rawId = attributes.get("request_id"); - if (rawId instanceof String) { - attributes.put("request_id", UUID.fromString((String) rawId)); - } - Object rawResponse = attributes.get("response"); - if (rawResponse instanceof Map) { - attributes.put("response", MAPPER.convertValue(rawResponse, ChatResult.class)); - } - return attributes; -} -``` -{{< /hint >}} - -## Built-in Events and Actions - -There are several built-in `Event` and `Action` in Flink-Agents: -* See [Chat Models]({{< ref "docs/development/chat_models#built-in-events-and-actions" >}}) for how to chat with a LLM leveraging built-in action and events. -* See [Tool Use]({{< ref "docs/development/tool_use#built-in-events-and-actions" >}}) for how to programmatically use a tool leveraging built-in action and events. -* See [Vector Stores]({{< ref "docs/development/vector_stores#built-in-events-and-actions" >}}) for how to retrieve context from vector stores leveraging built-in action and events. -* See [Model Routing]({{< ref "docs/development/model_routing#observability" >}}) for how a chat request selects between several models and how routing decisions are reported. diff --git a/docs/content/docs/development/create-agent/events.md b/docs/content/docs/development/create-agent/events.md new file mode 100644 index 0000000000..80c5e2c8d9 --- /dev/null +++ b/docs/content/docs/development/create-agent/events.md @@ -0,0 +1,223 @@ +--- +title: "Events" +weight: 3 +type: docs +--- + + +## Event + +Events are JSON-serializable messages passed between actions. Every event has a `type` string and an +`attributes` map that carries its payload. Action routing can match the exact event type or evaluate +a condition expression against the event data. One event may trigger multiple actions. + +### Special Events + +* `InputEvent`: Generated by the framework when an input record arrives. The record is available in + the event's `input` attribute. Actions whose trigger conditions match an `InputEvent` are the + agent's entry points. +* `OutputEvent`: When an action sends this event, the framework emits its `output` attribute + downstream as an agent output. It bypasses action matching. + +### Unified Event + +For simple cases, users can pass data between actions directly using `Event` with a custom `type` and `attributes`, without needing to define a subclass. For more structured events, see [Custom Event Subclasses](#custom-event-subclasses) below. + +{{< tabs "Unified Event" >}} + +{{< tab "Python" >}} +```python +# Send a unified event from one action +@action(EventType.InputEvent) +@staticmethod +def create_my_event(event: Event, ctx: RunnerContext) -> None: + ctx.send_event( + Event(type="my_event", attributes={"field1": "test", "field2": 42}) + ) + +# Consume it in another action +@action("my_event") +@staticmethod +def handle_my_event(event: Event, ctx: RunnerContext) -> None: + field1: str = event.get_attr("field1") + field2: int = event.get_attr("field2") +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +// Send a unified event from one action +@Action(EventType.InputEvent) +public static void createMyEvent(Event event, RunnerContext ctx) { + ctx.sendEvent(new Event("my_event", Map.of("field1", "test", "field2", 42))); +} + +// Consume it in another action +@Action("my_event") +public static void handleMyEvent(Event event, RunnerContext ctx) { + String field1 = (String) event.getAttr("field1"); + int field2 = (int) event.getAttr("field2"); +} +``` +{{< /tab >}} + +{{< /tabs >}} + +{{< hint info >}} +`upstreamEventId` and `upstreamActionName` (`upstream_event_id` and +`upstream_action_name` in Python) are framework-managed lineage metadata, set when an Action +emits the Event. User code should keep user data in `attributes`. `sendEvent` / `send_event` +rejects an Event that already carries lineage, such as an Event received from another Action or +a typed reconstruction of one: emit a new Event instead. Deserialization and typed +reconstruction keep the lineage an Event already has; after recovery, restored outputs are bound +to the replayed trigger again. +{{< /hint >}} + +### JSON Serialization + +Events are serialized as JSON when passed between Python actions or across the Java-Python boundary. When an event crosses that boundary, its **type** and its **payload** are handled differently. + +**Built-in event types are restored automatically.** At the JSON boundary (`Event.fromJson` in Java, `Event.from_json` in Python) the framework restores every built-in event type to its concrete subclass and validates its fixed attribute schema. So inside an action, a dispatched `InputEvent`, `ChatRequestEvent`, `ChatResponseEvent`, or any other built-in event is already its concrete type: use it directly (with a cast in Java) instead of calling `from_event` / `fromEvent`. Those reconstruction methods stay available and idempotent, so code that still calls them keeps working. + +**Custom event types stay generic.** A user-defined event type is not known to the framework, so it arrives as a generic `Event`; call your subclass's `from_event` / `fromEvent` to reconstruct it (see [Custom Event Subclasses](#custom-event-subclasses)). + +**Arbitrary payload values are not reconstructed.** Restoration recovers the event's type and its typed built-in attributes (for example, `ChatRequestEvent.messages` become typed `ChatMessage` objects), but it does not rebuild arbitrary payload values. Non-trivial values nested in `attributes` — such as Pydantic models, or the record carried by `InputEvent.input` — lose their type information and arrive as plain `dict` / `Map` objects, so you must reconstruct them yourself: + +```python +# `event` is already a concrete InputEvent (auto-restored at the JSON boundary); +# only its opaque `input` payload needs manual reconstruction. +input_data = ItemData.model_validate(event.input) +``` + +### Custom Event Subclasses + +Users can also define custom event subclasses for reusable, structured events. Data should be stored in the `attributes` map, and the subclass must implement a `from_event` / `fromEvent` factory method that validates required attributes and reconstructs typed objects from the deserialized data. + +{{< tabs "Custom Event" >}} + +{{< tab "Python" >}} +```python +class MyEvent(Event): + EVENT_TYPE: ClassVar[str] = "my_event" + + def __init__(self, value: str) -> None: + super().__init__(type=MyEvent.EVENT_TYPE, attributes={"value": value}) + + @classmethod + @override + def from_event(cls, event: Event) -> "MyEvent": + assert "value" in event.attributes + result = cls(value=event.attributes["value"]) + return result.reconstruct_from(event) + + @property + def value(self) -> str: + return self.get_attr("value") +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class MyEvent extends Event { + public static final String EVENT_TYPE = "my_event"; + + public MyEvent(String value) { + super(EVENT_TYPE); + setAttr("value", value); + } + + @JsonCreator + public MyEvent( + @JsonProperty("id") UUID id, + @JsonProperty("attributes") Map attributes) { + super(id, EVENT_TYPE, attributes); + } + + public static MyEvent fromEvent(Event event) { + return reconstructFrom(event, MyEvent::new); + } + + public String getValue() { + return (String) getAttr("value"); + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +{{< hint warning >}} +Python Event IDs are immutable. Custom `from_event` implementations written against earlier +versions may assign `result.id = event.id`; that assignment now raises a Pydantic +`ValidationError`. Replace `result.id = event.id` followed by `return result` with +`return result.reconstruct_from(event)`. + +Typed reconstruction represents the same Event occurrence, so it must preserve the base Event's +identity and framework-managed metadata: + +- **Python**: return `reconstruct_from(event)` after constructing the typed object. + It returns a new typed object with the Event's UUIDv4 `id`, `upstream_event_id`, and + `upstream_action_name`; the returned Event's `id` remains immutable. +- **Java**: implement the typed constructor that accepts `(UUID id, Map + attributes)`, then have `fromEvent` return + `reconstructFrom(event, MyEvent::new)`. The framework supplies the original ID and + attributes, validates that the ID is preserved, and carries over `sourceTimestamp`, + `upstreamEventId`, and `upstreamActionName`. +{{< /hint >}} + +{{< hint info >}} +All attribute values must be JSON-serializable. In Python, this means `BaseModel`-serializable or primitive types. In Java, values must be Jackson-serializable. +{{< /hint >}} + +{{< hint warning >}} +When defining a custom `Event` subclass in Java, annotate its `(UUID id, Map attributes)` constructor with `@JsonCreator`, and annotate both parameters with `@JsonProperty`. Durable execution stores events in `ActionState` and uses Jackson to restore concrete event subclasses during recovery. Without these annotations, recovery deserialization fails because the base `Event` constructor's `@JsonCreator` is not inherited by subclasses. + +If the subclass stores typed objects in `attributes`, convert them back to their typed forms inside this constructor too, as the built-in events do. For example, `ChatResponseEvent` restores its typed attributes like this: + +```java +@JsonCreator +public ChatResponseEvent( + @JsonProperty("id") UUID id, + @JsonProperty("attributes") Map attributes) { + super(id, EVENT_TYPE, normalizeAttributes(attributes)); +} + +/** Converts nested attributes back to their typed forms. */ +private static Map normalizeAttributes(Map attributes) { + Object rawId = attributes.get("request_id"); + if (rawId instanceof String) { + attributes.put("request_id", UUID.fromString((String) rawId)); + } + Object rawResponse = attributes.get("response"); + if (rawResponse instanceof Map) { + attributes.put("response", MAPPER.convertValue(rawResponse, ChatResult.class)); + } + return attributes; +} +``` +{{< /hint >}} + +## Built-in Events and Actions + +There are several built-in `Event` and `Action` in Flink-Agents: +* See [Chat Flow]({{< ref "docs/development/calling-llms/chat#chat-events" >}}) for how to chat with a LLM leveraging built-in action and events. +* See [Tool Use]({{< ref "docs/development/calling-llms/tool_use#built-in-events-and-actions" >}}) for how to programmatically use a tool leveraging built-in action and events. +* See [Vector Stores]({{< ref "docs/development/vector-search/vector_stores#built-in-events-and-actions" >}}) for how to retrieve context from vector stores leveraging built-in action and events. +* See [Model Routing]({{< ref "docs/development/calling-llms/model_routing#observability" >}}) for how a chat request selects between several models and how routing decisions are reported. diff --git a/docs/content/docs/development/create-agent/overview.md b/docs/content/docs/development/create-agent/overview.md new file mode 100644 index 0000000000..0ccb73277b --- /dev/null +++ b/docs/content/docs/development/create-agent/overview.md @@ -0,0 +1,238 @@ +--- +title: "Overview" +weight: 1 +type: docs +aliases: + - /docs/development/workflow_agent/ +--- + + +## Overview + +An Agent in Flink-Agents is an agent whose reasoning and behavior are organized as a directed workflow of modular steps, called actions, connected by events. This design is inspired by the need to orchestrate complex, multi-stage tasks in a transparent, extensible, and data-centric way, leveraging Apache Flink's streaming architecture. + +In Flink-Agents, an Agent is a class that inherits from the `Agent` base class. Its workflow +is defined by actions: functions decorated with `@action(...)` in Python or methods annotated with +`@Action(...)` in Java. Each action declares one or more trigger conditions, which may be exact event +types or condition expressions. Actions consume events, perform reasoning or tool calls, and emit new +events that may trigger other actions. This event-driven workflow forms a directed graph that may +contain cycles. Each node is an action, and each edge represents an event flow between actions. + +An Agent is well-suited for scenarios where the solution requires explicit orchestration, branching, or multi-step reasoning, such as data enrichment, multi-tool pipelines, or complex business logic. + +{{< hint info >}} +For guidance on choosing Java or Python, see [Should I choose Java or Python?]({{< ref "docs/faq/faq#q3-should-i-choose-java-or-python" >}}). +{{< /hint >}} + +## Agent Example + +{{< tabs "Agent Example" >}} + +{{< tab "Python" >}} +```python +class ReviewAnalysisAgent(Agent): + """An agent that uses a large language model (LLM) to analyze product reviews + and generate a satisfaction score and potential reasons for dissatisfaction. + + This agent receives a product review and produces a satisfaction score and a list + of reasons for dissatisfaction. It handles prompt construction, LLM interaction, + and output parsing. + """ + + @prompt + @staticmethod + def review_analysis_prompt() -> Prompt: + """Prompt for review analysis.""" + return review_analysis_prompt + + @tool + @staticmethod + def notify_shipping_manager(id: str, review: str) -> None: + """Notify the shipping manager when product received a negative review due to + shipping damage. + + Parameters + ---------- + id : str + The id of the product that received a negative review due to shipping damage + review: str + The negative review content + """ + # reuse the declared function, but for parsing the tool metadata, we write doc + # string here again. + notify_shipping_manager(id=id, review=review) + + @chat_model_setup + @staticmethod + def review_analysis_model() -> ResourceDescriptor: + """ChatModel which focus on review analysis.""" + return ResourceDescriptor( + clazz=ResourceName.ChatModel.OLLAMA_SETUP, + connection="ollama_server", + model="qwen3:8b", + prompt="review_analysis_prompt", + tools=["notify_shipping_manager"], + extract_reasoning=True, + ) + + @action(EventType.InputEvent) + @staticmethod + def process_input(event: Event, ctx: RunnerContext) -> None: + """Process input event and send chat request for review analysis.""" + input_event = InputEvent.from_event(event) + input: ProductReview = input_event.input + ctx.short_term_memory.set("id", input.id) + + content = f""" + "id": {input.id}, + "review": {input.review} + """ + msg = ChatMessage(role=MessageRole.USER) + ctx.send_event( + ChatRequestEvent( + model="review_analysis_model", + messages=[msg], + prompt_args={"input": content}, + ) + ) + + @action(EventType.ChatResponseEvent) + @staticmethod + def process_chat_response(event: Event, ctx: RunnerContext) -> None: + """Process chat response event and send output event.""" + chat_response = ChatResponseEvent.from_event(event) + try: + json_content = json.loads(chat_response.response.text) + ctx.send_event( + OutputEvent( + output=ProductReviewAnalysisRes( + id=ctx.short_term_memory.get("id"), + score=json_content["score"], + reasons=json_content["reasons"], + ) + ) + ) + except Exception: + logging.exception( + f"Error processing chat response {chat_response.response.text}" + ) + + # To fail the agent, you can raise an exception here. +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +/** + * An agent that uses a large language model (LLM) to analyze product reviews and generate a + * satisfaction score and potential reasons for dissatisfaction. + * + *

This agent receives a product review and produces a satisfaction score and a list of reasons + * for dissatisfaction. It handles prompt construction, LLM interaction, and output parsing. + */ +public class ReviewAnalysisAgent extends Agent { + + private static final ObjectMapper MAPPER = new ObjectMapper(); + + @Prompt + public static org.apache.flink.agents.api.prompt.Prompt reviewAnalysisPrompt() { + return REVIEW_ANALYSIS_PROMPT; + } + + @ChatModelSetup + public static ResourceDescriptor reviewAnalysisModel() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) + .addInitialArgument("connection", "ollamaChatModelConnection") + .addInitialArgument("model", "qwen3:8b") + .addInitialArgument("prompt", "reviewAnalysisPrompt") + .addInitialArgument("tools", Collections.singletonList("notifyShippingManager")) + .addInitialArgument("extract_reasoning", true) + .build(); + } + + /** + * Tool for notifying the shipping manager when product received a negative review due to + * shipping damage. + * + * @param id The id of the product that received a negative review due to shipping damage + * @param review The negative review content + */ + @Tool( + description = + "Notify the shipping manager when product received a negative review due to shipping damage.") + public static void notifyShippingManager( + @ToolParam(name = "id") String id, @ToolParam(name = "review") String review) { + CustomTypesAndResources.notifyShippingManager(id, review); + } + + /** Process input event and send chat request for review analysis. */ + @Action(EventType.InputEvent) + public static void processInput(Event event, RunnerContext ctx) throws Exception { + InputEvent inputEvent = InputEvent.fromEvent(event); + String input = (String) inputEvent.getInput(); + MAPPER.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); + CustomTypesAndResources.ProductReview inputObj = + MAPPER.readValue(input, CustomTypesAndResources.ProductReview.class); + + ctx.getShortTermMemory().set("id", inputObj.getId()); + + String content = + String.format( + "{\n" + "\"id\": %s,\n" + "\"review\": \"%s\"\n" + "}", + inputObj.getId(), inputObj.getReview()); + ChatMessage msg = new ChatMessage(MessageRole.USER, ""); + + ctx.sendEvent( + new ChatRequestEvent( + "reviewAnalysisModel", List.of(msg), Map.of("input", content), null)); + } + + @Action(EventType.ChatResponseEvent) + public static void processChatResponse(Event event, RunnerContext ctx) + throws Exception { + ChatResponseEvent chatResponse = ChatResponseEvent.fromEvent(event); + JsonNode jsonNode = MAPPER.readTree(chatResponse.getResponse().getText()); + JsonNode scoreNode = jsonNode.findValue("score"); + JsonNode reasonsNode = jsonNode.findValue("reasons"); + if (scoreNode == null || reasonsNode == null) { + throw new IllegalStateException( + "Invalid response from LLM: missing 'score' or 'reasons' field."); + } + List result = new ArrayList<>(); + if (reasonsNode.isArray()) { + for (JsonNode node : reasonsNode) { + result.add(node.asText()); + } + } + + ctx.sendEvent( + new OutputEvent( + new CustomTypesAndResources.ProductReviewAnalysisRes( + ctx.getShortTermMemory().get("id").getValue().toString(), + scoreNode.asInt(), + result))); + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +Continue with [Actions]({{< ref "docs/development/create-agent/actions" >}}) and [Events]({{< ref "docs/development/create-agent/events" >}}). diff --git a/docs/content/docs/development/yaml.md b/docs/content/docs/development/create-agent/yaml.md similarity index 98% rename from docs/content/docs/development/yaml.md rename to docs/content/docs/development/create-agent/yaml.md index c2a83d30c0..6de28fd681 100644 --- a/docs/content/docs/development/yaml.md +++ b/docs/content/docs/development/create-agent/yaml.md @@ -1,7 +1,9 @@ --- -title: YAML API -weight: 3 +title: "YAML API" +weight: 5 type: docs +aliases: + - /docs/development/yaml/ --- + +- [Sub-Agents]({{< ref "docs/development/multi-agent/subagents" >}}) +- [Add Multiple Agents in One Flink Job]({{< ref "docs/development/multi-agent/multiple-agents" >}}) diff --git a/docs/content/docs/development/multi-agent/multiple-agents.md b/docs/content/docs/development/multi-agent/multiple-agents.md new file mode 100644 index 0000000000..3ffe6277a2 --- /dev/null +++ b/docs/content/docs/development/multi-agent/multiple-agents.md @@ -0,0 +1,27 @@ +--- +title: "Add Multiple Agents in One Flink Job" +weight: 2 +type: docs +--- + + +Compose Agents in a Flink pipeline using the [Connect Multiple Agents]({{< ref "docs/get-started/quickstart/create-agent/multiple-agents" >}}). + +See [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}) for the DataStream and Table API integration reference. diff --git a/docs/content/docs/development/subagents.md b/docs/content/docs/development/multi-agent/subagents.md similarity index 98% rename from docs/content/docs/development/subagents.md rename to docs/content/docs/development/multi-agent/subagents.md index 166dadc405..86e6108972 100644 --- a/docs/content/docs/development/subagents.md +++ b/docs/content/docs/development/multi-agent/subagents.md @@ -1,7 +1,9 @@ --- -title: Sub-agents -weight: 13 +title: "Sub-Agents" +weight: 1 type: docs +aliases: + - /docs/development/subagents/ --- + +- [Embedding Models]({{< ref "docs/development/vector-search/embedding_models" >}}) + - [Built-in Embedding Model Providers]({{< ref "docs/development/vector-search/embedding_models/providers" >}}) +- [Vector Stores]({{< ref "docs/development/vector-search/vector_stores" >}}) + - [Built-in Vector Store Providers]({{< ref "docs/development/vector-search/vector_stores/providers" >}}) diff --git a/docs/content/docs/development/vector-search/embedding_models/_index.md b/docs/content/docs/development/vector-search/embedding_models/_index.md new file mode 100644 index 0000000000..13a4f11131 --- /dev/null +++ b/docs/content/docs/development/vector-search/embedding_models/_index.md @@ -0,0 +1,407 @@ +--- +title: "Embedding Models" +weight: 1 +type: docs +aliases: + - /docs/development/embedding_models/ +--- + + +# Embedding Models + +{{< hint info >}} +This page covers text-based embedding models. Flink Agents does not currently support multimodal embeddings. +{{< /hint >}} + +## Overview + +Embedding models convert text strings into high-dimensional vectors that capture semantic meaning, enabling powerful semantic search and retrieval capabilities. These vector representations allow agents to understand and work with text similarity, semantic search, and knowledge retrieval patterns. + +In Flink Agents, embedding models are essential for: +- **Semantic Search**: Finding relevant documents or information based on meaning rather than exact keyword matches +- **Text Similarity**: Measuring how similar two pieces of text are in meaning +- **Knowledge Retrieval**: Enabling agents to find and retrieve relevant context from large knowledge bases +- **Vector Databases**: Storing and querying embeddings for efficient similarity search + +## Getting Started + +To use embedding models in your agents, you need to define both a connection and setup using decorators/annotations, then access the embedding model through the runtime context. + +### Resource Declaration + +Flink Agents provides decorators(in python) and annotations(in java) to simplify embedding model setup within agents: + +#### Declare an embedding model connection + +The **`@embedding_model_connection`** decorator/ **`@EmbeddingModelConnection`** annotation marks a method that creates an embedding model connection. +This is typically defined once and shared across multiple embedding model setups. + +{{< tabs "Declare an embedding model connection" >}} + +{{< tab "Python" >}} +```python +@embedding_model_connection +@staticmethod +def embedding_model_connection() -> ResourceDescriptor: + ... +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +@EmbeddingModelConnection +public static ResourceDescriptor embeddingModelConnection() { + ... +} +``` +{{< /tab >}} + +{{< /tabs >}} + +#### Declare an embedding model setup + +The **`@embedding_model_setup`** decorator/ **`@EmbeddingModelSetup`** annotation marks a method that creates an embedding model setup. +This references an embedding model connection and adds embed-specific configuration like model and dimensions. + +{{< tabs "Declare an embedding model setup" >}} + +{{< tab "Python" >}} +```python +@embedding_model_setup +@staticmethod +def embedding_model_setup() -> ResourceDescriptor: + ... +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +@EmbeddingModelSetup +public static ResourceDescriptor embeddingModelSetup() { + ... +} +``` +{{< /tab >}} + +{{< /tabs >}} + +### Usage Example + +Here's how to define and use embedding models in your agent: + +{{< tabs "Usage example" >}} + +{{< tab "Python" >}} +```python +class MyAgent(Agent): + + @embedding_model_connection + @staticmethod + def openai_connection() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.EmbeddingModel.OPENAI_CONNECTION, + api_key="your-api-key-here", + base_url="https://api.openai.com/v1", + request_timeout=30.0 + ) + + @embedding_model_setup + @staticmethod + def openai_embedding() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.EmbeddingModel.OPENAI_SETUP, + connection="openai_connection", + model="your-embedding-model-here" + ) + + @action(EventType.InputEvent) + @staticmethod + def process_text(event: Event, ctx: RunnerContext) -> None: + # Get the embedding model from the runtime context + embedding_model = ctx.get_resource("openai_embedding", ResourceType.EMBEDDING_MODEL) + + # Use the embedding model to generate embeddings + input_event = InputEvent.from_event(event) + user_query = str(input_event.input) + embedding = embedding_model.embed(user_query) + + # Handle the embedding + # Process the embedding vector as needed for your use case +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class MyAgent extends Agent { + + @EmbeddingModelConnection + public static ResourceDescriptor ollamaConnection() { + return ResourceDescriptor.Builder.newBuilder( + ResourceName.EmbeddingModel.OLLAMA_CONNECTION) + .addInitialArgument("host", "http://localhost:11434") + .build(); + } + + @EmbeddingModelSetup + public static ResourceDescriptor ollamaEmbedding() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.OLLAMA_SETUP) + .addInitialArgument("connection", "ollamaConnection") + .addInitialArgument("model", "nomic-embed-text") + .build(); + } + + @Action(EventType.InputEvent) + public static void processText(Event event, RunnerContext ctx) + throws Exception { + InputEvent inputEvent = InputEvent.fromEvent(event); + // Get the embedding model from the runtime context + BaseEmbeddingModelSetup embeddingModel = + (BaseEmbeddingModelSetup) + ctx.getResource("embeddingModel", ResourceType.EMBEDDING_MODEL); + + // Use the embedding model to generate embeddings + String input = (String) inputEvent.getInput(); + float[] embedding = embeddingModel.embed(input); + + // Handle the embedding + // Process the embedding vector as needed for your use case + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +## Using Cross-Language Providers + +Flink Agents supports cross-language embedding model integration, allowing you to use embedding models implemented in one language (Java or Python) from agents written in the other language. This is particularly useful when an embedding model provider is only available in one language (e.g., the DashScope embedding integration for Alibaba Cloud Model Studio is currently Python-only). + +{{< hint warning >}} +**Limitations:** +- Cross-language resources are currently supported only when [running in Flink]({{< ref "docs/operations/deployment#run-in-flink" >}}), not in local development mode +- Complex object serialization between languages may have limitations +{{< /hint >}} +### How To Use + +To use a resource implemented in another language, declare its implementation class with the corresponding cross-language resource descriptor: + +- **Using Java embedding models in Python**: Use `JavaResourceDescriptor` with the actual Java connection or setup class as `clazz`. +- **Using Python embedding models in Java**: Use `PythonResourceDescriptor.Builder.newBuilder(...)` with the actual Python connection or setup class, including its module. + +### Usage Example + +{{< tabs "Cross-Language Embedding Model Usage Example" >}} + +{{< tab "Using Java Embedding Model in Python" >}} + +```python +class MyAgent(Agent): + + @embedding_model_connection + @staticmethod + def java_embedding_connection() -> ResourceDescriptor: + # In pure Java, the equivalent ResourceDescriptor would be: + # ResourceDescriptor.Builder + # .newBuilder(ResourceName.EmbeddingModel.OLLAMA_CONNECTION) + # .addInitialArgument("host", "http://localhost:11434") + # .build(); + return JavaResourceDescriptor( + clazz=ResourceName.EmbeddingModel.Java.OLLAMA_CONNECTION, + host="http://localhost:11434" + ) + + @embedding_model_setup + @staticmethod + def java_embedding_model() -> ResourceDescriptor: + # In pure Java, the equivalent ResourceDescriptor would be: + # ResourceDescriptor.Builder + # .newBuilder(ResourceName.EmbeddingModel.OLLAMA_SETUP) + # .addInitialArgument("connection", "java_embedding_connection") + # .addInitialArgument("model", "nomic-embed-text") + # .build(); + return JavaResourceDescriptor( + clazz=ResourceName.EmbeddingModel.Java.OLLAMA_SETUP, + connection="java_embedding_connection", + model="nomic-embed-text" + ) + + @action(EventType.InputEvent) + @staticmethod + def process_input(event: Event, ctx: RunnerContext) -> None: + # Use the Java embedding model from Python + input_event = InputEvent.from_event(event) + embedding_model = ctx.get_resource("java_embedding_model", ResourceType.EMBEDDING_MODEL) + embedding = embedding_model.embed(str(input_event.input)) + # Process the embedding vector as needed +``` + +{{< /tab >}} + +{{< tab "Using Python Embedding Model in Java" >}} + +```java +public class MyAgent extends Agent { + + @EmbeddingModelConnection + public static ResourceDescriptor pythonEmbeddingConnection() { + // In pure Python, the equivalent ResourceDescriptor would be: + // ResourceDescriptor( + // clazz=ResourceName.EmbeddingModel.OLLAMA_CONNECTION, + // base_url="http://localhost:11434" + // ) + return PythonResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.Python.OLLAMA_CONNECTION) + .addInitialArgument("base_url", "http://localhost:11434") + .build(); + } + + @EmbeddingModelSetup + public static ResourceDescriptor pythonEmbeddingModel() { + // In pure Python, the equivalent ResourceDescriptor would be: + // ResourceDescriptor( + // clazz=ResourceName.EmbeddingModel.OLLAMA_SETUP, + // connection="ollama_connection", + // model="nomic-embed-text" + // ) + return PythonResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.Python.OLLAMA_SETUP) + .addInitialArgument("connection", "pythonEmbeddingConnection") + .addInitialArgument("model", "nomic-embed-text") + .build(); + } + + @Action(EventType.InputEvent) + public static void processInput(Event event, RunnerContext ctx) throws Exception { + InputEvent inputEvent = InputEvent.fromEvent(event); + // Use the Python embedding model from Java + BaseEmbeddingModelSetup embeddingModel = + (BaseEmbeddingModelSetup) ctx.getResource( + "pythonEmbeddingModel", + ResourceType.EMBEDDING_MODEL); + float[] embedding = embeddingModel.embed((String) inputEvent.getInput()); + // Process the embedding vector as needed + } +} +``` + +{{< /tab >}} + +{{< /tabs >}} + +## Custom Providers + +{{< hint warning >}} +The custom provider APIs are experimental and unstable, subject to incompatible changes in future releases. +{{< /hint >}} + +If you want to use embedding models not offered by the built-in providers, you can extend the base embedding classes and implement your own! The embedding system is built around two main abstract classes: + +### BaseEmbeddingModelConnection + +Handles the connection to embedding services and provides the core embedding functionality. +{{< tabs "Custom Embedding Connection" >}} + +{{< tab "Python" >}} + +```python +class MyEmbeddingConnection(BaseEmbeddingModelConnection): + + @abstractmethod + def embed(self, text: str | Sequence[str], **kwargs: Any) -> list[float] | list[list[float]]: + # Core method: convert text to embedding vector + # - text: Input text to embed + # - kwargs: Additional parameters from model_kwargs + # - Returns: List of float values representing the embedding + pass +``` + +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class MyEmbeddingConnection extends BaseEmbeddingModelConnection { + + @Override + public float[] embed(String text, Map parameters) { + // Core method: convert text to embedding vector + // - text: Input text to embed + // - parameters: Additional parameters + // - Returns: Float array representing the embedding + float[] embedding = ...; + return embedding; + } + + @Override + public List embed(List texts, Map parameters) { + // Core method: convert texts to embedding vectors + // - text: Input texts to embed + // - parameters: Additional parameters + // - Returns: List of float array representing the embeddings + List embeddings = ...; + return embeddings; + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +### BaseEmbeddingModelSetup + +The setup class acts as a high-level configuration interface that defines which connection to use and how to configure the embedding model. + +{{< tabs "Custom Embedding Setup" >}} + +{{< tab "Python" >}} +```python +class MyEmbeddingSetup(BaseEmbeddingModelSetup): + # Add your custom configuration fields here + + @property + def model_kwargs(self) -> Dict[str, Any]: + # Return model-specific configuration passed to embed() + # This dictionary is passed as **kwargs to the embed() method + return {"model": self.model, ...} +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class MyEmbeddingSetup extends BaseEmbeddingModelSetup { + + @Override + public Map getParameters() { + // Return model-specific configuration passed to embed() + // This dictionary is passed as parameters to the embed() method + Map parameters = new HashMap<>(); + + if (model != null) { + parameters.put("model", model); + } + ... + + return parameters; + } + +} +``` +{{< /tab >}} + +{{< /tabs >}} + +[Built-in Embedding Model Providers]({{< ref "docs/development/vector-search/embedding_models/providers" >}}). diff --git a/docs/content/docs/development/embedding_models.md b/docs/content/docs/development/vector-search/embedding_models/providers.md similarity index 57% rename from docs/content/docs/development/embedding_models.md rename to docs/content/docs/development/vector-search/embedding_models/providers.md index c26c1ee93b..7860a5989e 100644 --- a/docs/content/docs/development/embedding_models.md +++ b/docs/content/docs/development/vector-search/embedding_models/providers.md @@ -1,7 +1,9 @@ --- -title: Embedding Models -weight: 7 +title: "Built-in Embedding Model Providers" +weight: 1 type: docs +aliases: + - /docs/development/vector-search/embedding_models_providers/ --- -# Embedding Models - -{{< hint info >}} -This page covers text-based embedding models. Flink Agents does not currently support multimodal embeddings. -{{< /hint >}} - -## Overview - -Embedding models convert text strings into high-dimensional vectors that capture semantic meaning, enabling powerful semantic search and retrieval capabilities. These vector representations allow agents to understand and work with text similarity, semantic search, and knowledge retrieval patterns. - -In Flink Agents, embedding models are essential for: -- **Semantic Search**: Finding relevant documents or information based on meaning rather than exact keyword matches -- **Text Similarity**: Measuring how similar two pieces of text are in meaning -- **Knowledge Retrieval**: Enabling agents to find and retrieve relevant context from large knowledge bases -- **Vector Databases**: Storing and querying embeddings for efficient similarity search - -## Getting Started - -To use embedding models in your agents, you need to define both a connection and setup using decorators/annotations, then access the embedding model through the runtime context. - -### Resource Declaration - -Flink Agents provides decorators(in python) and annotations(in java) to simplify embedding model setup within agents: - -#### Declare an embedding model connection - -The **`@embedding_model_connection`** decorator/ **`@EmbeddingModelConnection`** annotation marks a method that creates an embedding model connection. -This is typically defined once and shared across multiple embedding model setups. - -{{< tabs "Declare an embedding model connection" >}} - -{{< tab "Python" >}} -```python -@embedding_model_connection -@staticmethod -def embedding_model_connection() -> ResourceDescriptor: - ... -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -@EmbeddingModelConnection -public static ResourceDescriptor embeddingModelConnection() { - ... -} -``` -{{< /tab >}} - -{{< /tabs >}} - -#### Declare an embedding model setup - -The **`@embedding_model_setup`** decorator/ **`@EmbeddingModelSetup`** annotation marks a method that creates an embedding model setup. -This references an embedding model connection and adds embed-specific configuration like model and dimensions. - -{{< tabs "Declare an embedding model setup" >}} - -{{< tab "Python" >}} -```python -@embedding_model_setup -@staticmethod -def embedding_model_setup() -> ResourceDescriptor: - ... -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -@EmbeddingModelSetup -public static ResourceDescriptor embeddingModelSetup() { - ... -} -``` -{{< /tab >}} - -{{< /tabs >}} - -### Usage Example - -Here's how to define and use embedding models in your agent: - -{{< tabs "Usage example" >}} - -{{< tab "Python" >}} -```python -class MyAgent(Agent): - - @embedding_model_connection - @staticmethod - def openai_connection() -> ResourceDescriptor: - return ResourceDescriptor( - clazz=ResourceName.EmbeddingModel.OPENAI_CONNECTION, - api_key="your-api-key-here", - base_url="https://api.openai.com/v1", - request_timeout=30.0 - ) - - @embedding_model_setup - @staticmethod - def openai_embedding() -> ResourceDescriptor: - return ResourceDescriptor( - clazz=ResourceName.EmbeddingModel.OPENAI_SETUP, - connection="openai_connection", - model="your-embedding-model-here" - ) - - @action(EventType.InputEvent) - @staticmethod - def process_text(event: Event, ctx: RunnerContext) -> None: - # Get the embedding model from the runtime context - embedding_model = ctx.get_resource("openai_embedding", ResourceType.EMBEDDING_MODEL) - - # Use the embedding model to generate embeddings - input_event = InputEvent.from_event(event) - user_query = str(input_event.input) - embedding = embedding_model.embed(user_query) - - # Handle the embedding - # Process the embedding vector as needed for your use case -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyAgent extends Agent { - - @EmbeddingModelConnection - public static ResourceDescriptor ollamaConnection() { - return ResourceDescriptor.Builder.newBuilder( - ResourceName.EmbeddingModel.OLLAMA_CONNECTION) - .addInitialArgument("host", "http://localhost:11434") - .build(); - } - - @EmbeddingModelSetup - public static ResourceDescriptor ollamaEmbedding() { - return ResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.OLLAMA_SETUP) - .addInitialArgument("connection", "ollamaConnection") - .addInitialArgument("model", "nomic-embed-text") - .build(); - } - - @Action(EventType.InputEvent) - public static void processText(Event event, RunnerContext ctx) - throws Exception { - InputEvent inputEvent = InputEvent.fromEvent(event); - // Get the embedding model from the runtime context - BaseEmbeddingModelSetup embeddingModel = - (BaseEmbeddingModelSetup) - ctx.getResource("embeddingModel", ResourceType.EMBEDDING_MODEL); - - // Use the embedding model to generate embeddings - String input = (String) inputEvent.getInput(); - float[] embedding = embeddingModel.embed(input); - - // Handle the embedding - // Process the embedding vector as needed for your use case - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - ## Built-in Providers ### Amazon Bedrock @@ -194,7 +31,7 @@ public class MyAgent extends Agent { Amazon Bedrock provides embedding capabilities through the Amazon Titan Text Embeddings V2 model via the [InvokeModel API](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). The integration supports configurable output dimensions (256, 512, or 1024) and parallelizes batch embedding via a configurable thread pool, since the Titan V2 model processes one text per API call. Authentication is handled via SigV4 using the AWS default credentials chain. {{< hint info >}} -Amazon Bedrock embedding models are only supported in Java currently. To use Amazon Bedrock embeddings from Python agents, see [Using Cross-Language Providers](#using-cross-language-providers). +Amazon Bedrock embedding models are only supported in Java currently. To use Amazon Bedrock embeddings from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/embedding_models#using-cross-language-providers" >}}). {{< /hint >}} #### Prerequisites @@ -571,7 +408,7 @@ Model availability and specifications may change. Always check the official Open This integration connects to Alibaba Cloud Model Studio (Bailian) through the DashScope SDK to access embedding models. {{< hint info >}} -The DashScope integration is currently supported in the Python API only. To use it from Java agents, see [Using Cross-Language Providers](#using-cross-language-providers). +The DashScope integration is currently supported in the Python API only. To use it from Java agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/embedding_models#using-cross-language-providers" >}}). {{< /hint >}} #### Prerequisites @@ -633,218 +470,3 @@ Some popular options include: {{< hint warning >}} Model availability and specifications may change. Always check the official Alibaba Cloud Model Studio documentation for the latest information before implementing in production. {{< /hint >}} - -## Using Cross-Language Providers - -Flink Agents supports cross-language embedding model integration, allowing you to use embedding models implemented in one language (Java or Python) from agents written in the other language. This is particularly useful when an embedding model provider is only available in one language (e.g., the DashScope embedding integration for Alibaba Cloud Model Studio is currently Python-only). - -{{< hint warning >}} -**Limitations:** -- Cross-language resources are currently supported only when [running in Flink]({{< ref "docs/operations/deployment#run-in-flink" >}}), not in local development mode -- Complex object serialization between languages may have limitations -{{< /hint >}} -### How To Use - -To use a resource implemented in another language, declare its implementation class with the corresponding cross-language resource descriptor: - -- **Using Java embedding models in Python**: Use `JavaResourceDescriptor` with the actual Java connection or setup class as `clazz`. -- **Using Python embedding models in Java**: Use `PythonResourceDescriptor.Builder.newBuilder(...)` with the actual Python connection or setup class, including its module. - -### Usage Example - -{{< tabs "Cross-Language Embedding Model Usage Example" >}} - -{{< tab "Using Java Embedding Model in Python" >}} - -```python -class MyAgent(Agent): - - @embedding_model_connection - @staticmethod - def java_embedding_connection() -> ResourceDescriptor: - # In pure Java, the equivalent ResourceDescriptor would be: - # ResourceDescriptor.Builder - # .newBuilder(ResourceName.EmbeddingModel.OLLAMA_CONNECTION) - # .addInitialArgument("host", "http://localhost:11434") - # .build(); - return JavaResourceDescriptor( - clazz=ResourceName.EmbeddingModel.Java.OLLAMA_CONNECTION, - host="http://localhost:11434" - ) - - @embedding_model_setup - @staticmethod - def java_embedding_model() -> ResourceDescriptor: - # In pure Java, the equivalent ResourceDescriptor would be: - # ResourceDescriptor.Builder - # .newBuilder(ResourceName.EmbeddingModel.OLLAMA_SETUP) - # .addInitialArgument("connection", "java_embedding_connection") - # .addInitialArgument("model", "nomic-embed-text") - # .build(); - return JavaResourceDescriptor( - clazz=ResourceName.EmbeddingModel.Java.OLLAMA_SETUP, - connection="java_embedding_connection", - model="nomic-embed-text" - ) - - @action(EventType.InputEvent) - @staticmethod - def process_input(event: Event, ctx: RunnerContext) -> None: - # Use the Java embedding model from Python - input_event = InputEvent.from_event(event) - embedding_model = ctx.get_resource("java_embedding_model", ResourceType.EMBEDDING_MODEL) - embedding = embedding_model.embed(str(input_event.input)) - # Process the embedding vector as needed -``` - -{{< /tab >}} - -{{< tab "Using Python Embedding Model in Java" >}} - -```java -public class MyAgent extends Agent { - - @EmbeddingModelConnection - public static ResourceDescriptor pythonEmbeddingConnection() { - // In pure Python, the equivalent ResourceDescriptor would be: - // ResourceDescriptor( - // clazz=ResourceName.EmbeddingModel.OLLAMA_CONNECTION, - // base_url="http://localhost:11434" - // ) - return PythonResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.Python.OLLAMA_CONNECTION) - .addInitialArgument("base_url", "http://localhost:11434") - .build(); - } - - @EmbeddingModelSetup - public static ResourceDescriptor pythonEmbeddingModel() { - // In pure Python, the equivalent ResourceDescriptor would be: - // ResourceDescriptor( - // clazz=ResourceName.EmbeddingModel.OLLAMA_SETUP, - // connection="ollama_connection", - // model="nomic-embed-text" - // ) - return PythonResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.Python.OLLAMA_SETUP) - .addInitialArgument("connection", "pythonEmbeddingConnection") - .addInitialArgument("model", "nomic-embed-text") - .build(); - } - - @Action(EventType.InputEvent) - public static void processInput(Event event, RunnerContext ctx) throws Exception { - InputEvent inputEvent = InputEvent.fromEvent(event); - // Use the Python embedding model from Java - BaseEmbeddingModelSetup embeddingModel = - (BaseEmbeddingModelSetup) ctx.getResource( - "pythonEmbeddingModel", - ResourceType.EMBEDDING_MODEL); - float[] embedding = embeddingModel.embed((String) inputEvent.getInput()); - // Process the embedding vector as needed - } -} -``` - -{{< /tab >}} - -{{< /tabs >}} - -## Custom Providers - -{{< hint warning >}} -The custom provider APIs are experimental and unstable, subject to incompatible changes in future releases. -{{< /hint >}} - -If you want to use embedding models not offered by the built-in providers, you can extend the base embedding classes and implement your own! The embedding system is built around two main abstract classes: - -### BaseEmbeddingModelConnection - -Handles the connection to embedding services and provides the core embedding functionality. -{{< tabs "Custom Embedding Connection" >}} - -{{< tab "Python" >}} - -```python -class MyEmbeddingConnection(BaseEmbeddingModelConnection): - - @abstractmethod - def embed(self, text: str | Sequence[str], **kwargs: Any) -> list[float] | list[list[float]]: - # Core method: convert text to embedding vector - # - text: Input text to embed - # - kwargs: Additional parameters from model_kwargs - # - Returns: List of float values representing the embedding - pass -``` - -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyEmbeddingConnection extends BaseEmbeddingModelConnection { - - @Override - public float[] embed(String text, Map parameters) { - // Core method: convert text to embedding vector - // - text: Input text to embed - // - parameters: Additional parameters - // - Returns: Float array representing the embedding - float[] embedding = ...; - return embedding; - } - - @Override - public List embed(List texts, Map parameters) { - // Core method: convert texts to embedding vectors - // - text: Input texts to embed - // - parameters: Additional parameters - // - Returns: List of float array representing the embeddings - List embeddings = ...; - return embeddings; - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - -### BaseEmbeddingModelSetup - -The setup class acts as a high-level configuration interface that defines which connection to use and how to configure the embedding model. - -{{< tabs "Custom Embedding Setup" >}} - -{{< tab "Python" >}} -```python -class MyEmbeddingSetup(BaseEmbeddingModelSetup): - # Add your custom configuration fields here - - @property - def model_kwargs(self) -> Dict[str, Any]: - # Return model-specific configuration passed to embed() - # This dictionary is passed as **kwargs to the embed() method - return {"model": self.model, ...} -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class MyEmbeddingSetup extends BaseEmbeddingModelSetup { - - @Override - public Map getParameters() { - // Return model-specific configuration passed to embed() - // This dictionary is passed as parameters to the embed() method - Map parameters = new HashMap<>(); - - if (model != null) { - parameters.put("model", model); - } - ... - - return parameters; - } - -} -``` -{{< /tab >}} - -{{< /tabs >}} diff --git a/docs/content/docs/development/vector_stores.md b/docs/content/docs/development/vector-search/vector_stores/_index.md similarity index 55% rename from docs/content/docs/development/vector_stores.md rename to docs/content/docs/development/vector-search/vector_stores/_index.md index a1d001ecb1..6c3186b82e 100644 --- a/docs/content/docs/development/vector_stores.md +++ b/docs/content/docs/development/vector-search/vector_stores/_index.md @@ -1,7 +1,9 @@ --- -title: Vector Stores -weight: 8 +title: "Vector Stores" +weight: 2 type: docs +aliases: + - /docs/development/vector_stores/ --- + +## Built-in Providers + +### Amazon OpenSearch + +[Amazon OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/) is a managed vector search service available in two flavors: OpenSearch Service (provisioned domains) and OpenSearch Serverless (AOSS). The Flink Agents integration supports both via a single `service_type` parameter, with IAM (SigV4) or basic authentication. + +{{< hint info >}} +Amazon OpenSearch is only supported in Java currently. To use Amazon OpenSearch from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/vector_stores#using-cross-language-providers" >}}). +{{< /hint >}} + +{{< hint info >}} +Amazon OpenSearch implements `CollectionManageableVectorStore`, enabling [Long-Term Memory]({{< ref "docs/development/memory/long_term_memory" >}}) support. Collections map to OpenSearch indices. OpenSearch indices do not natively support attaching arbitrary metadata, so any `metadata` passed to `createCollectionIfNotExists` is ignored. Callers needing per-document attributes should put them on the documents themselves. +{{< /hint >}} + +#### Prerequisites + +1. Either an [OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/) provisioned domain with KNN enabled (version 2.x+), or an [OpenSearch Serverless](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless.html) collection of type `VECTORSEARCH` +2. For IAM auth: IAM credentials configured via the [AWS Default Credentials Provider](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html) with appropriate access policies (or a Serverless data-access policy) +3. For basic auth (Service domains only): username and password for the OpenSearch domain + +#### OpenSearchVectorStore Parameters + +{{< tabs "OpenSearchVectorStore Parameters" >}} + +{{< tab "Java" >}} + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `embedding_model` | String | Required | Reference to embedding model resource name | +| `endpoint` | String | Required | OpenSearch endpoint URL (e.g. `https://my-domain.us-east-1.es.amazonaws.com` for a domain, or the `*.aoss.amazonaws.com` endpoint for Serverless) | +| `index` | String | Required | Default index name for document operations | +| `service_type` | String | `"serverless"` | OpenSearch flavor: `"serverless"` (AOSS) or `"domain"` (OpenSearch Service) | +| `auth` | String | `"iam"` | Authentication method: `"iam"` (SigV4) or `"basic"`. Basic auth is supported on Service domains only | +| `username` | String | None | Username for basic authentication (required if `auth=basic`) | +| `password` | String | None | Password for basic authentication (required if `auth=basic`) | +| `vector_field` | String | `"embedding"` | Name of the KNN vector field in the index | +| `content_field` | String | `"content"` | Name of the text content field in the index | +| `region` | String | `"us-east-1"` | AWS region | +| `dims` | int | `1024` | Vector dimensionality used when this integration creates an index | +| `max_bulk_mb` | int | `5` | Maximum bulk payload size in MB | + +{{< /tab >}} + +{{< /tabs >}} + +#### Usage Example + +{{< tabs "Amazon OpenSearch Usage Example" >}} + +{{< tab "Java" >}} + +For an OpenSearch Serverless (AOSS) collection with IAM auth (the default): + +```java +public class MyAgent extends Agent { + + @EmbeddingModelConnection + public static ResourceDescriptor bedrockEmbeddingConnection() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.BEDROCK_CONNECTION) + .addInitialArgument("region", "us-east-1") + .build(); + } + + @EmbeddingModelSetup + public static ResourceDescriptor bedrockEmbedding() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.BEDROCK_SETUP) + .addInitialArgument("connection", "bedrockEmbeddingConnection") + .addInitialArgument("dimensions", 1024) + .build(); + } + + @VectorStore + public static ResourceDescriptor opensearchStore() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.VectorStore.OPENSEARCH_VECTOR_STORE) + .addInitialArgument("embedding_model", "bedrockEmbedding") + .addInitialArgument("endpoint", "https://abc123.us-east-1.aoss.amazonaws.com") + .addInitialArgument("index", "my-vectors") + // service_type defaults to "serverless"; auth defaults to "iam" + .addInitialArgument("dims", 1024) + .build(); + } + + ... +} +``` + +For an OpenSearch Service provisioned domain with IAM auth: + +```java +@VectorStore +public static ResourceDescriptor opensearchDomainStore() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.VectorStore.OPENSEARCH_VECTOR_STORE) + .addInitialArgument("embedding_model", "bedrockEmbedding") + .addInitialArgument("endpoint", "https://my-domain.us-east-1.es.amazonaws.com") + .addInitialArgument("index", "my-vectors") + .addInitialArgument("service_type", "domain") + .addInitialArgument("auth", "iam") + .addInitialArgument("dims", 1024) + .build(); +} +``` + +For an OpenSearch Service domain with basic auth: + +```java +@VectorStore +public static ResourceDescriptor opensearchDomainBasicAuth() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.VectorStore.OPENSEARCH_VECTOR_STORE) + .addInitialArgument("embedding_model", "bedrockEmbedding") + .addInitialArgument("endpoint", "https://my-domain.us-east-1.es.amazonaws.com") + .addInitialArgument("index", "my-vectors") + .addInitialArgument("service_type", "domain") + .addInitialArgument("auth", "basic") + .addInitialArgument("username", "admin") + .addInitialArgument("password", "your-password") + .addInitialArgument("dims", 1024) + .build(); +} +``` + +{{< /tab >}} + +{{< /tabs >}} + +### Amazon S3 Vectors + +[Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html) is a purpose-built vector storage service from Amazon S3 that provides native support for storing and querying vector embeddings with sub-second query performance. It uses the S3 Vectors SDK for PutVectors, QueryVectors, GetVectors, and DeleteVectors operations. + +{{< hint info >}} +Amazon S3 Vectors is only supported in Java currently. To use Amazon S3 Vectors from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/vector_stores#using-cross-language-providers" >}}). +{{< /hint >}} + +{{< hint warning >}} +Amazon S3 Vectors does **not** implement `CollectionManageableVectorStore`, so it does not support [Long-Term Memory]({{< ref "docs/development/memory/long_term_memory" >}}) features. It also does not support `size()` or get-all operations: explicit document IDs are required for `get()` and `delete()`. +{{< /hint >}} + +#### Prerequisites + +1. An [S3 Vectors vector bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-buckets.html) and vector index created in your AWS account +2. IAM credentials configured via the [AWS Default Credentials Provider](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html) with appropriate S3 Vectors permissions + +#### S3VectorsVectorStore Parameters + +{{< tabs "S3VectorsVectorStore Parameters" >}} + +{{< tab "Java" >}} + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `embedding_model` | String | Required | Reference to embedding model resource name | +| `vector_bucket` | String | Required | S3 Vectors bucket name | +| `vector_index` | String | Required | S3 Vectors index name within the bucket | +| `region` | String | `"us-east-1"` | AWS region | + +{{< /tab >}} + +{{< /tabs >}} + +#### Usage Example + +{{< tabs "Amazon S3 Vectors Usage Example" >}} + +{{< tab "Java" >}} + +```java +public class MyAgent extends Agent { + + @EmbeddingModelConnection + public static ResourceDescriptor bedrockEmbeddingConnection() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.BEDROCK_CONNECTION) + .addInitialArgument("region", "us-east-1") + .build(); + } + + @EmbeddingModelSetup + public static ResourceDescriptor bedrockEmbedding() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.EmbeddingModel.BEDROCK_SETUP) + .addInitialArgument("connection", "bedrockEmbeddingConnection") + .addInitialArgument("dimensions", 1024) + .build(); + } + + @VectorStore + public static ResourceDescriptor s3VectorsStore() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.VectorStore.S3_VECTORS_VECTOR_STORE) + .addInitialArgument("embedding_model", "bedrockEmbedding") + .addInitialArgument("vector_bucket", "my-vector-bucket") + .addInitialArgument("vector_index", "my-index") + .addInitialArgument("region", "us-east-1") + .build(); + } + + ... +} +``` + +{{< /tab >}} + +{{< /tabs >}} + +### Chroma + +[Chroma](https://www.trychroma.com/home) is an open-source vector database that provides efficient storage and querying of embeddings with support for multiple deployment modes. + +{{< hint info >}} +Chroma is currently supported in the Python API only. To use Chroma from Java agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/vector_stores#using-cross-language-providers" >}}). +{{< /hint >}} + +#### Prerequisites + +1. Install ChromaDB: `pip install chromadb` +2. For server mode, start ChromaDB server: `chroma run --path /db_path` +3. For cloud mode, get API key from [ChromaDB Cloud](https://www.trychroma.com/) + +#### ChromaVectorStore Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `embedding_model` | str | Required | Reference to embedding model method name | +| `persist_directory` | str | None | Directory for persistent storage. If None, uses in-memory client | +| `host` | str | None | Host for ChromaDB server connection | +| `port` | int | `8000` | Port for ChromaDB server connection | +| `api_key` | str | None | API key for Chroma Cloud connection | +| `client_settings` | Settings | None | ChromaDB client settings for advanced configuration | +| `tenant` | str | `"default_tenant"` | ChromaDB tenant for multi-tenancy support | +| `database` | str | `"default_database"` | ChromaDB database name | +| `collection` | str | `"flink_agents_chroma_collection"` | Name of the ChromaDB collection to use | +| `collection_metadata` | dict | `{}` | Metadata for the collection (applied only when the read / write paths auto-create it) | +| `auto_create_collection` | bool | `True` | Whether read / write paths auto-create the collection when it's missing | + +#### Usage Example + +{{< tabs "Chroma Usage Example" >}} + +{{< tab "Python" >}} + +```python +class MyAgent(Agent): + + # Embedding model setup (required for vector store) + @embedding_model_connection + @staticmethod + def openai_connection() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.EmbeddingModel.OPENAI_CONNECTION, + api_key="your-api-key-here" + ) + + @embedding_model_setup + @staticmethod + def openai_embedding() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.EmbeddingModel.OPENAI_SETUP, + connection="openai_connection", + model="your-embedding-model-here" + ) + + # Vector store setup + @vector_store + @staticmethod + def chroma_store() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.VectorStore.CHROMA_VECTOR_STORE, + embedding_model="openai_embedding", + persist_directory="/path/to/chroma/data", # For persistent storage + collection="my_documents", + auto_create_collection=True + # Or use other modes: + # "host": "localhost", "port": 8000 # For server mode + # "api_key": "your-chroma-cloud-key" # For cloud mode + ) + + ... +``` + +{{< /tab >}} + +{{< /tabs >}} + +#### Deployment Modes + +ChromaDB supports multiple deployment modes: + +**In-Memory Mode** +```python +@vector_store +@staticmethod +def chroma_store() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.VectorStore.CHROMA_VECTOR_STORE, + embedding_model="your_embedding_model", + collection="my_documents" + # No connection configuration needed for in-memory mode + ) +``` + +**Persistent Mode** +```python +@vector_store +@staticmethod +def chroma_store() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.VectorStore.CHROMA_VECTOR_STORE, + embedding_model="your_embedding_model", + persist_directory="/path/to/chroma/data", + collection="my_documents" + ) +``` + +**Server Mode** +```python +@vector_store +@staticmethod +def chroma_store() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.VectorStore.CHROMA_VECTOR_STORE, + embedding_model="your_embedding_model", + host="your-chroma-server.com", + port=8000, + collection="my_documents" + ) +``` + +**Cloud Mode** +```python +@vector_store +@staticmethod +def chroma_store() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.VectorStore.CHROMA_VECTOR_STORE, + embedding_model="your_embedding_model", + api_key="your-chroma-cloud-api-key", + collection="my_documents" + ) +``` + +### Elasticsearch + +[Elasticsearch](https://www.elastic.co/elasticsearch/) is a distributed, RESTful search and analytics engine that supports vector search through dense vector fields and K-Nearest Neighbors (KNN). + +{{< hint info >}} +Elasticsearch is currently supported in the Java API only. To use Elasticsearch from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/vector_stores#using-cross-language-providers" >}}). +{{< /hint >}} + +#### Prerequisites + +1. An Elasticsearch cluster (version 8.0 or later for KNN support). + +#### ElasticsearchVectorStore Parameters + +| Parameter | Type | Default | Description | +|-------------------|------|---------------------------|--------------------------------------------------------------------| +| `embedding_model` | str | Required | Reference to embedding model resource name | +| `index` | str | None | Default target Elasticsearch index name | +| `vector_field` | str | `"_vector"` | Name of the dense vector field used for KNN | +| `dims` | int | `768` | Vector dimensionality | +| `k` | int | None | Number of nearest neighbors to return; can be overridden per query | +| `num_candidates` | int | None | Candidate set size for ANN search; can be overridden per query | +| `filter_query` | str | None | Raw JSON Elasticsearch filter query (DSL) restricting which documents a KNN query can match | +| `host` | str | `"http://localhost:9200"` | Elasticsearch endpoint | +| `hosts` | str | None | Comma-separated list of Elasticsearch endpoints | +| `username` | str | None | Username for basic authentication | +| `password` | str | None | Password for basic authentication | +| `api_key_base64` | str | None | Base64-encoded API key for authentication | +| `api_key_id` | str | None | API key ID for authentication | +| `api_key_secret` | str | None | API key secret for authentication | + +{{< hint warning >}} +For an index not created by flink-agents, the index must already contain a `dense_vector` field, and the user must specify its name via `vector_field`. +{{< /hint >}} +#### Usage Example + +{{< tabs "Elasticsearch Usage Example" >}} + +{{< tab "Java" >}} + +Here's how to define an Elasticsearch vector store in your Java agent: + +```java +@VectorStore +public static ResourceDescriptor vectorStore() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.VectorStore.ELASTICSEARCH_VECTOR_STORE) + .addInitialArgument("embedding_model", "embeddingModel") + .addInitialArgument("host", "http://localhost:9200") + .addInitialArgument("index", "my_documents") + .addInitialArgument("vector_field", "content_vector") + .addInitialArgument("dims", 1536) + // Optional authentication + // .addInitialArgument("username", "elastic") + // .addInitialArgument("password", "secret") + .build(); +} +``` + +{{< /tab >}} + +{{< /tabs >}} + +### Mem0 + +[Mem0](https://docs.mem0.ai/) ships its own ecosystem of vector-store backends (pgvector, Milvus, Qdrant, Redis, Weaviate, ...). `Mem0VectorStore` is a gateway that exposes any of them through Flink Agents' resource system, so you can reach Mem0-supported backends without a dedicated integration for each. + +{{< hint info >}} +Mem0 is currently supported in the Python API only. To use it from Java agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/vector_stores#using-cross-language-providers" >}}). +{{< /hint >}} + +{{< hint info >}} +`Mem0VectorStore` implements `CollectionManageableVectorStore`, enabling [Long-Term Memory]({{< ref "docs/development/memory/long_term_memory" >}}) support. Filters use the unified equality-only [Filter DSL]({{< ref "docs/development/vector-search/vector_stores#filter-dsl" >}}) and are forwarded to the underlying Mem0 backend unchanged. +{{< /hint >}} + +#### Prerequisites + +1. Install Mem0: `pip install mem0ai` +2. Any extra dependency required by the chosen backend (e.g. `pip install qdrant-client` for Qdrant, `pip install pymilvus` for Milvus). See the [Mem0 vector store docs](https://docs.mem0.ai/components/vectordbs/overview) for per-provider requirements. + +#### Mem0VectorStore Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `embedding_model` | str | Required | Reference to embedding model resource name | +| `provider` | str | Required | Mem0 vector store provider name (e.g. `"chroma"`, `"qdrant"`, `"pgvector"`, `"milvus"`) | +| `provider_config` | dict | `{}` | Provider-specific config dict passed to Mem0's `VectorStoreFactory` (e.g. host, port, credentials). `collection_name` is injected automatically and need not be set | +| `collection` | str | `"flink_agents_mem0_vs"` | Default collection used when a caller does not specify one | + +#### Usage Example + +{{< tabs "Mem0 Usage Example" >}} + +{{< tab "Python" >}} + +```python +class MyAgent(Agent): + + # Embedding model setup (required for vector store) + @embedding_model_connection + @staticmethod + def openai_connection() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.EmbeddingModel.OPENAI_CONNECTION, + api_key="your-api-key-here" + ) + + @embedding_model_setup + @staticmethod + def openai_embedding() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.EmbeddingModel.OPENAI_SETUP, + connection="openai_connection", + model="your-embedding-model-here" + ) + + # Mem0 vector store backed by Qdrant + @vector_store + @staticmethod + def mem0_store() -> ResourceDescriptor: + return ResourceDescriptor( + clazz=ResourceName.VectorStore.MEM0_VECTOR_STORE, + embedding_model="openai_embedding", + provider="qdrant", + provider_config={"host": "localhost", "port": 6333, "embedding_model_dims": 1536}, + collection="my_documents" + ) + + ... +``` + +{{< /tab >}} + +{{< /tabs >}} + +### Milvus + +[Milvus](https://milvus.io/) is an open-source vector database designed for high-dimensional vector search at scale. + +{{< hint info >}} +Milvus is currently supported in the Java API only. To use Milvus from Python agents, see [Using Cross-Language Providers]({{< ref "docs/development/vector-search/vector_stores#using-cross-language-providers" >}}). +{{< /hint >}} + +#### Prerequisites + +1. A Milvus server. + +#### MilvusVectorStore Parameters + +| Parameter | Type | Default | Description | +|-----------------------------|------|--------------------------------------|-----------------------------------------------------------------------------| +| `embedding_model` | str | Required | Reference to embedding model resource name | +| `collection` | str | `"flink_agents_milvus_collection"` | Default target Milvus collection name | +| `collection_name` | str | None | Alias for `collection` | +| `index` | str | None | Alias for `collection`, mainly for cross-provider compatibility | +| `id_field` | str | `"id"` | Name of the primary key field | +| `content_field` | str | `"content"` | Name of the field storing document content | +| `metadata_field` | str | `"metadata"` | Name of the JSON field storing document metadata | +| `vector_field` | str | `"embedding"` | Name of the FloatVector field used for vector search | +| `dims` | int | `768` | Vector dimensionality | +| `id_max_length` | int | `65535` | Maximum length for the VarChar primary key field | +| `content_max_length` | int | `65535` | Maximum length for the VarChar content field | +| `metric_type` | str | `"COSINE"` | Milvus metric type used by vector search | +| `index_type` | str | `"AUTOINDEX"` | Milvus vector index type | +| `index_params` | map | `{}` | Extra vector index parameters passed to Milvus | +| `metadata_index_keys` | list | `user_id`, `agent_id`, `run_id`, `actor_id`, `category` | Additional metadata JSON keys indexed with path indexes | +| `metadata_index_cast_types` | map | Default keys use `"VARCHAR"` | Per-metadata-key JSON path index cast type overrides | +| `num_shards` | int | `1` | Number of Milvus shards for newly created collections | +| `consistency_level` | str | `"BOUNDED"` | Milvus consistency level for collection creation, query, and search | +| `max_get_limit` | int | `10000` | Maximum number of documents returned by `get` when no limit is specified | +| `load_timeout_ms` | long | `120000` | Timeout for loading collections | +| `uri` | str | `"http://localhost:19530"` | Milvus endpoint | +| `host` | str | `"localhost"` | Milvus host used when `uri` is not set | +| `port` | int | `19530` | Milvus port used when `uri` is not set | +| `db_name` | str | None | Milvus database name | +| `token` | str | None | Token for Milvus authentication | +| `username` | str | None | Username for basic authentication | +| `password` | str | None | Password for basic authentication | +| `enable_precheck` | bool | `false` | Whether to enable Milvus client precheck | + +{{< hint info >}} +When creating a collection, MilvusVectorStore creates a primary-key field, content field, JSON metadata field, vector field, vector index, and JSON metadata indexes. The default metadata JSON path indexes cover common filter keys such as `user_id`, `agent_id`, `run_id`, `actor_id`, and `category`; add `metadata_index_keys` for application-specific filter keys. + +The default shard count is `1`. As a rough capacity-planning rule, use about one shard per 100 million vectors, and increase it for heavier write throughput. +{{< /hint >}} + +#### Usage Example + +{{< tabs "Milvus Usage Example" >}} + +{{< tab "Java" >}} + +```java +@VectorStore +public static ResourceDescriptor vectorStore() { + return ResourceDescriptor.Builder.newBuilder(ResourceName.VectorStore.MILVUS_VECTOR_STORE) + .addInitialArgument("embedding_model", "embeddingModel") + .addInitialArgument("uri", "http://localhost:19530") + .addInitialArgument("collection", "my_documents") + .addInitialArgument("dims", 1536) + .addInitialArgument("metric_type", "COSINE") + .addInitialArgument("index_type", "AUTOINDEX") + // Optional metadata JSON path indexes + // .addInitialArgument("metadata_index_keys", List.of("user_id", "agent_id", "run_id")) + .build(); +} +``` + +{{< /tab >}} + +{{< /tabs >}} diff --git a/docs/content/docs/faq/faq.md b/docs/content/docs/faq/faq.md index c12031dbda..1ca449eca1 100644 --- a/docs/content/docs/faq/faq.md +++ b/docs/content/docs/faq/faq.md @@ -97,47 +97,47 @@ Flink Agents provides built-in integrations for many ecosystem providers. Some i | Provider | Python | Java | |---|---|---| -| [Amazon Bedrock]({{< ref "docs/development/chat_models#amazon-bedrock" >}}) | ❌ | ✅ | -| [Anthropic]({{< ref "docs/development/chat_models#anthropic" >}}) | ✅ | ✅ | -| [Azure OpenAI]({{< ref "docs/development/chat_models#azure-openai" >}}) | ✅ | ✅ | -| [Gemini]({{< ref "docs/development/chat_models#gemini" >}}) | ❌ | ✅ | -| [Ollama]({{< ref "docs/development/chat_models#ollama" >}}) | ✅ | ✅ | -| [OpenAI]({{< ref "docs/development/chat_models#openai" >}}) | ✅ | ✅ | -| [Alibaba Cloud Model Studio (DashScope)]({{< ref "docs/development/chat_models#alibaba-cloud-model-studio-dashscope" >}}) | ✅ | ❌ | -| [vLLM]({{< ref "docs/development/chat_models#vllm" >}}) | ✅ | ✅ | -| [Watsonx (IBM watsonx.ai)]({{< ref "docs/development/chat_models#watsonx-ibm-watsonxai" >}}) | ✅ | ✅ | +| [Amazon Bedrock]({{< ref "docs/development/calling-llms/chat-models/providers#amazon-bedrock" >}}) | ❌ | ✅ | +| [Anthropic]({{< ref "docs/development/calling-llms/chat-models/providers#anthropic" >}}) | ✅ | ✅ | +| [Azure OpenAI]({{< ref "docs/development/calling-llms/chat-models/providers#azure-openai" >}}) | ✅ | ✅ | +| [Gemini]({{< ref "docs/development/calling-llms/chat-models/providers#gemini" >}}) | ❌ | ✅ | +| [Ollama]({{< ref "docs/development/calling-llms/chat-models/providers#ollama" >}}) | ✅ | ✅ | +| [OpenAI]({{< ref "docs/development/calling-llms/chat-models/providers#openai" >}}) | ✅ | ✅ | +| [Alibaba Cloud Model Studio (DashScope)]({{< ref "docs/development/calling-llms/chat-models/providers#alibaba-cloud-model-studio-dashscope" >}}) | ✅ | ❌ | +| [vLLM]({{< ref "docs/development/calling-llms/chat-models/providers#vllm" >}}) | ✅ | ✅ | +| [Watsonx (IBM watsonx.ai)]({{< ref "docs/development/calling-llms/chat-models/providers#watsonx-ibm-watsonxai" >}}) | ✅ | ✅ | **Embedding Models** | Provider | Python | Java | |---|---|---| -| [Amazon Bedrock]({{< ref "docs/development/embedding_models#amazon-bedrock" >}}) | ❌ | ✅ | -| [Ollama]({{< ref "docs/development/embedding_models#ollama" >}}) | ✅ | ✅ | -| [OpenAI]({{< ref "docs/development/embedding_models#openai" >}}) | ✅ | ✅ | -| [Alibaba Cloud Model Studio (DashScope)]({{< ref "docs/development/embedding_models#alibaba-cloud-model-studio-dashscope" >}}) | ✅ | ❌ | +| [Amazon Bedrock]({{< ref "docs/development/vector-search/embedding_models/providers#amazon-bedrock" >}}) | ❌ | ✅ | +| [Ollama]({{< ref "docs/development/vector-search/embedding_models/providers#ollama" >}}) | ✅ | ✅ | +| [OpenAI]({{< ref "docs/development/vector-search/embedding_models/providers#openai" >}}) | ✅ | ✅ | +| [Alibaba Cloud Model Studio (DashScope)]({{< ref "docs/development/vector-search/embedding_models/providers#alibaba-cloud-model-studio-dashscope" >}}) | ✅ | ❌ | **Vector Stores** | Provider | Python | Java | |---|---|---| -| [Amazon OpenSearch]({{< ref "docs/development/vector_stores#amazon-opensearch" >}}) | ❌ | ✅ | -| [Amazon S3 Vectors]({{< ref "docs/development/vector_stores#amazon-s3-vectors" >}}) | ❌ | ✅ | -| [Chroma]({{< ref "docs/development/vector_stores#chroma" >}}) | ✅ | ❌ | -| [Elasticsearch]({{< ref "docs/development/vector_stores#elasticsearch" >}}) | ❌ | ✅ | -| [Mem0]({{< ref "docs/development/vector_stores#mem0" >}}) | ✅ | ❌ | -| [Milvus]({{< ref "docs/development/vector_stores#milvus" >}}) | ❌ | ✅ | +| [Amazon OpenSearch]({{< ref "docs/development/vector-search/vector_stores/providers#amazon-opensearch" >}}) | ❌ | ✅ | +| [Amazon S3 Vectors]({{< ref "docs/development/vector-search/vector_stores/providers#amazon-s3-vectors" >}}) | ❌ | ✅ | +| [Chroma]({{< ref "docs/development/vector-search/vector_stores/providers#chroma" >}}) | ✅ | ❌ | +| [Elasticsearch]({{< ref "docs/development/vector-search/vector_stores/providers#elasticsearch" >}}) | ❌ | ✅ | +| [Mem0]({{< ref "docs/development/vector-search/vector_stores/providers#mem0" >}}) | ✅ | ❌ | +| [Milvus]({{< ref "docs/development/vector-search/vector_stores/providers#milvus" >}}) | ❌ | ✅ | **MCP Server** | Provider | Python | Java | |---|---|---| -| [MCP Server]({{< ref "docs/development/mcp" >}}) | ✅ | ✅ | +| [MCP Server]({{< ref "docs/development/calling-llms/mcp" >}}) | ✅ | ✅ | -> **Note**: Java native MCP support requires **JDK 17+**. For JDK 11-16, the framework automatically uses the Python SDK via cross-language support, which works seamlessly without additional configuration. See [MCP]({{< ref "docs/development/mcp" >}}) for details. +> **Note**: Java native MCP support requires **JDK 17+**. For JDK 11-16, the framework automatically uses the Python SDK via cross-language support, which works seamlessly without additional configuration. See [MCP]({{< ref "docs/development/calling-llms/mcp" >}}) for details. ## Q4: How to run agent in IDE. -To avoid potential conflict with Flink cluster, the scope of the dependencies related to Flink and Flink Agents for agent job are provided. See [Maven Dependencies]({{< ref "docs/get-started/installation#maven-dependencies-for-java" >}}) for details. +To avoid potential conflict with Flink cluster, the scope of the dependencies related to Flink and Flink Agents for agent job are provided. See [Maven Dependencies]({{< ref "docs/get-started/setup#maven-dependencies-for-java" >}}) for details. To run the examples in IDE, users must enable the IDE feature: `add dependencies with provided scope to classpath`. * For **IDEA**, edit the **`Run/Debug Configuration`** and enable **`add dependencies with provided scope to classpath`**. See [Run/Debug Configuration](https://www.jetbrains.com/help/idea/run-debug-configuration-scala.html) for details. diff --git a/docs/content/docs/get-started/overview.md b/docs/content/docs/get-started/overview.md index 2fbef0c32c..fb8defa92a 100644 --- a/docs/content/docs/get-started/overview.md +++ b/docs/content/docs/get-started/overview.md @@ -41,12 +41,13 @@ The key features of Apache Flink Agents include: ## Getting Started -To get started with Apache Flink Agents, you can checkout the following quickstarts: +Start with [Setup]({{< ref "docs/get-started/setup" >}}), then choose a Quickstart: -- [Workflow Agent Quickstart]({{< ref "docs/get-started/quickstart/workflow_agent" >}}) -- [ReAct Agent Quickstart]({{< ref "docs/get-started/quickstart/react_agent" >}}) -- [Skills Agent Quickstart]({{< ref "docs/get-started/quickstart/skills_agent" >}}) -- [YAML Agent Quickstart]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) +- [Build an Agent]({{< ref "docs/get-started/quickstart/create-agent" >}}) +- [Use the Built-in ReAct Agent]({{< ref "docs/get-started/quickstart/react_agent" >}}) +- [Define an Agent with YAML]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) + +For an optional authoring aid, see the [Coding-agent Skill](https://github.com/apache/flink-agents/tree/main/dev/agent-skills/flink-agents-dev). ## Preview Status diff --git a/docs/content/docs/get-started/quickstart/_index.md b/docs/content/docs/get-started/quickstart/_index.md index ade34fbdce..75d963218a 100644 --- a/docs/content/docs/get-started/quickstart/_index.md +++ b/docs/content/docs/get-started/quickstart/_index.md @@ -2,7 +2,7 @@ title: Quickstart bold: true bookCollapseSection: true -weight: 2 +weight: 3 --- + +- [Build an Agent]({{< ref "docs/get-started/quickstart/create-agent" >}}) +- [Use the Built-in ReAct Agent]({{< ref "docs/get-started/quickstart/react_agent" >}}) +- [Define an Agent with YAML]({{< ref "docs/get-started/quickstart/yaml_agent" >}}) diff --git a/docs/content/docs/get-started/quickstart/create-agent/_index.md b/docs/content/docs/get-started/quickstart/create-agent/_index.md new file mode 100644 index 0000000000..c2d61c525c --- /dev/null +++ b/docs/content/docs/get-started/quickstart/create-agent/_index.md @@ -0,0 +1,42 @@ +--- +title: "Build an Agent" +weight: 1 +type: docs +bookCollapseSection: true +aliases: + - /docs/get-started/quickstart/workflow_agent/ +--- + + +## Overview + +An Agent in Flink-Agents is an agent whose reasoning and behavior are organized as a directed workflow of modular steps, called actions, connected by events. This design is inspired by the need to orchestrate complex, multi-stage tasks in a transparent, extensible, and data-centric way, leveraging Apache Flink's streaming architecture. + +This quickstart introduces two small, progressive streaming examples that demonstrate how to build LLM-powered workflows with Flink Agents: + +- **Review Analysis**: Processes a stream of product reviews and uses a single agent to extract a rating (1–5) and unsatisfied reasons from each review. + +- **Product Improvement Suggestions**: Builds on the first example by aggregating per-review analysis in windows to produce product-level summaries (score distribution and common complaints), then applies a second agent to generate concrete improvement suggestions for each product. + +Together, these examples show how to build a multi-agent workflow with Flink Agents and run it on a Flink standalone cluster. + + +- [Build Your First Agent]({{< ref "docs/get-started/quickstart/create-agent/single-agent" >}}) +- [Connect Multiple Agents]({{< ref "docs/get-started/quickstart/create-agent/multiple-agents" >}}) diff --git a/docs/content/docs/get-started/quickstart/create-agent/multiple-agents.md b/docs/content/docs/get-started/quickstart/create-agent/multiple-agents.md new file mode 100644 index 0000000000..555fac8dad --- /dev/null +++ b/docs/content/docs/get-started/quickstart/create-agent/multiple-agents.md @@ -0,0 +1,154 @@ +--- +title: "Connect Multiple Agents" +weight: 2 +type: docs +--- + + +## Overview + +This example builds on [Build Your First Agent]({{< ref "docs/get-started/quickstart/create-agent/single-agent" >}}). It connects two Agents in one Flink job, aggregating per-review analysis in windows and applying a second Agent to generate product improvement suggestions. + +Reuse the [environment and connection setup]({{< ref "docs/get-started/quickstart/create-agent/single-agent#prepare-agents-execution-environment" >}}) and [example preparation]({{< ref "docs/get-started/quickstart/create-agent/single-agent#run-the-example" >}}) from the first walkthrough. + +The code for the `ProductSuggestionAgent`, which is used to generate product improvement suggestions based on the aggregated analysis results, is similar to the `ReviewAnalysisAgent`. + +### Read Input via the Table API + +The multiple-agent example reads product reviews through the Flink Table API instead of a DataStream. Because Table rows arrive as `Row` (Java) / `dict` (Python) instead of a `ProductReview` POJO, it uses a dedicated `TableReviewAnalysisAgent` — identical to `ReviewAnalysisAgent` except its `process_input` action reads the `id` and `review` columns out of the row, and it ships a key selector that extracts the key from that row. + +{{< tabs "Create the Table Agent" >}} + +{{< tab "Python" >}} +```python +class TableKeySelector(KeySelector): + """Extract the partition key from a Table row (dict).""" + + def get_key(self, value: Any) -> str: + return str(value["id"]) + + +class TableReviewAnalysisAgent(Agent): + # prompt / tool / chat_model_setup are identical to ReviewAnalysisAgent. + + @action(InputEvent.EVENT_TYPE) + @staticmethod + def process_input(event: Event, ctx: RunnerContext) -> None: + # Table input arrives as a dict keyed by column name, not a POJO. + input_dict = InputEvent.from_event(event).input + product_id = str(input_dict["id"]) + review_text = str(input_dict["review"]) + ... +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +public class TableReviewAnalysisAgent extends Agent { + + /** Extract the partition key from a Table Row. */ + public static class RowKeySelector implements KeySelector { + @Override + public String getKey(Object value) { + return (String) ((Row) value).getField("id"); + } + } + + // prompt / tool / chat model setup are identical to ReviewAnalysisAgent. + + @Action(EventType.InputEvent) + public static void processInput(Event event, RunnerContext ctx) { + // Table input arrives as a Row keyed by column name, not a POJO. + Row row = (Row) InputEvent.fromEvent(event).getInput(); + String productId = (String) row.getField("id"); + String reviewText = (String) row.getField("review"); + ... + } +} +``` +{{< /tab >}} + +{{< /tabs >}} + +Register the input file as a table and feed it to the agent with `from_table` / `fromTable`, providing the key selector: + +{{< tabs "Integrate via Table API" >}} + +{{< tab "Python" >}} +```python +input_table = t_env.from_path("product_reviews") + +review_analysis_res_stream = ( + agents_env.from_table(input=input_table, key_selector=TableKeySelector()) + .apply(TableReviewAnalysisAgent()) + .to_datastream(ProductReviewAnalysisRes) +) +``` +{{< /tab >}} + +{{< tab "Java" >}} +```java +Table inputTable = tableEnv.from("product_reviews"); + +DataStream reviewAnalysisResStream = + agentsEnv + .fromTable(inputTable, new TableReviewAnalysisAgent.RowKeySelector()) + .apply(new TableReviewAnalysisAgent()) + .toDataStream(ProductReviewAnalysisRes.class); +``` +{{< /tab >}} + +{{< /tabs >}} + +See [Integrate with Flink]({{< ref "docs/development/integrate_with_flink#fromto-flink-table-api" >}}) for the full Table API integration reference. + + +## Run the Example + +### Submit Flink Agents Job to Standalone Flink Cluster + +#### Submit to Flink Cluster + +{{< tabs "Submit to Flink Cluster" >}} + +{{< tab "Python" >}} +```bash +export PYTHONPATH=$(python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])') + +# Run product suggestion example +$FLINK_HOME/bin/flink run -py ./flink-agents/python/flink_agents/examples/quickstart/workflow_multiple_agent_example.py +# or submit the example python file in installed flink-agents wheel +$FLINK_HOME/bin/flink run -py $PYTHONPATH/flink_agents/examples/quickstart/workflow_multiple_agent_example.py +``` +{{< /tab >}} + +{{< tab "Java" >}} +```bash +# Run product suggestion example +$FLINK_HOME/bin/flink run -c org.apache.flink.agents.examples.WorkflowMultipleAgentExample ./flink-agents/examples/target/flink-agents-examples-$VERSION.jar +``` +{{< /tab >}} + +{{< /tabs >}} + +Now you should see a Flink job submitted to the Flink Cluster in Flink web UI [localhost:8081]( +localhost:8081) + +After a few minutes, you can check for the output in the TaskManager output log. diff --git a/docs/content/docs/get-started/quickstart/workflow_agent.md b/docs/content/docs/get-started/quickstart/create-agent/single-agent.md similarity index 74% rename from docs/content/docs/get-started/quickstart/workflow_agent.md rename to docs/content/docs/get-started/quickstart/create-agent/single-agent.md index d71ca7ff9e..526b7108e7 100644 --- a/docs/content/docs/get-started/quickstart/workflow_agent.md +++ b/docs/content/docs/get-started/quickstart/create-agent/single-agent.md @@ -1,5 +1,5 @@ --- -title: 'Workflow Agent' +title: "Build Your First Agent" weight: 1 type: docs --- @@ -24,15 +24,7 @@ under the License. ## Overview -A workflow style agent in Flink-Agents is an agent whose reasoning and behavior are organized as a directed workflow of modular steps, called actions, connected by events. This design is inspired by the need to orchestrate complex, multi-stage tasks in a transparent, extensible, and data-centric way, leveraging Apache Flink's streaming architecture. - -This quickstart introduces two small, progressive streaming examples that demonstrate how to build LLM-powered workflows with Flink Agents: - -- **Review Analysis**: Processes a stream of product reviews and uses a single agent to extract a rating (1–5) and unsatisfied reasons from each review. - -- **Product Improvement Suggestions**: Builds on the first example by aggregating per-review analysis in windows to produce product-level summaries (score distribution and common complaints), then applies a second agent to generate concrete improvement suggestions for each product. - -Together, these examples show how to build a multi-agent workflow with Flink Agents and run it on a Flink standalone cluster. +Build an Agent that analyzes a stream of product reviews. ## Code Walkthrough @@ -77,7 +69,7 @@ agentsEnv.addResource( ### Create the Agents -Below is the example code for the `ReviewAnalysisAgent`, which is used to analyze the product reviews and generate a satisfaction score and potential reasons for dissatisfaction. It demonstrates how to define the prompt, tool, chat model, and action for the agent. Also, it shows how to process the chat response and send the output event. For more details, please refer to the [Workflow Agent]({{< ref "docs/development/workflow_agent" >}}) documentation. +Below is the example code for the `ReviewAnalysisAgent`, which is used to analyze the product reviews and generate a satisfaction score and potential reasons for dissatisfaction. It demonstrates how to define the prompt, tool, chat model, and action for the agent. Also, it shows how to process the chat response and send the output event. For more details, please refer to the [Agent]({{< ref "docs/development/create-agent/overview" >}}) documentation. {{< tabs "Create the Agents" >}} @@ -270,8 +262,6 @@ public class ReviewAnalysisAgent extends Agent { {{< /tabs >}} -The code for the `ProductSuggestionAgent`, which is used to generate product improvement suggestions based on the aggregated analysis results, is similar to the `ReviewAnalysisAgent`. - ### Integrate the Agents with Flink Create the input DataStream by reading the product reviews from a text file as a streaming source, and use the `ReviewAnalysisAgent` to analyze the product reviews and generate the result DataStream. Finally print the result DataStream. @@ -346,95 +336,6 @@ agentsEnv.execute("Workflow Agent Example Job"); {{< /tabs >}} -### Read Input via the Table API - -The multiple-agent example reads product reviews through the Flink Table API instead of a DataStream. Because Table rows arrive as `Row` (Java) / `dict` (Python) instead of a `ProductReview` POJO, it uses a dedicated `TableReviewAnalysisAgent` — identical to `ReviewAnalysisAgent` except its `process_input` action reads the `id` and `review` columns out of the row, and it ships a key selector that extracts the key from that row. - -{{< tabs "Create the Table Agent" >}} - -{{< tab "Python" >}} -```python -class TableKeySelector(KeySelector): - """Extract the partition key from a Table row (dict).""" - - def get_key(self, value: Any) -> str: - return str(value["id"]) - - -class TableReviewAnalysisAgent(Agent): - # prompt / tool / chat_model_setup are identical to ReviewAnalysisAgent. - - @action(InputEvent.EVENT_TYPE) - @staticmethod - def process_input(event: Event, ctx: RunnerContext) -> None: - # Table input arrives as a dict keyed by column name, not a POJO. - input_dict = InputEvent.from_event(event).input - product_id = str(input_dict["id"]) - review_text = str(input_dict["review"]) - ... -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -public class TableReviewAnalysisAgent extends Agent { - - /** Extract the partition key from a Table Row. */ - public static class RowKeySelector implements KeySelector { - @Override - public String getKey(Object value) { - return (String) ((Row) value).getField("id"); - } - } - - // prompt / tool / chat model setup are identical to ReviewAnalysisAgent. - - @Action(EventType.InputEvent) - public static void processInput(Event event, RunnerContext ctx) { - // Table input arrives as a Row keyed by column name, not a POJO. - Row row = (Row) InputEvent.fromEvent(event).getInput(); - String productId = (String) row.getField("id"); - String reviewText = (String) row.getField("review"); - ... - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - -Register the input file as a table and feed it to the agent with `from_table` / `fromTable`, providing the key selector: - -{{< tabs "Integrate via Table API" >}} - -{{< tab "Python" >}} -```python -input_table = t_env.from_path("product_reviews") - -review_analysis_res_stream = ( - agents_env.from_table(input=input_table, key_selector=TableKeySelector()) - .apply(TableReviewAnalysisAgent()) - .to_datastream(ProductReviewAnalysisRes) -) -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -Table inputTable = tableEnv.from("product_reviews"); - -DataStream reviewAnalysisResStream = - agentsEnv - .fromTable(inputTable, new TableReviewAnalysisAgent.RowKeySelector()) - .apply(new TableReviewAnalysisAgent()) - .toDataStream(ProductReviewAnalysisRes.class); -``` -{{< /tab >}} - -{{< /tabs >}} - -See [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}#fromto-flink-table-api) for the full Table API integration reference. - ## Run the Example ### Prerequisites @@ -448,7 +349,7 @@ See [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}#f #### Prepare Flink and Flink Agents -Follow the [installation]({{< ref "docs/get-started/installation" >}}) instructions to setup Flink and the Flink Agents. +Follow the [Setup]({{< ref "docs/get-started/setup" >}}) instructions to setup Flink and the Flink Agents. #### Clone the Flink Agents Repository (if not done already) @@ -475,7 +376,7 @@ $FLINK_HOME/bin/start-cluster.sh {{< /tab >}} {{< tab "Java" >}} -1. Build Flink Agents from source to generate example jar. See [installation]({{< ref "docs/get-started/installation" >}}) for more details. +1. Build Flink Agents from source to generate example jar. See [Setup]({{< ref "docs/get-started/setup" >}}) for more details. 2. Start the Flink cluster ```bash $FLINK_HOME/bin/start-cluster.sh @@ -521,11 +422,6 @@ export PYTHONPATH=$(python -c 'import sysconfig; print(sysconfig.get_paths()["pu $FLINK_HOME/bin/flink run -py ./flink-agents/python/flink_agents/examples/quickstart/workflow_single_agent_example.py # or submit the example python file in installed flink-agents wheel $FLINK_HOME/bin/flink run -py $PYTHONPATH/flink_agents/examples/quickstart/workflow_single_agent_example.py - -# Run product suggestion example -$FLINK_HOME/bin/flink run -py ./flink-agents/python/flink_agents/examples/quickstart/workflow_multiple_agent_example.py -# or submit the example python file in installed flink-agents wheel -$FLINK_HOME/bin/flink run -py $PYTHONPATH/flink_agents/examples/quickstart/workflow_multiple_agent_example.py ``` {{< /tab >}} @@ -533,9 +429,6 @@ $FLINK_HOME/bin/flink run -py $PYTHONPATH/flink_agents/examples/quickstart/work ```bash # Run review analysis example $FLINK_HOME/bin/flink run -c org.apache.flink.agents.examples.WorkflowSingleAgentExample ./flink-agents/examples/target/flink-agents-examples-$VERSION.jar - -# Run product suggestion example -$FLINK_HOME/bin/flink run -c org.apache.flink.agents.examples.WorkflowMultipleAgentExample ./flink-agents/examples/target/flink-agents-examples-$VERSION.jar ``` {{< /tab >}} @@ -545,3 +438,5 @@ Now you should see a Flink job submitted to the Flink Cluster in Flink web UI [l localhost:8081) After a few minutes, you can check for the output in the TaskManager output log. + +Continue with [Connect Multiple Agents]({{< ref "docs/get-started/quickstart/create-agent/multiple-agents" >}}), or explore [Skills]({{< ref "docs/development/calling-llms/skills" >}}). diff --git a/docs/content/docs/get-started/quickstart/parallel_llm.md b/docs/content/docs/get-started/quickstart/parallel_llm.md index 0f941a350b..c161c34b32 100644 --- a/docs/content/docs/get-started/quickstart/parallel_llm.md +++ b/docs/content/docs/get-started/quickstart/parallel_llm.md @@ -2,6 +2,7 @@ title: 'Parallel LLM Calls' weight: 4 type: docs +bookHidden: true --- - -## Overview - -A [Skill]({{< ref "docs/development/skills" >}}) is a self-contained package of instructions, and optionally scripts and reference files, that teaches the agent how to perform a specialized task. Skills are loaded with *progressive disclosure*: only each skill's name and description are shown to the agent up front, and the full instructions are pulled in on demand when the agent decides a skill is relevant. - -This quickstart builds a small streaming agent that answers arithmetic questions. Rather than letting the LLM compute by itself, the agent exposes a `math-calculator` skill; for each question the agent loads the skill and follows its instructions to compute the result with the `bc` calculator through the built-in `bash` tool. It demonstrates the full skill lifecycle — discovery, activation, and execution — in a Flink streaming job. - -## Code Walkthrough - -### Define the Skill - -A skill is a directory containing a `SKILL.md` file with YAML frontmatter (loaded at discovery time) and a Markdown body (loaded on activation). Here is `skills/math-calculator/SKILL.md`: - -````markdown ---- -name: math-calculator -description: Calculate mathematical expressions using shell commands. Use when the user asks to perform arithmetic calculations like addition, subtraction, multiplication, division, or powers. -license: Apache-2.0 -compatibility: Requires bash with bc (basic calculator) ---- - -# Math Calculator Skill - -## When to Use -Use this skill when the user asks to evaluate a numeric expression. - -## Method -Pipe the expression into the `bc` (basic calculator) command: - -```bash -echo "(2 + 3) * 4" | bc -# Output: 20 -``` -```` - -### Create the Agent - -The agent declares where to load skills from with the `@skills`/`@Skills` decorator/annotation, and enables the skill on its chat model by listing it in `skills` together with the `allowed_commands` whitelist for the `bash` tool. For more details, please refer to the [Skills]({{< ref "docs/development/skills" >}}) documentation. - -{{< tabs "Create the Agent" >}} - -{{< tab "Python" >}} -```python -class MathAgent(Agent): - """An agent that answers arithmetic questions using the math-calculator skill.""" - - @skills - @staticmethod - def my_skills() -> Skills: - """Declare where to load skills from.""" - # Skills are bundled under this example package, loaded by package name. - return Skills.from_package( - ("flink_agents.examples.quickstart", "resources/skills") - ) - - @prompt - @staticmethod - def system_prompt() -> Prompt: - """System prompt instructing the agent to use the skill.""" - return Prompt.from_messages( - messages=[ - ChatMessage.system( - "You are a helpful math assistant. Use the " - "math-calculator skill when asked to evaluate an expression. " - "You must load the skill first and strictly follow its " - "instructions. Reply with only the final numeric result." - ) - ], - ) - - @chat_model_setup - @staticmethod - def math_model() -> ResourceDescriptor: - """ChatModel with the math-calculator skill enabled.""" - return ResourceDescriptor( - clazz=ResourceName.ChatModel.OLLAMA_SETUP, - connection="ollama_server", - model="qwen3.5:9b", - prompt="system_prompt", - # Expose the declared skill to this model by name. - skills=["math-calculator"], - # Whitelist the shell commands the built-in bash tool may run. - allowed_commands=["echo", "bc"], - ) - - @action(EventType.InputEvent) - @staticmethod - def process_input(event: Event, ctx: RunnerContext) -> None: - """Process input event and send a chat request to evaluate the question.""" - question: str = InputEvent.from_event(event).input - ctx.send_event( - ChatRequestEvent( - model="math_model", - messages=[ChatMessage.user(question)], - ) - ) - - @action(EventType.ChatResponseEvent) - @staticmethod - def process_chat_response(event: Event, ctx: RunnerContext) -> None: - """Process chat response event and send the answer as output.""" - chat_response = ChatResponseEvent.from_event(event) - ctx.send_event(OutputEvent(output=chat_response.response.text)) -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -/** An agent that answers arithmetic questions using the math-calculator skill. */ -public class MathAgent extends Agent { - - /** Load skills from the skills/ directory packaged on the classpath. */ - @org.apache.flink.agents.api.annotation.Skills - public static Skills mySkills() { - return Skills.fromClasspath("skills"); - } - - /** System prompt instructing the agent to use the skill. */ - @Prompt - public static org.apache.flink.agents.api.prompt.Prompt systemPrompt() { - return org.apache.flink.agents.api.prompt.Prompt.fromMessages( - Collections.singletonList( - new ChatMessage( - MessageRole.SYSTEM, - "You are a helpful math assistant. Use the math-calculator skill " - + "when asked to evaluate an expression. You must load the " - + "skill first and strictly follow its instructions. Reply " - + "with only the final numeric result."))); - } - - /** ChatModel with the math-calculator skill enabled. */ - @ChatModelSetup - public static ResourceDescriptor mathModel() { - return ResourceDescriptor.Builder.newBuilder(ResourceName.ChatModel.OLLAMA_SETUP) - .addInitialArgument("connection", "ollamaChatModelConnection") - .addInitialArgument("model", "qwen3.5:9b") - .addInitialArgument("prompt", "systemPrompt") - // Expose the declared skill to this model by name. - .addInitialArgument("skills", List.of("math-calculator")) - // Whitelist the shell commands the built-in bash tool may run. - .addInitialArgument("allowed_commands", List.of("echo", "bc")) - .build(); - } - - /** Process input event and send a chat request to evaluate the question. */ - @Action(EventType.InputEvent) - public static void processInput(InputEvent event, RunnerContext ctx) { - ctx.sendEvent( - new ChatRequestEvent( - "mathModel", - Collections.singletonList( - new ChatMessage(MessageRole.USER, (String) event.getInput())))); - } - - /** Process chat response event and send the answer as output. */ - @Action(EventType.ChatResponseEvent) - public static void processChatResponse(ChatResponseEvent event, RunnerContext ctx) { - ctx.sendEvent(new OutputEvent(event.getResponse().getText())); - } -} -``` -{{< /tab >}} - -{{< /tabs >}} - -**Key points:** -- `@skills`/`@Skills` declares a skill source. `Skills.from_package` (Python) loads skills bundled inside an installed package by `(package, resource)`; `Skills.fromClasspath` (Java) loads them from a classpath resource packaged in the jar. -- A declared skill is exposed to a model only when the model lists it in `skills`. The `load_skill` and `bash` tools are then added automatically. -- `allowed_commands` whitelists the shell commands the `bash` tool may run — keep it as narrow as the skill requires. -- The built-in `bash` tool rejects file redirects and execution-changing environment assignments. File-descriptor duplication and closure remain supported; see the [Skills]({{< ref "docs/development/skills" >}}) documentation for the complete security contract. - -### Integrate the Agent with Flink - -Register the Ollama chat model connection, create a stream of questions, and apply the agent. - -{{< tabs "Integrate the Agent with Flink" >}} - -{{< tab "Python" >}} -```python -# Set up the Flink streaming environment and the Agents execution environment. -env = StreamExecutionEnvironment.get_execution_environment() -agents_env = AgentsExecutionEnvironment.get_execution_environment(env) - -# Add Ollama chat model connection to be used by the MathAgent. -agents_env.add_resource( - "ollama_server", ResourceType.CHAT_MODEL_CONNECTION, ollama_server_descriptor -) - -# A small stream of arithmetic questions to answer. -question_stream = env.from_collection( - ["What is (2 + 3) * 4?", "Compute 2 ^ 10.", "What is 144 divided by 12?"] -) - -# Use the MathAgent to answer each question with the math-calculator skill. -answer_stream = ( - agents_env.from_datastream(input=question_stream, key_selector=lambda x: x) - .apply(MathAgent()) - .to_datastream() -) - -# Print the answers to stdout. -answer_stream.print() -# Execute the Flink pipeline with the Flink job name. -agents_env.execute("Skills Agent Example Job") -``` -{{< /tab >}} - -{{< tab "Java" >}} -```java -// Set up the Flink streaming environment and the Agents execution environment. -StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); -env.setParallelism(1); -AgentsExecutionEnvironment agentsEnv = - AgentsExecutionEnvironment.getExecutionEnvironment(env); - -// Add Ollama chat model connection to be used by the MathAgent. -agentsEnv.addResource( - "ollamaChatModelConnection", - ResourceType.CHAT_MODEL_CONNECTION, - CustomTypesAndResources.OLLAMA_SERVER_DESCRIPTOR); - -// A small stream of arithmetic questions to answer. -DataStream questionStream = - env.fromData("What is (2 + 3) * 4?", "Compute 2 ^ 10.", "What is 144 divided by 12?"); - -// Use the MathAgent to answer each question with the math-calculator skill. -DataStream answerStream = - agentsEnv.fromDataStream(questionStream).apply(new MathAgent()).toDataStream(); - -// Print the answers to stdout. -answerStream.print(); -// Execute the Flink pipeline with the Flink job name. -agentsEnv.execute("Skills Agent Example Job"); -``` -{{< /tab >}} - -{{< /tabs >}} - -{{< hint info >}} -See [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}) for details on integrating agents with the Flink DataStream and Table API. -{{< /hint >}} - -## Run the Example - -### Prerequisites - -* Unix-like environment (we use Linux, Mac OS X, Cygwin, WSL) -* Git -* Java 11+ -* Python 3.10, 3.11 or 3.12 -* `bc` (basic calculator), used by the `math-calculator` skill — preinstalled on most Unix-like systems - -### Preparation - -#### Prepare Flink and Flink Agents - -Follow the [installation]({{< ref "docs/get-started/installation" >}}) instructions to setup Flink and the Flink Agents. - -#### Clone the Flink Agents Repository (if not done already) - -```bash -git clone https://github.com/apache/flink-agents.git -cd flink-agents -``` - -{{< hint info >}} -For python examples, you can skip this step and submit the python file in installed flink-agents wheel. -{{< /hint >}} - -#### Deploy a Standalone Flink Cluster - -You can deploy a standalone Flink cluster in your local environment with the following command. - -{{< tabs "Deploy a Standalone Flink Cluster" >}} - -{{< tab "Python" >}} -```bash -export PYTHONPATH=$(python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])') -$FLINK_HOME/bin/start-cluster.sh -``` -{{< /tab >}} - -{{< tab "Java" >}} -1. Build Flink Agents from source to generate example jar. See [installation]({{< ref "docs/get-started/installation" >}}) for more details. -2. Start the Flink cluster - ```bash - $FLINK_HOME/bin/start-cluster.sh - ``` - -{{< hint info >}} -To run example on JDK 21+, append jvm option `--add-exports=java.base/jdk.internal.vm=ALL-UNNAMED` to [env.java.opts.all](https://nightlies.apache.org/flink/flink-docs-stable/docs/deployment/config/#env-java-opts-all) in `$FLINK_HOME/conf/config.yaml` before starting the Flink cluster. -{{< /hint >}} -{{< /tab >}} - -{{< /tabs >}} -You can refer to the [local cluster](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/try-flink/local_installation/#starting-and-stopping-a-local-cluster) instructions for more detailed steps. - -{{< hint info >}} -If you can't navigate to the web UI at [localhost:8081](localhost:8081), you can find the reason in `$FLINK_HOME/log`. If the reason is port conflict, you can change the port in `$FLINK_HOME/conf/config.yaml`. -{{< /hint >}} - -#### Prepare Ollama - -Download and install Ollama from the official [website](https://ollama.com/download). - -{{< hint info >}} -Ollama server **0.9.0** or higher is required. -{{< /hint >}} - -Then pull the qwen3.5:9b model, which is required by the quickstart examples. - -```bash -ollama pull qwen3.5:9b -``` - -### Submit Flink Agents Job to Standalone Flink Cluster - -#### Submit to Flink Cluster - -{{< tabs "Submit to Flink Cluster" >}} - -{{< tab "Python" >}} -```bash -export PYTHONPATH=$(python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])') - -# Run the agent skills example -$FLINK_HOME/bin/flink run -py ./flink-agents/python/flink_agents/examples/quickstart/skills_agent_example.py -# or submit the example python file in installed flink-agents wheel -$FLINK_HOME/bin/flink run -py $PYTHONPATH/flink_agents/examples/quickstart/skills_agent_example.py -``` -{{< /tab >}} - -{{< tab "Java" >}} -```bash -$FLINK_HOME/bin/flink run -c org.apache.flink.agents.examples.SkillsAgentExample ./flink-agents/examples/target/flink-agents-examples-$VERSION.jar -``` -{{< /tab >}} - -{{< /tabs >}} - -Now you should see a Flink job submitted to the Flink Cluster in Flink web UI [localhost:8081](localhost:8081). - -After a few minutes, you can check for the output in the TaskManager output log. diff --git a/docs/content/docs/get-started/quickstart/yaml_agent.md b/docs/content/docs/get-started/quickstart/yaml_agent.md index 949995c48e..d0253cacde 100644 --- a/docs/content/docs/get-started/quickstart/yaml_agent.md +++ b/docs/content/docs/get-started/quickstart/yaml_agent.md @@ -1,5 +1,5 @@ --- -title: 'YAML Agent' +title: "Define an Agent with YAML" weight: 3 type: docs --- @@ -26,15 +26,15 @@ under the License. The YAML API lets you declare a Flink Agents application — the agent's actions, tools, prompts, chat models, and other resources — as a single declarative file, and load it into an `AgentsExecutionEnvironment` with one call. Your action and tool implementations still live in Python or Java; the YAML file only describes how those pieces are wired together. -This quickstart runs the same **Review Analysis** workflow as the [Workflow Agent Quickstart]({{< ref "docs/get-started/quickstart/workflow_agent" >}}) — extracting a satisfaction score and dissatisfaction reasons from a stream of product reviews — but declares the agent in YAML and loads it with `load_yaml` / `loadYaml`. The action functions and the `notify_shipping_manager` tool are reused from the workflow-agent example as static methods, so this quickstart shows the **smallest possible delta** between a code-defined agent and a YAML-declared agent. +This quickstart runs the same **Review Analysis** workflow as the [Build Your First Agent]({{< ref "docs/get-started/quickstart/create-agent/single-agent" >}}) — extracting a satisfaction score and dissatisfaction reasons from a stream of product reviews — but declares the agent in YAML and loads it with `load_yaml` / `loadYaml`. The action functions and the `notify_shipping_manager` tool are reused from the single-Agent example as static methods, so this quickstart shows the **smallest possible delta** between a code-defined agent and a YAML-declared agent. -For the full reference of the YAML format, see the [YAML API]({{< ref "docs/development/yaml" >}}) documentation. +For the full reference of the YAML format, see the [YAML API]({{< ref "docs/development/create-agent/yaml" >}}) documentation. ## Code Walkthrough ### Declare the Agent in YAML -The whole agent — chat model, prompt, tool, and the two actions — is declared in `yaml_review_analysis_agent.yaml`. The `function:` strings point at static methods on the existing `ReviewAnalysisAgent` class from the workflow-agent quickstart. +The whole agent — chat model, prompt, tool, and the two actions — is declared in `yaml_review_analysis_agent.yaml`. The `function:` strings point at static methods on the existing `ReviewAnalysisAgent` class from the single-Agent quickstart. {{< tabs "Declare Agent YAML" >}} @@ -44,7 +44,7 @@ agents: - name: review_analysis_agent description: | YAML-declared review analysis agent. Reuses the static methods of - ReviewAnalysisAgent from the workflow_agent quickstart as actions + ReviewAnalysisAgent from the single-Agent quickstart as actions and tool, but wires everything together through this YAML file instead of class-level decorators. @@ -112,7 +112,7 @@ agents: - name: review_analysis_agent description: | YAML-declared review analysis agent. Reuses the static methods of - ReviewAnalysisAgent from the workflow_agent quickstart as actions + ReviewAnalysisAgent from the single-Agent quickstart as actions and tool, but wires everything together through this YAML file instead of class-level annotations. @@ -187,7 +187,7 @@ agents: A few things to notice in the YAML above: -- `clazz: ollama` is an alias resolved by the loader to the full Ollama chat-model class — see the alias table in the [YAML API]({{< ref "docs/development/yaml#class-aliases" >}}) doc. +- `clazz: ollama` is an alias resolved by the loader to the full Ollama chat-model class — see the alias table in the [YAML API]({{< ref "docs/development/create-agent/yaml#class-aliases" >}}) doc. - `trigger_conditions: [input]` / `[chat_response]` use **event aliases** for the framework's built-in events. An alias replaces only a complete event-type entry; it never rewrites text inside a condition expression or quoted event type. - `function:` strings use the `:` format. The right side is the class-qualified method name, so the YAML reuses the same `process_input` / `processInput` static methods the original `ReviewAnalysisAgent` already defines. - The prompt is declared inline as a `messages:` list and referenced from the chat-model setup by name. @@ -280,7 +280,7 @@ agentsEnv.execute("YAML Agent Example Job"); {{< /tabs >}} -The key difference from the code-defined workflow agent quickstart is the pair of calls: +The key difference from the code-defined Agent quickstart is the pair of calls: 1. `agents_env.load_yaml(...)` / `agentsEnv.loadYaml(...)` — parses the YAML and registers the declared agent(s) and shared resources on the environment. 2. `.apply("review_analysis_agent")` — looks the agent up **by name** instead of passing an `Agent` instance. @@ -304,7 +304,7 @@ See [Integrate with Flink]({{< ref "docs/development/integrate_with_flink" >}}) #### Prepare Flink and Flink Agents -Follow the [installation]({{< ref "docs/get-started/installation" >}}) instructions to set up Flink and Flink Agents. +Follow the [Setup]({{< ref "docs/get-started/setup" >}}) instructions to set up Flink and Flink Agents. #### Clone the Flink Agents Repository (if not done already) @@ -331,7 +331,7 @@ $FLINK_HOME/bin/start-cluster.sh {{< /tab >}} {{< tab "Java" >}} -1. Build Flink Agents from source to generate example jar. See [installation]({{< ref "docs/get-started/installation" >}}) for more details. +1. Build Flink Agents from source to generate example jar. See [Setup]({{< ref "docs/get-started/setup" >}}) for more details. 2. Start the Flink cluster ```bash $FLINK_HOME/bin/start-cluster.sh diff --git a/docs/content/docs/get-started/installation.md b/docs/content/docs/get-started/setup.md similarity index 99% rename from docs/content/docs/get-started/installation.md rename to docs/content/docs/get-started/setup.md index 9e0a9d7a5e..8e10e69dc8 100644 --- a/docs/content/docs/get-started/installation.md +++ b/docs/content/docs/get-started/setup.md @@ -1,7 +1,9 @@ --- -title: 'Installation' +title: "Setup" weight: 2 type: docs +aliases: + - /docs/get-started/installation/ ---