Victor Dibia

[Draft] Enable File Upload/Paste as Task in AGS (#6091) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? https://github.com/user-attachments/assets/e160f16d-f42d-49e2-a6c6-687e4e6786f4 Enable file upload/paste as a task in AGS. Enables tasks like - Can you research and fact check the ideas in this screenshot? - Summarize this file Only text and images supported for now Underneath, it constructs TextMessage and Multimodal messages as the task. <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5773 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed. --------- Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>

Enable Auth in AGS (#5928) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? https://github.com/user-attachments/assets/b649053b-c377-40c7-aa51-ee64af766fc2 <img width="100%" alt="image" src="https://github.com/user-attachments/assets/03ba1df5-c9a2-4734-b6a2-0eb97ec0b0e0" /> ## Authentication This PR implements an experimental authentication feature to enable personalized experiences (multiple users). Currently, only GitHub authentication is supported. You can extend the base authentication class to add support for other authentication methods. By default authenticatio is disabled and only enabled when you pass in the `--auth-config` argument when running the application. ### Enable GitHub Authentication To enable GitHub authentication, create a `auth.yaml` file in your app directory: ```yaml type: github jwt_secret: "your-secret-key" token_expiry_minutes: 60 github: client_id: "your-github-client-id" client_secret: "your-github-client-secret" callback_url: "http://localhost:8081/api/auth/callback" scopes: ["user:email"] ``` Please see the documentation on [GitHub OAuth](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authenticating-to-the-rest-api-with-an-oauth-app) for more details on obtaining the `client_id` and `client_secret`. To pass in this configuration you can use the `--auth-config` argument when running the application: ```bash autogenstudio ui --auth-config /path/to/auth.yaml ``` Or set the environment variable: ```bash export AUTOGENSTUDIO_AUTH_CONFIG="/path/to/auth.yaml" ``` ```{note} - Authentication is currently experimental and may change in future releases - User data is stored in your configured database - When enabled, all API endpoints require authentication except for the authentication endpoints - WebSocket connections require the token to be passed as a query parameter (`?token=your-jwt-token`) ``` ## Related issue number <!-- For example: "Closes #1234" --> Closes #4350 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed. --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.

add anthropic native support (#5695) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> Claude 3.7 just came out. Its a pretty capable model and it would be great to support it in Autogen. This will could augment the already excellent support we have for Anthropic via the SKAdapters in the following ways - Based on the ChatCompletion API similar to the ollama and openai client - Configurable/serializable (can be dumped) .. this means it can be used easily in AGS. ## What is Supported (video below shows the client being used in autogen studio) https://github.com/user-attachments/assets/8fb7c17c-9f9c-4525-aa9c-f256aad0f40b - streaming - tool callign / function calling - drop in integration with assistant agent. - multimodal support ```python from dotenv import load_dotenv import os load_dotenv() from autogen_agentchat.agents import AssistantAgent from autogen_agentchat.ui import Console from autogen_ext.models.openai import OpenAIChatCompletionClient from autogen_ext.models.anthropic import AnthropicChatCompletionClient model_client = AnthropicChatCompletionClient( model="claude-3-7-sonnet-20250219" ) async def get_weather(city: str) -> str: """Get the weather for a given city.""" return f"The weather in {city} is 73 degrees and Sunny." agent = AssistantAgent( name="weather_agent", model_client=model_client, tools=[get_weather], system_message="You are a helpful assistant.", # model_client_stream=True, ) # Run the agent and stream the messages to the console. async def main() -> None: await Console(agent.run_stream(task="What is the weather in New York?")) await main() ``` result ``` messages = [ UserMessage(content="Write a very short story about a dragon.", source="user"), ] # Create a stream. stream = model_client.create_stream(messages=messages) # Iterate over the stream and print the responses. print("Streamed responses:") async for response in stream: # type: ignore if isinstance(response, str): # A partial response is a string. print(response, flush=True, end="") else: # The last response is a CreateResult object with the complete message. print("\n\n------------\n") print("The complete response:", flush=True) print(response.content, flush=True) print("\n\n------------\n") print("The token usage was:", flush=True) print(response.usage, flush=True) ``` <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5205 Closes #5708 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed. cc @rohanthacker

AGS - Test Model Component in UI, Compare Sessions (#5963) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? - Adds ability to test model clients in UI after configrations, before they are used in agents or teams - Adds UI side by side comparison of sessions <img width="1878" alt="image" src="https://github.com/user-attachments/assets/f792d8d6-3f5d-4d8c-a365-5a9e9c00f49e" /> <img width="1877" alt="image" src="https://github.com/user-attachments/assets/5a115a5a-95e1-4956-a733-5f0065711fe3" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #4273 Closes #5728 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.

Fix warnings in AGS (#5320) This PR does the following: - Fix warning messages in AGS on launch. - Improve Cli message to include app URL on startup from command line - Minor improvements default gallery generator. (add more default tools) - Improve new session behaviour. ## Related issue number Closes #5097 ## Checks

Improve Gallery Editor UX in AGS (#5613) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? Fixes the problem where Gallery items could only be modified via JSON This PR does the following - Refactor TeamBuilder to have modular component editor UI primarily focused on editing each component type. - Refactor the Gallery UX - improve layout to use tabs for each component type - enable editing of each component item by reusing the component editor - Enable switching between form editing and UI editing for coponent editor view This way, gallery items can be readily modified and then reused in the component library in team builder. It also implements an upate to the Gallery data structure to make it more intuitive - it has a components field that has teams, agents, models ... <img width="1598" alt="image" src="https://github.com/user-attachments/assets/3c3a228a-0bd2-4fc1-85ec-c9685c80bf72" /> <img width="1614" alt="image" src="https://github.com/user-attachments/assets/5b6ed840-9c48-47bc-8c17-2aa50c7dcb99" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5465 Closes #5047 cc @nour-bouzid @balakreshnan @EItanya @joslat @IustinT @leonG7 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.

feat: [draft] update version of azureaiagent (#6581) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> There have been updates to the azure ai agent foundry sdk (azure-ai-project). This PR updates the autogen `AzureAIAgent` which wraps the azure ai agent. A list of some changes - Update docstring samples to use `endpoint` (instead of connection string previously) - Update imports and arguments e.g, from `azure.ai.agents` etc - Add a guide in ext docs showing Bing Search Grounding tool example. <img width="1423" alt="image" src="https://github.com/user-attachments/assets/0b7c8fa6-8aa5-4c20-831b-b525ac8243b7" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #6601 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.

Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.

Add support for default model client, in AGS updates to settings UI (#5763) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? Add support for default model client, in AGS updates to settings UI. A default model client will be used for subsequent background tasks (e.g, automatically naming sessions to meaningful names). <img width="1441" alt="image" src="https://github.com/user-attachments/assets/31675ef4-8f80-4a8c-9762-3d6bcbc6d33d" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5685 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.