jif-oai

feat: spreadsheet artifact (#13345)

feat: presentation artifact p1 (#13341) Part 1 of presentation tool artifact

feat: exec-server prep for unified exec (#15691) This PR partially rebase `unified_exec` on the `exec-server` and adapt the `exec-server` accordingly. ## What changed in `exec-server` 1. Replaced the old "broadcast-driven; process-global" event model with process-scoped session events. The goal is to be able to have dedicated handler for each process. 2. Add to protocol contract to support explicit lifecycle status and stream ordering: - `WriteResponse` now returns `WriteStatus` (Accepted, UnknownProcess, StdinClosed, Starting) instead of a bool. - Added seq fields to output/exited notifications. - Added terminal process/closed notification. 3. Demultiplexed remote notifications into per-process channels. Same as for the event sys 4. Local and remote backends now both implement ExecBackend. 5. Local backend wraps internal process ID/operations into per-process ExecProcess objects. 6. Remote backend registers a session channel before launch and unregisters on failed launch. ## What changed in `unified_exec` 1. Added unified process-state model and backend-neutral process wrapper. This will probably disappear in the future, but it makes it easier to keep the work flowing on both side. - `UnifiedExecProcess` now handles both local PTY sessions and remote exec-server processes through a shared `ProcessHandle`. - Added `ProcessState` to track has_exited, exit_code, and terminal failure message consistently across backends. 2. Routed write and lifecycle handling through process-level methods. ## Some rationals 1. The change centralizes execution transport in exec-server while preserving policy and orchestration ownership in core, avoiding duplicated launch approval logic. This comes from internal discussion. 2. Session-scoped events remove coupling/cross-talk between processes and make stream ordering and terminal state explicit (seq, closed, failed). 3. The failure-path surfacing (remote launch failures, write failures, transport disconnects) makes command tool output and cleanup behavior deterministic ## Follow-ups: * Unify the concept of thread ID behind an obfuscated struct * FD handling * Full zsh-fork compatibility * Full network sandboxing compatibility * Handle ws disconnection

feat: add layered --profile-v2 config files (#17141) ## Why `--profile-v2 <name>` gives launchers and runtime entry points a named profile config without making each profile duplicate the base user config. The base `$CODEX_HOME/config.toml` still loads first, then `$CODEX_HOME/<name>.config.toml` layers above it and becomes the active writable user config for that session. That keeps shared defaults, plugin/MCP setup, and managed/user constraints in one place while letting a named profile override only the pieces that need to differ. ## What Changed - Added the shared `--profile-v2 <name>` runtime option with validated plain names, now represented by `ProfileV2Name`. - Extended config layer state so the base user config and selected profile config are both `User` layers; APIs expose the active user layer and merged effective user config. - Threaded profile selection through runtime entry points: `codex`, `codex exec`, `codex review`, `codex resume`, `codex fork`, and `codex debug prompt-input`. - Made user-facing config writes go to the selected profile file when active, including TUI/settings persistence, app-server config writes, and MCP/app tool approval persistence. - Made plugin, marketplace, MCP, hooks, and config reload paths read from the merged user config so base and profile layers both participate. - Updated app-server config layer schemas to mark profile-backed user layers. ## Limits `--profile-v2` is still rejected for config-management subcommands such as feature, MCP, and marketplace edits. Those paths remain tied to the base `config.toml` until they have explicit profile-selection semantics. Some adjacent background writes may still update base or global state rather than the selected profile: - marketplace auto-upgrade metadata - automatic MCP dependency installs from skills - remote plugin sync or uninstall config edits - personality migration marker/default writes ## Verification Added targeted coverage for profile name validation, layer ordering/merging, selected-profile writes, app-server config writes, session hot reload, plugin config merging, hooks/config fixture updates, and MCP/app approval persistence. --------- Co-authored-by: Codex <noreply@openai.com>

feat: change multi-agent to use path-like system instead of uuids (#15313) This PR add an URI-based system to reference agents within a tree. This comes from a sync between research and engineering. The main agent (the one manually spawned by a user) is always called `/root`. Any sub-agent spawned by it will be `/root/agent_1` for example where `agent_1` is chosen by the model. Any agent can contact any agents using the path. Paths can be used either in absolute or relative to the calling agents Resume is not supported for now on this new path

feat: artifact presentation part 7 (#13360)

feat: move exec-server ownership (#16344) This introduces session-scoped ownership for exec-server so ws disconnects no longer immediately kill running remote exec processes, and it prepares the protocol for reconnect-based resume. - add session_id / resume_session_id to the exec-server initialize handshake - move process ownership under a shared session registry - detach sessions on websocket disconnect and expire them after a TTL instead of killing processes immediately (we will resume based on this) - allow a new connection to resume an existing session and take over notifications/ownership - I use UUID to make them not predictable as we don't have auth for now - make detached-session expiry authoritative at resume time so teardown wins at the TTL boundary - reject long-poll process/read calls that get resumed out from under an older attachment --------- Co-authored-by: Codex <noreply@openai.com>

Move memory state to a dedicated SQLite DB (#24591) ## Summary Generated memory rows and their stage-one/stage-two job state currently live in `state_5.sqlite` alongside thread metadata. That makes memory cleanup and regeneration share the main state schema even though those rows are memory-pipeline data and can be rebuilt independently from the durable thread records. This PR moves the memory-owned tables into a dedicated `memories_1.sqlite` runtime database while keeping thread metadata in `state_5.sqlite`. ## Changes - Adds a separate memories DB runtime, migrator, path helpers, telemetry kind, and Bazel compile data for `state/memory_migrations`. - Introduces `MemoryStore` behind `StateRuntime::memories()` and moves memory table/job operations onto that store. - Drops the old memory tables from the state DB and recreates their schema in `state/memory_migrations/0001_memories.sql`. - Updates memory startup, citation usage tracking, rollout pollution handling, `debug clear-memories`, and app-server `memory/reset` to operate through the memories DB. - Preserves cross-DB behavior by hydrating thread metadata from the state DB when selecting visible memory outputs and checking stage-one staleness. ## Verification - Added/updated `codex-state` tests for deleted-thread memory visibility and already-polluted phase-two enqueue behavior. - Updated `debug clear-memories`, app-server `memory/reset`, and memories startup tests to seed and assert memory rows through `memories_1.sqlite`.

feat: spreadsheet part 3 (#13350) =

chore: ultra-clean artifacts (#13577) See the readme

feat: ultra polish package manager (#13573) See the readme

Make turn diff tracking operation backed (#21180) ## Summary - replace filesystem-based turn diff tracking with an operation-backed accumulator - preserve enough verified apply_patch state to render move-overwrite cases correctly - keep the turn/diff/updated contract intact while removing remote-only turn-diff test skips This takes the assumption that no 3P services rely on the output format of `apply_patch` ## Why For the CCA file system isolation push --------- Co-authored-by: Codex <noreply@openai.com>

feat: use git-backed workspace diffs for memory consolidation (#18982) ## Why This PR make the `morpheus` agent (memory phase 2) use a git diff to start it's consolidation. The workflow is the following: 1. The agent acquire a lock 2. If `.codex/memories` does not exist or is not a git root, initialize everything (and make a first empty commit) 3. Update `raw_memories.md` and `rollout_summaries/` as before. Basically we select max N phase 1 memories based on a given policy 4. We use git (`gix`) to get a diff between the current state of `.codex/memories` and the last commit. 5. Dump the diff in `phase2_workspace_diff.md` 6. Spawn `morpheus` and point it to `phase2_workspace_diff.md` 7. Wait for `morpheus` to be done 8. Re-create a new `.git` and make one single commit on it. We do this because we don't want to preserve history through `.git` and this is cheap anyway 9. We release the lock On top of this, we keep the retry policies etc etc The goals of this new workflow are: * Better support of any memory extensions such as `chronicle` * Allow the user to manually edit memories and this will be considered by the phase 2 agent As a follow-up we will need to add support for user's edition while `morpheus` is running ## What Changed - Added memory workspace helpers that prepare the git baseline, compute the diff, write `phase2_workspace_diff.md`, and reset the baseline after successful consolidation. - Updated Phase 2 to sync current inputs into `raw_memories.md` and `rollout_summaries/`, prune old extension resources, skip clean workspaces, and run the consolidation subagent only when the workspace has changes. - Tightened Phase 2 job ownership around long-running consolidation with heartbeats and an ownership check before resetting the baseline. - Simplified the prompt and state APIs so DB watermarks are bookkeeping, while workspace dirtiness decides whether consolidation work exists. - Updated the memory pipeline README and tests for workspace diffs, extension-resource cleanup, pollution-driven forgetting, selection ranking, and baseline persistence. ## Verification - Added/updated coverage in `core/src/memories/tests.rs`, `core/src/memories/workspace_tests.rs`, `state/src/runtime/memories.rs`, and `core/tests/suite/memories.rs`. --------- Co-authored-by: Codex <noreply@openai.com>

fix: FS watcher when file does not exist yet (#18492) The initial goal of this PR was to stabilise the test `fs_watch_allows_missing_file_targets`. After further investigation, it turns out that this test was always failing and the unstability was coming from a race between timeouts mostly The goal of the test was to test what happens if a notifier gets subscribed while a file does not exist yet. But actually the main code was broken and in case of a file not existing yet, the notifier used to never notify anything (even if the file ended up being created) This PR fixes the main code (and the test). For this, we basically watch the sup-directory when a file does not exist and refresh on it when the files gets created

Simplify tool executor and registry plumbing (#22636) ## Why The tool runtime path still had a typed output associated type on `ToolExecutor`, plus a core-only `RegisteredTool` adapter and extension-only executor aliases. That made every new shared tool runtime carry extra adapter plumbing before it could participate in core dispatch, extension tools, hook payloads, telemetry, and model-visible spec generation. This PR moves output erasure to the shared executor boundary so core and extension tools can use the same execution contract directly. ## What Changed - Changed `codex_tools::ToolExecutor` to return `Box<dyn ToolOutput>` instead of an associated `Output` type. - Removed the extension-specific `ExtensionToolExecutor` / `ExtensionToolOutput` aliases and exposed `ToolExecutor<ToolCall>` plus `ToolOutput` through `codex-extension-api`. - Reworked core tool registration around `CoreToolRuntime` and `ToolRegistry::from_tools`, removing the extra `RegisteredTool` / `ToolRegistryBuilder` layer. - Consolidated model-visible spec planning and registry construction in `core/src/tools/spec_plan.rs`, including deferred tool search and code-mode-only filtering. - Added `ToolOutput` helpers for post-tool-use hook ids and inputs so MCP, unified exec, extension, and other boxed outputs preserve the same hook payload behavior. - Updated core handlers, memories tools, and the related registry/spec/router tests to use the simplified contract. ## Test Coverage - Updated coverage for tool spec planning, registry lookup, deferred tool search registration, extension tool routing, post-tool-use hook payloads, dispatch tracing, guardian output extraction, and memories extension tool execution.

feat: split memories part 2 (#19860) Keep extracting memories out of core and moving the write trigger in the app-server This is temporary and it should move at the client level as a follow-up This makes core fully independant from `codex-memories-write` --------- Co-authored-by: Codex <noreply@openai.com>

Add process-scoped SQLite telemetry (#22154) ## Summary - add SQLite init, backfill-gate, and fallback telemetry without introducing a cross-cutting state-db access wrapper - install one process-scoped telemetry sink after OTEL startup and let low-level state/rollout paths emit through it directly - add process-start metrics for the process owners that initialize SQLite --------- Co-authored-by: Owen Lin <owen@openai.com>

feat: wire extension tool bundles into core (#22147) ## Why This is the next narrow step toward moving concrete tool families out of core. After #22138 introduced `codex-tool-api`, we still needed a real end-to-end seam that lets an extension own an executable tool definition once and have core install it without the temporary `extension-api` wrapper or a dependency on `codex-tools`. `codex-tool-api` is the small extension-facing execution contract, while `codex-tools` still has a different job: host-side shared tool metadata and planning logic that is not “run this contributed tool”, like spec shaping, namespaces, discovery, code-mode augmentation, and MCP/dynamic-to-Responses API conversion ## What changed - Moved the shared leaf tool-spec and JSON Schema types into `codex-tool-api`, so the executable contract now lives with [`ToolBundle`](https://github.com/openai/codex/blob/c538758095337d4fe0a52a172363ccede4066bda/codex-rs/tool-api/src/bundle.rs#L19-L70). - Replaced the temporary extension-side tool wrapper with direct `ToolBundle` use in `codex-extension-api`. - Taught core to collect contributed bundles, include them in spec planning, register them through [`ToolRegistryBuilder::register_tool_bundle`](https://github.com/openai/codex/blob/c538758095337d4fe0a52a172363ccede4066bda/codex-rs/core/src/tools/registry.rs#L653-L667), and dispatch them through the existing router/runtime path. - Added focused coverage for contributed tools becoming model-visible and dispatchable, plus spec-planning coverage for contributed function and freeform tools. ## Verification - Added `extension_tool_bundles_are_model_visible_and_dispatchable` in `core/src/tools/router_tests.rs`. - Added spec-plan coverage in `core/src/tools/spec_plan_tests.rs` for contributed extension bundles. ## Related - Follow-up to #22138

Run compact hooks for remote compaction v2 (#22828) ## Why Remote compaction v2 is the `/responses` implementation of session-history compaction, but it still needs to preserve the observable contract of the legacy `/responses/compact` path. In particular, users and integrations that rely on `PreCompact` and `PostCompact` hooks should not see different behavior when `remote_compaction_v2` is enabled. ## What Changed - Runs `PreCompact` before issuing the remote compaction v2 request, including `Interrupted` analytics when a pre-hook stops execution. - Runs `PostCompact` after a successful v2 compaction and aborts the turn if the post-hook stops execution. - Adds `compact_remote_parity` coverage that compares legacy and v2 compaction across manual transcript shapes, automatic pre-turn compaction, automatic mid-turn compaction, hook payloads, replacement history, follow-up request payloads, and API-key `service_tier=fast` behavior. - Registers the new parity suite under `core/tests/suite`. Relevant code: - [`compact_remote_v2.rs`](https://github.com/openai/codex/blob/af63745cb502183a6fc447d0240f8150934d70b7/codex-rs/core/src/compact_remote_v2.rs) - [`compact_remote_parity.rs`](https://github.com/openai/codex/blob/af63745cb502183a6fc447d0240f8150934d70b7/codex-rs/core/tests/suite/compact_remote_parity.rs) ## Verification - Added `core/tests/suite/compact_remote_parity.rs` to assert parity between legacy remote compaction and remote compaction v2 for the affected request, hook, rollout-history, and follow-up paths. - Existing `compact_remote_v2` unit coverage still exercises v2 replacement-history retention and compaction-output collection.

feat: add graph representation of agent network (#15056) Add a representation of the agent graph. This is now used for: * Cascade close agents (when I close a parent, it close the kids) * Cascade resume (oposite) Later, this will also be used for post-compaction stuffing of the context Direct fix for: https://github.com/openai/codex/issues/14458