Chris Olszewski

feat(turbo_json): add `$TURBO_EXTENDS$` (#10763) ### Description This PR adds a new DSL keyword for array task definitions that alters the behavior when using `extends` in a package `turbo.json`. Usually a field from a package task definition will completely override the root field e.g. Root: `{ "env": ["FOO"] }` Package: `{ "env": ["BAR"] }` Result: `{"env": ["BAR"] }` If the package uses the new `$TURBO_EXTENDS$` the root field value will be extended from instead of overridden. e.g. Root: `{ "env": ["FOO"] }` Package: `{ "env": ["$TURBO_EXTENDS$", "BAR"] }` Result: `{"env": ["FOO", "BAR"] }` This functionality allows for greater flexibility when specializing package task definitions without needing to copy/paste the shared values and remembering to update them if the root changed. It is currently gated by `turboExtends` feature flag in the root `turbo.json`. Reviewing this each commit individually in this PR is *highly* suggested. The actual feature add is all done in the final commit, the rest are prefactors. The primary prefactors are: - Plumbing through future flags to be read at config time, but usable at task resolution time. This is necessary since this is set at root and needs to be known when reading task definitions from package `turbo.json`s - Adding a new intermediate step for task definition creation named "processed". Removes a majority of the DSL concepts and validations that are exclusive to a single field. This paves the way for future extensions to the DSL. Nerd note: Technically, we could extend `$TURBO_EXTENDS$` to additional fields that are monoids. Extension behavior is really: ``` let a = if extends { base } else { mempty } a mappend b ``` The only interesting addition we could make would be around boolean fields which has 2 different monoids: any and all. Which one to choose would depend on the default value of the flag. e.g. `cache` defaults to false so we would want an any (or) behavior. I am not suggesting we do this, it is more confusing than it is worthwhile. ### Testing Instructions Added a fair amount of unit tests for the whole task definition resolution steps.

feat(turbo_json): allow for extending form non-root `turbo.json` (#10812) ### Description TL;DR Allow for `"extends": ["//", "@acem/my-pkg"]` to allow for shared task configuration outside of the root. This PR adds non-root extensions of `turbo.json` files. We still require all `turbo.json` files to extend from root first, but now you can extend from an additional package `turbo.json` files. The task definitions are collapsed left to right with the right fields taking precedence. (This can be more powerful when using `$TURBO_EXTENDS$` added in https://github.com/vercel/turborepo/pull/10763) We do have to check for cycles in `turbo.json` extends statements to ensure we can resolve the `extends` chain. Prefactor steps taken: - Splitting of the `RawTurboJson` schema into a root schema and a package schema. This removes the need for our ad-hoc checks of all fields in a package `turbo.json` that are invalid. (We were missing quite a few, for example you could put `globalEnv` in a package `turbo.json` and it would be silently ignored) - Changed ordering of task definition chain construction to first load all necessary `turbo.json` before fetching task definitions - Created a `Validator` struct which is used to share context which affect validation outcomes. This is necessary to feature flag this behavior I highly suggest reviewing each commit by itself as there is a lot of prefactor work that muddies the feature addition. ### Testing Instructions

fix(tui): no longer error if preferences cannot be read (#10004) ### Description Fixes #9999 The TUI should function even if we fail at loading preferences. ### Testing Instructions Add test to verify that if the preferences file contains invalid JSON a loader is still created. User probably was hitting a perms issue or some other FS issue and not an invalid JSON, but the test is just meant to show we don't crash.

chore(turbo_json): remove exterior mutability from loader (#10066) ### Description `TurboJsonLoader::load` only took a mutable self reference because we were using `HashMap` to cache the results. This PR changes the data type we use to cache results and removes the exterior mutability. We move to use a `FixedMap` which has the following properties: - Only works with the keys provided at construction - Append only - Thread safe I went with this just because the implementation is straightforward and matches our use case since we know all of the possible places a `turbo.json` will be before we start loading them. Future work can be reworking boundaries code to no longer eagerly load all of the `turbo.json`s at the start of execution. ### Testing Instructions Existing test suite. Additional unit tests for `FixedMap` --------- Co-authored-by: Nicholas Yang <nicholas.yang@vercel.com>

feat(login): add `--manual` flag (#9998) ### Description Currently setting up a custom remote cache requires editing multiple files and setting the `TURBO_TOKEN` env var for every command you run that requires the remote cache. Introducing `turbo login --manual` which will edit all configuration files required to allow for `turbo` to use the remote cache without setting any env vars or editing any files. This command is equivalent to `turbo login` and `turbo link` when using the Vercel remote cache. This PR also fixes a bug in our `rewrite_json` module where we didn't correctly rewrite paths if they had more than one element. I suggest reviewing this PR commit by commit. ### Testing Instructions Added unit tests, but going through the flow is the best way to test this. https://github.com/user-attachments/assets/d9215605-08c8-4d62-88a9-1a87b0f94833 --------- Co-authored-by: Anthony Shew <anthony.shew@vercel.com>

feat(bun): support `bun.lock` (#9783) ### Description Closes #9628 Switch over from our usage of `yarn.lock` and directly support `bun.lock` parsing. We use `biome` to strip out the trailing commas that appear in the `workspaces` and `packages` objects to leverage `serde` which I am more competent in. We can switch to `biome` in the future and avoid a second pass, but I do not know how to do correct deserialization logic for `PackageEntry` (see `de.rs` for examples of what this looks like) in `biome`. I will call out is the fact that the lockfile keys in `bun.lock` aren't used directly unlike other lockfile implementation. Instead we use resolved package identifiers e.g. `package@(protocol:)?version`. The keys themselves can shift when the underlying package does not change resulting in incorrect behavior. A quick example to illustrate: - `a` depends on `shared@1.0.0` - `b` depends on `shared@2.0.0` - `shared@1.0.0` will get they key of `shared` since `a` is before `b` - `shared@2.0.0` will get a key of `b/shared` - If `a` updates to `shared@2.0.0`, then the `shared` key will now point to `shared@2.0.0` - If we used the key instead of the ident we would rebuild `b` since it's key changed from `b/shared` to `shared` and not rebuild `a` since it's key remained `shared` This PR does not add support for `turbo prune`. Reviewing each commit on it's own would probably be helpful. ### Testing Instructions Unit tests. Manual testing on a `create-turbo` repository.

fix(pnpm): read linkWorkspacePackages from pnpm-workspace.yaml (#10391) ### Description Fixes #10387 by adding partial support for the new configuration location added in https://github.com/pnpm/pnpm/pull/9121. There will be future work to add additional configuration options that lived in `.npmrc` and `package.json` to the YAML definition. I highly suggest reviewing the first 2 commits on their own as they are prefactors. They both get a slightly smaller to slimming down the `impl PackageManager` so we can move to an interface instead of a single struct with `match`s for every package manager x version. ### Testing Instructions Added new unit tests for this behavior, also manually verified behavior change in repro provided in #10387. Note that after this PR the topological `^lint` dependency is correctly applied so the application lints to not start until `@repo/lint` completes. Before ``` [0 olszewski@macbookpro] /tmp/turbo-respect-pnpm-configs $ turbo lint turbo 2.5.2 • Packages in scope: @repo/eslint-config, @repo/typescript-config, @repo/ui, docs, web • Running lint in 5 packages • Remote caching disabled ┌ web#lint > cache miss, executing b9dab19525365b14 │ │ │ > web@0.1.0 lint /private/tmp/turbo-respect-pnpm-configs/apps/web │ > next lint --max-warnings 0 └────> ┌ docs#lint > cache miss, executing 72e63e1338f32778 │ │ │ > docs@0.1.0 lint /private/tmp/turbo-respect-pnpm-configs/apps/docs │ > next lint --max-warnings 0 └────> ┌ @repo/ui#lint > cache miss, executing 1e7e4e418db8cf29 │ │ │ > @repo/ui@0.0.0 lint /private/tmp/turbo-respect-pnpm-configs/packages/ui │ > echo 'failing lint' && exit 1 │ │ failing lint │  ELIFECYCLE  Command failed with exit code 1. │ command finished with error: command (/private/tmp/turbo-respect-pnpm-configs/packages/ui) /User │ s/olszewski/.nvm/versions/node/v22.13.0/bin/pnpm run lint exited (1) └────> @repo/ui#lint: command (/private/tmp/turbo-respect-pnpm-configs/packages/ui) /Users/olszewski/.nvm/versions/node/v22.13.0/bin/pnpm run lint exited (1) Tasks: 0 successful, 3 total Cached: 0 cached, 3 total Time: 1.096s Failed: @repo/ui#lint ERROR run failed: command exited (1) ``` After ``` [1 olszewski@macbookpro] /tmp/turbo-respect-pnpm-configs $ turbo_dev --skip-infer lint turbo 2.5.2 • Packages in scope: @repo/eslint-config, @repo/typescript-config, @repo/ui, docs, web • Running lint in 5 packages • Remote caching disabled ┌ @repo/ui#lint > cache miss, executing 5e80cac629794173 │ │ │ > @repo/ui@0.0.0 lint /private/tmp/turbo-respect-pnpm-configs/packages/ui │ > echo 'failing lint' && exit 1 │ │ failing lint │  ELIFECYCLE  Command failed with exit code 1. │ command finished with error: command (/private/tmp/turbo-respect-pnpm-configs/packages/ui) /User │ s/olszewski/.nvm/versions/node/v22.13.0/bin/pnpm run lint exited (1) └────> @repo/ui#lint: command (/private/tmp/turbo-respect-pnpm-configs/packages/ui) /Users/olszewski/.nvm/versions/node/v22.13.0/bin/pnpm run lint exited (1) Tasks: 0 successful, 1 total Cached: 0 cached, 1 total Time: 363ms Failed: @repo/ui#lint ERROR run failed: command exited (1) ```

chore(turbo_json): prefactor loading logic (#10719) ### Description This PR is a prefactor to make the process of constructing a task definition clearer: - move `TaskId`/`TaskName` out of `turborepo-lib` so they can be available if `turbo.json` ever moves to a new crate - move configuration logic to configuration crate - documenting how `turbo.json`s on disk become task definitions. Highly suggest looking at each commit on it's own. ### Testing Instructions 👀

chore(release): remove goreleaser (#9918) ### Description Third try is the charm. See https://github.com/vercel/turborepo/pull/9905 and https://github.com/vercel/turborepo/pull/9909 for details. Additional changes: - Make sure `turbo` binaries are marked as executable. Since they get downloaded via `@actions/download-artifact` which strips permissions: [docs](https://github.com/actions/download-artifact?tab=readme-ov-file#permission-loss) - Correctly set `os: ["win32"]` for the windows package for `turbo-windows-${arch}` ### Testing Instructions Changes from previous PRs: added an e2e test to make sure that the tarball is installable and the included executable is just that.

fix(dry): do not perform runtime validations on dry runs (#10375) ### Description I was poking around and hit a case where I couldn't use dry run if it included interactive tasks or too many persistent tasks for the concurrency. These checks don't make sense for `--graph` or `--dry` since we're not actually executing the graph. ### Testing Instructions Added unit tests making sure that these checks are skipped. Quick manual test: ``` [0 olszewski@macbookpro] $ turbo run dev --dry=json > /dev/null turbo 2.5.1-canary.1 x Invalid task configuration |-> x You have 19 persistent tasks but `turbo` is configured for concurrency of 10. Set --concurrency to at least 20 `-> x Cannot run interactive task "vercel-ship#dev" without Terminal UI. Set `"ui": true` in `turbo.json`, use the `--ui=tui` flag, or set `TURBO_UI=true` as | an environment variable. [1 olszewski@macbookpro] $ turbo_dev --skip-infer dev --dry=json > /dev/null turbo 2.5.1 [0 olszewski@macbookpro] $ ```