Re: #3872 #3706 #3847, @jakemac53 what do you think, please? :)

Proposal

I propose that we create a new repository dart-lang/macros, and use that for

1. macros packages that don't need SDK internals, such as dart_model, and related tooling;
2. issues and docs related to the implementation rather than the language feature,
3. (maybe?) end to end testing and benchmarks that don't need SDK internals.

I propose to use dart_model and other macros packages from SDK tools via a version in DEPS pulling into third_party/pkg.

Discussion

This idea came out of thinking about where dart_model code should live, then noticing we could create a repo for it, then noticing that we are currently overwhelming the language repo with macro things when much of the noise is now about the implementation, not the spec.

Re: where dart_model should live, the package should be a "lowest common denominator" in the sense that we want everyone to depend on it, including internal SDK tools:

flowchart LR
Analyzer --> dart_model
CFE --> dart_model
package:some_user_macro --> dart_model

The analyzer and CFE want to depend only on code in the SDK, which suggests putting dart_model under pkg in the SDK repo. But, this has some pretty big downsides:

• dart_model wants to come with a bunch of related tooling, helper packages (codegen/schema validation), possibly utility packages (json_buffer) and probably additional macro-related packages (macro protocol) , ... squashing these into the flat pkg namespace is awkward.
• it doesn't look quite correct; if we develop dart_model under pkg then the analyzer and CFE always use it from main. That makes it look like the analyzer and CFE always have the latest dart_model; but that's simply not true: we care about cases where the user has an older SDK.

I suspect that keeping them physically decoupled so that the setup matches the design could work better. For example, it's perfectly valid to work on dart_model ahead of what the SDK supports, when adding support for unlaunched features; and it's valid to work on dart_model features that are only used by the client, with no relevance at all for the SDK. There is no obvious advantage to coupling these changes to the SDK test and release, and decoupling may have the advantage of encouraging a cleanly decoupled package and tests.

So, let's keep that physical separation by using dart_model like an external dep.

This layer of separation also opens possibilities around doing source transformations useful to the host, for example if we discover that it's convenient for the analyzer to have access to multiple versions of the package we can copy in multiple versions, renaming the old one(s), so they are available simultaneously.

Edited to refer to the DEPS mechanism for pulling in packages by version.