[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IDE Gradle sync being slow because of lockfile generation #147837

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

bartekpacia
Copy link
Member
@bartekpacia bartekpacia commented May 4, 2024

I lost way too much time waiting for Gradle syncs to finish because of this problem, so I decided to take a stab at it.

This PR attempts to fix #110559

Revival of #112723

Pre-launch Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] wiki page, which explains my responsibilities.
  • I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
  • I signed the [CLA].
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is [test-exempt].
  • I followed the [breaking change policy] and added [Data Driven Fixes] where supported.
  • All existing and new tests are passing.

@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact "@test-exemption-reviewer" in the #hackers channel in Chat (don't just cc them here, they won't see it! Use Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@bartekpacia bartekpacia marked this pull request as draft May 4, 2024 20:51
@github-actions github-actions bot added the tool Affects the "flutter" command-line tool. See also t: labels. label May 4, 2024
@bartekpacia
Copy link
Member Author

Some notes/questions:

  • Are the lockfiles actually generated? If so, where? I ran ./gradlew :generateLockfiles --write-locks, but no .lockfile files were generated, neither in my app nor in Android plugins under ~/.pub-cache.
  • Why does "Sync project with Gradle files" in IntelliJ trigger lockfile generation on every sync, but when I build my app normally (./gradlew :app:assembleDebug), then lockfile generation isn't triggered?
  • How can I run the equivalent of "Sync project with Gradle files" from the terminal? I can image it might be useful for testing this.

cc gradle gurus @GaryQian @gmackall @reidbaker

@reidbaker
Copy link
Contributor

Some notes/questions:

  • Are the lockfiles actually generated? If so, where? I ran ./gradlew :generateLockfiles --write-locks, but no .lockfile files were generated, neither in my app nor in Android plugins under ~/.pub-cache.
  • Why does "Sync project with Gradle files" in IntelliJ trigger lockfile generation on every sync, but when I build my app normally (./gradlew :app:assembleDebug), then lockfile generation isn't triggered?
  • How can I run the equivalent of "Sync project with Gradle files" from the terminal? I can image it might be useful for testing this.

cc gradle gurus @GaryQian @gmackall @reidbaker

I don't know the answer to most of these off the top of my head. IIRC flutter 2.* used a beta gradle feature to generateLockfiles which has since been made stable but the activation might have a different name than the preview feature.

@jwren could you point me towards the code that run in Intellij when we sync gradle files? If so then I can dig into the differences between the IDE behavior and the command line behavior and give a command line equivalent.

@gmackall
Copy link
Member

Is this ready for review? Happy to take a look if so, but just want to be sure because the PR is still in draft

@reidbaker
Copy link
Contributor

Added you for the discussion on #147837 (comment)

@jwren
Copy link
Member
jwren commented Jun 10, 2024

@reidbaker we have a dart cli which calls gradle on our behalf, cli tool is here: https://github.com/flutter/flutter-intellij/tree/master/tool/plugin

@reidbaker
Copy link
Contributor

Thank you @jwren! @bartekpacia let me dig in this week and give you better answers.

@bartekpacia
Copy link
Member Author

Thanks a ton, this looks like a good starting point.

@reidbaker thanks! If you're busy though, don't bother really, I'll do it myself and if I fail, then I'll ping you back:)

@zsolt-marta-bitrise
Copy link
zsolt-marta-bitrise commented Jun 26, 2024

We've found that this actually gets into an infinite loop if we have a plugin added in an init script in the gradle home directory, the Flutter plugin gets applied to the :dependencies call made by :generateLockfiles task init code, which in turns calls :dependencies again.

@zsolt-marta-bitrise
Copy link

I can confirm that your fix fixes the infinite loop problem too.

Additional interesting find is that the task :dependencies is not run normally, only when additional plugins are loaded by our init script in the home directory, so

Are the lockfiles actually generated?

I'd say no?

Why does "Sync project with Gradle files" in IntelliJ trigger lockfile generation on every sync, but when I build my app normally (./gradlew :app:assembleDebug), then lockfile generation isn't triggered?

Yes, exactly what we're seeing

@zsolt-marta-bitrise
Copy link

After further digging, I've found that our plugin evaluates the project's tasks, which triggers the :generateLockfiles initialization code that runs Gradle, and that starts an infinite loop because our plugin is applied again. That concerns this PR because evaluating a project's task is what triggers the :generateLockfiles init code that triggers gradle :dependencies, adding to the build time.

How can I run the equivalent of "Sync project with Gradle files" from the terminal? I can image it might be useful for testing this.

Just run gradlew tasks in a flutter test app project's android/ folder. It will evaluate all tasks, and also runs :dependencies because of the task init code.

@bartekpacia
Copy link
Member Author

I just talked with @runningcode, a Gradle engineer, on droidcon Berlin and asked him how we can trigger Gradle Sync without an IDE (so we can actually test that our fix will work). He suggested using https://github.com/gradle/gradle-profiler. It has a lot of various config options, some of the ones we might wanna take a look at are apply-build-script-change-to and clear-android-studio-cache-before.

Thanks a lot @runningcode! :)

@Piinks
Copy link
Contributor
Piinks commented Sep 24, 2024

(PR Triage): Hey @bartekpacia do you have plans to continue with this PR? Thanks!

@bartekpacia
Copy link
Member Author

Hey @Piinks! Yes I do!

@bartekpacia
Copy link
Member Author

Some notes

Will likely have to create a scenario file.

gradle-profiler --benchmark --scenario-file performance.scenarios clean_build

See also

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tool Affects the "flutter" command-line tool. See also t: labels.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Android Gradle Sync is time-consuming
6 participants