[go: nahoru, domu]

Page MenuHomePhabricator

Deprecate buster-backports
Closed, ResolvedPublic

Description

The buster-backports repo is no longer available. This is preventing us from building any images that include his repo on apt-update failure

Get:1 http://security.debian.org buster/updates InRelease [34.8 kB]
Hit:2 http://apt.wikimedia.org/wikimedia buster-wikimedia InRelease
Hit:3 http://mirrors.wikimedia.org/debian buster InRelease
Hit:4 http://mirrors.wikimedia.org/debian buster-updates InRelease
Ign:5 http://mirrors.wikimedia.org/debian buster-backports InRelease
Err:6 http://mirrors.wikimedia.org/debian buster-backports Release
  404  Not Found [IP: 208.80.154.74 8080]
Get:7 http://security.debian.org buster/updates/main amd64 Packages [595 kB]
Reading package lists...
E: The repository 'http://mirrors.wikimedia.org/debian buster-backports Release' no longer has a Release file.The command '/bin/sh -c apt-get update && apt-get install -y php7.4-tideways php7.4-ldap' returned a non-zero code: 100
make[1]: *** [Makefile:17: build-debug-image] Error 100
make[1]: Leaving directory '/srv/mwbuilder/release/make-container-image'
make: *** [Makefile:35: build-and-push-all-images] Error 2
make: Leaving directory '/srv/mwbuilder/release/make-container-image'
09:01:05 Finished build-and-push-container-images (duration: 00m 30s)
09:01:05 Build of K8s images failed (non-K8s deployment will continue normally)

According to CodeSearch, affected images are:

  • shellbox (lilypond and lilypond-data)
  • integration/config/operations-puppet (shellcheck)
  • integration/config/tox-buster (shellcheck)
  • integration/config/tox-pywikibot (shellcheck)
  • mediawiki/vagrant (php7.4-wikidiff2)

The base buster image also needs to have it removed, as it blocks all deployment of MW-on-K8s on apt-update failure

  • create component/lilypond and import the last version from buster-backports
  • Rebuild base-images
  • Rebuild buster-based production-images
  • Do a full rebuild deployment of mw-on-k8s scap sync-world --k8s-only -D full_image_build:true
  • Restart docker-reporter-base-images and docker-reporter-k8s-images
  • Start docker-system-prune-all.service when the above two are done to clean up build2001

Details

SubjectRepoBranchLines +/-
operations/deployment-chartsmaster+1 -1
generated-data-platform/datasets/image-suggestionsmain+2 -2
operations/deployment-chartsmaster+2 -2
mediawiki/libs/Shellboxmaster+0 -3
operations/deployment-chartsmaster+2 -2
operations/software/thumbor-pluginsmaster+0 -0
operations/docker-images/production-imagesmaster+0 -243
integration/configmaster+75 -75
integration/configmaster+343 -8
operations/puppetproduction+24 -3
operations/puppetproduction+4 -0
operations/docker-images/production-imagesmaster+57 -9
operations/puppetproduction+13 -10
mediawiki/libs/Shellboxmaster+6 -3
operations/puppetproduction+0 -6
operations/puppetproduction+9 -2
operations/puppetproduction+0 -22
operations/puppetproduction+6 -4
operations/puppetproduction+1 -0
operations/puppetproduction+1 -0
operations/puppetproduction+2 -0
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Production images rebuild is done:

Apr 15 13:46:55 build2001 update-production-images[4127624]: Successfully published image docker-registry.discovery.wmnet/php7.4-fpm:7.4.33-4-20240415
Apr 15 13:46:55 build2001 update-production-images[4127624]: Successfully published image docker-registry.discovery.wmnet/php7.4-fpm-multiversion-base:7.4.33-9-20240415
Apr 15 13:46:55 build2001 update-production-images[4127624]: Successfully published image docker-registry.discovery.wmnet/php7.4-cli:7.4.33-1-s2-20240415
akosiaris lowered the priority of this task from Unbreak Now! to High.Apr 15 2024, 2:34 PM

The immediate issue blocking the train has been resolved and new images have been pushed. Hence, lowering to High. There's a tail of images being rebuilt still and it's going to take a while longer, but this is no longer a UBN

Change #1019730 merged by Muehlenhoff:

[operations/puppet@production] beta::mediawiki_packages: Install lilypond from component

https://gerrit.wikimedia.org/r/1019730

Change #1019721 merged by Muehlenhoff:

[operations/puppet@production] Remove obsolete apt::pin for buster-backports

https://gerrit.wikimedia.org/r/1019721

Change #1019716 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] blubber: Use lilypond component

https://gerrit.wikimedia.org/r/1019716

The following images fail docker-reporter checks because they haven't been rebuilt on top of the new buster base image:

base images
docker-registry.wikimedia.org/docker-gc:1.0.0-20230402              [FAIL]
docker-registry.wikimedia.org/golang:1.14-1-20240407                [FAIL]
docker-registry.wikimedia.org/httpd-fcgi:2.4.38-10-u5-20240407      [FAIL]
docker-registry.wikimedia.org/kubeflow-kfserving-agent:0.6.0-1-20211017[FAIL]
docker-registry.wikimedia.org/kubeflow-kfserving-controller:0.6.0-1-20211017[FAIL]
docker-registry.wikimedia.org/kubeflow-kfserving-storage-initializer:0.6.0-5-20211010[FAIL]
docker-registry.wikimedia.org/loki:1.5.0-2-20230604                 [FAIL]
docker-registry.wikimedia.org/mediawiki-httpd:0.1.8-s2-20240407     [FAIL]
docker-registry.wikimedia.org/php7.2-cli:0.2.0-s3-20221204          [FAIL]
docker-registry.wikimedia.org/php7.2-fpm:0.4.0-20221204             [FAIL]
docker-registry.wikimedia.org/php7.2-fpm-multiversion-base:1.0.7-20221204[FAIL]
docker-registry.wikimedia.org/php7.4-cli-icu67:7.4.33-1-s2-20231106-20231106[FAIL]
docker-registry.wikimedia.org/php7.4-fpm-icu67:7.4.33-3-20231106-20231106[FAIL]
docker-registry.wikimedia.org/wikimedia-buster:20210523             [FAIL]
service images
docker-registry.wikimedia.org/wikimedia/blubber:stable              [FAIL]
docker-registry.wikimedia.org/wikimedia/generated-data-platform-datasets-image-suggestions:stable[FAIL]
docker-registry.wikimedia.org/wikimedia/labs-libraryupgrader:web    [FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki:wmf-1.37.0-wmf.17 [FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox:icu67[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-apple-search:production[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-example-node-api:2021-11-09-141841-candidate[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api:2023-01-31-172753-production[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-kartotherian:kartotherian[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-similar-users:2022-08-17-104210-production[FAIL]
docker-registry.wikimedia.org/wikimedia/operations-container-miscweb:fifteenwp[FAIL]
docker-registry.wikimedia.org/wikimedia/operations-software-thumbor-plugins:2023-10-06-153556-production[FAIL]
docker-registry.wikimedia.org/wikimedia/research-mwaddlink:test     [FAIL]
docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater:2023-02-16-151433-production[FAIL]
docker-registry.wikimedia.org/wikimedia/wikimedia-portals:2024-04-15-124732-production[FAIL]

This has also broken building CI images. Will have to migrate them to bullseye immediately, I suppose.

This has also broken building CI images. Will have to migrate them to bullseye immediately, I suppose.

See my comment above: T362518#9712748

This has also broken building CI images. Will have to migrate them to bullseye immediately, I suppose.

To unblock the current images I created a repository component/shellcheck on apt.wikimedia.org and imported the version of shellcheck which was previously on buster-backports, with that we can easily unblock the current images.

Change #1019726 merged by Muehlenhoff:

[operations/puppet@production] Only install Go from backports on bullseye-based stat hosts

https://gerrit.wikimedia.org/r/1019726

This has also broken building CI images. Will have to migrate them to bullseye immediately, I suppose.

See my comment above: T362518#9712748

Yes, sure, those were also broken, but as discussed yesterday in IRC my concern is about quibble images which are unbuildable at the first apt step (apt-get install --yes build-essential unzip python --no-install-recommends fails with The repository 'http://mirrors.wikimedia.org/debian buster-backports Release' does not have a Release file.).

Nothing that I'm blocked on scap development/testing in train-dev due to docker-registry.wikimedia.org/mediawiki-httpd:latest getting the The repository 'http://mirrors.wikimedia.org/debian buster-backports Release' no longer has a Release file error during apt-get update. (T362931 (now merged into this ticket))

httpd-fcgi + dependent images seem to not have successfully rebuild on Monday. checking.

Successfully published image docker-registry.discovery.wmnet/httpd-fcgi:2.4.38-10-u5-20240415
Successfully published image docker-registry.discovery.wmnet/mediawiki-httpd:0.1.8-s2-20240415

docker run --rm -it --pull=always --user 0 --entrypoint /bin/bash docker-registry.wikimedia.org/mediawiki-httpd:latest -c "apt-get update" LGTM

Change #1021877 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] build-bare-slim: Stop building wikimedia-buster

https://gerrit.wikimedia.org/r/1021877

Change #1021878 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/docker-images/production-images@master] Replace wikimedia-buster base images with buster

https://gerrit.wikimedia.org/r/1021878

Change #1022215 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] codsearch: use thirdparty-ci repo to get docker-ce on buster

https://gerrit.wikimedia.org/r/1022215

According to CodeSearch, affected images are:

Codesearch didn't find itself ?:)

https://gerrit.wikimedia.org/r/c/operations/puppet/+/1022215

Change #1021878 merged by JMeybohm:

[operations/docker-images/production-images@master] Replace wikimedia-buster base images with buster

https://gerrit.wikimedia.org/r/1021878

Change #1021877 merged by Clément Goubert:

[operations/puppet@production] build-bare-slim: Date tag wikimedia-buster images

https://gerrit.wikimedia.org/r/1021877

Change #1022215 abandoned by Dzahn:

[operations/puppet@production] codsearch: use thirdparty-ci repo to get docker-ce on buster

Reason:

per comments above, real fix is to stop using buster

https://gerrit.wikimedia.org/r/1022215

Change #1023853 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: rm buster-backports and use component/shellcheck

https://gerrit.wikimedia.org/r/1023853

Change #1023853 merged by jenkins-bot:

[integration/config@master] dockerfiles: rm buster-backports and use component/shellcheck

https://gerrit.wikimedia.org/r/1023853

Mentioned in SAL (#wikimedia-releng) [2024-04-24T13:45:14Z] <hashar> Building Buster CI images to get rid of buster-backports | T362518 | https://gerrit.wikimedia.org/r/1023853

Change #1023870 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update jobs for buster-backports removal

https://gerrit.wikimedia.org/r/1023870

Change #1023870 merged by jenkins-bot:

[integration/config@master] jjb: update jobs for buster-backports removal

https://gerrit.wikimedia.org/r/1023870

Change #1026976 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/docker-images/production-images@master] Remove golang 1.14

https://gerrit.wikimedia.org/r/1026976

Change #1026976 merged by Elukey:

[operations/docker-images/production-images@master] Remove golang 1.14

https://gerrit.wikimedia.org/r/1026976

Still left to either remove or rebuild:

  • base images
docker-registry.wikimedia.org/docker-gc:1.0.0-20230402              [FAIL]
docker-registry.wikimedia.org/golang:1.14-1-20240407                [FAIL]
docker-registry.wikimedia.org/kubeflow-kfserving-agent:0.6.0-1-20211017[FAIL]
docker-registry.wikimedia.org/kubeflow-kfserving-controller:0.6.0-1-20211017[FAIL]
docker-registry.wikimedia.org/kubeflow-kfserving-storage-initializer:0.6.0-5-20211010[FAIL]
docker-registry.wikimedia.org/loki:1.5.0-2-20230604                 [FAIL]
docker-registry.wikimedia.org/php7.2-cli:0.2.0-s3-20221204          [FAIL]
docker-registry.wikimedia.org/php7.2-fpm:0.4.0-20221204             [FAIL]
docker-registry.wikimedia.org/php7.2-fpm-multiversion-base:1.0.7-20221204[FAIL]
docker-registry.wikimedia.org/php7.4-cli-icu67:7.4.33-1-s2-20231106-20231106[FAIL]
docker-registry.wikimedia.org/php7.4-fpm-icu67:7.4.33-3-20231106-20231106[FAIL]
  • service images
docker-registry.wikimedia.org/wikimedia/blubber:stable              [FAIL]
docker-registry.wikimedia.org/wikimedia/generated-data-platform-datasets-image-suggestions:stable[FAIL]
docker-registry.wikimedia.org/wikimedia/labs-libraryupgrader:web    [FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki:wmf-1.37.0-wmf.17 [FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox:icu67[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-apple-search:production[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-example-node-api:2021-11-09-141841-candidate[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api:2023-01-31-172753-production[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-kartotherian:kartotherian[FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki-services-similar-users:2022-08-17-104210-production[FAIL]
docker-registry.wikimedia.org/wikimedia/operations-container-miscweb:fifteenwp[FAIL]
docker-registry.wikimedia.org/wikimedia/operations-software-thumbor-plugins:2023-10-06-153556-production[FAIL]
docker-registry.wikimedia.org/wikimedia/research-mwaddlink:test     [FAIL]
docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater:2023-02-16-151433-production[FAIL]

Still left to either remove or rebuild:

  • base images
docker-registry.wikimedia.org/docker-gc:1.0.0-20230402              [FAIL]

OK to delete. 1.2.0 is current.

  • service images
docker-registry.wikimedia.org/wikimedia/blubber:stable              [FAIL]
docker-registry.wikimedia.org/wikimedia/mediawiki:wmf-1.37.0-wmf.17 [FAIL]

OK to delete.

Just to be completely sure before deleting

Still left to either remove or rebuild:

  • base images
docker-registry.wikimedia.org/docker-gc:1.0.0-20230402              [FAIL]

OK to delete. 1.2.0 is current.

This is replaced by docker-registry.wikimedia.org/repos/releng/docker-gc/docker-gc and I can remove the image that is not in the releng namespace completely?

  • service images
docker-registry.wikimedia.org/wikimedia/blubber:stable              [FAIL]

This is replaced by docker-registry.wikimedia.org/repos/releng/blubber and I can remove the image that is not in the releng namespace completely?

docker-registry.wikimedia.org/wikimedia/mediawiki:wmf-1.37.0-wmf.17 [FAIL]

This is completely deprecated and/or replaced by docker-registry.wikimedia.org/wikimedia/mediawiki-core and I can remove the image completely?

Just to be completely sure before deleting

Still left to either remove or rebuild:

  • base images
docker-registry.wikimedia.org/docker-gc:1.0.0-20230402              [FAIL]

OK to delete. 1.2.0 is current.

This is replaced by docker-registry.wikimedia.org/repos/releng/docker-gc/docker-gc and I can remove the image that is not in the releng namespace completely?

Confirmed.

  • service images
docker-registry.wikimedia.org/wikimedia/blubber:stable              [FAIL]

This is replaced by docker-registry.wikimedia.org/repos/releng/blubber and I can remove the image that is not in the releng namespace completely?

Confirmed.

docker-registry.wikimedia.org/wikimedia/mediawiki:wmf-1.37.0-wmf.17 [FAIL]

This is completely deprecated and/or replaced by docker-registry.wikimedia.org/wikimedia/mediawiki-core and I can remove the image completely?

Production mediawiki repos are:

docker-registry.discovery.wmnet/restricted/mediawiki-multiversion
docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-debug
docker-registry.discovery.wmnet/restricted/mediawiki-webserver

These are old/experimental repos that can be deleted:

docker-registry.wikimedia.org/wikimedia/mediawiki
docker-registry.wikimedia.org/wikimedia/mediawiki-core
docker-registry.wikimedia.org/wikimedia/mediawiki-core-pipeline-wmf-publish

Just to be completely sure before deleting

Still left to either remove or rebuild:

  • base images
docker-registry.wikimedia.org/docker-gc:1.0.0-20230402              [FAIL]

OK to delete. 1.2.0 is current.

This is replaced by docker-registry.wikimedia.org/repos/releng/docker-gc/docker-gc and I can remove the image that is not in the releng namespace completely?

Confirmed.

Deleted.

  • service images
docker-registry.wikimedia.org/wikimedia/blubber:stable              [FAIL]

This is replaced by docker-registry.wikimedia.org/repos/releng/blubber and I can remove the image that is not in the releng namespace completely?

Confirmed.

Deleted.

docker-registry.wikimedia.org/wikimedia/mediawiki:wmf-1.37.0-wmf.17 [FAIL]

This is completely deprecated and/or replaced by docker-registry.wikimedia.org/wikimedia/mediawiki-core and I can remove the image completely?

Production mediawiki repos are:

docker-registry.discovery.wmnet/restricted/mediawiki-multiversion
docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-debug
docker-registry.discovery.wmnet/restricted/mediawiki-webserver

These are old/experimental repos that can be deleted:

docker-registry.wikimedia.org/wikimedia/mediawiki
docker-registry.wikimedia.org/wikimedia/mediawiki-core
docker-registry.wikimedia.org/wikimedia/mediawiki-core-pipeline-wmf-publish

Deleted.

I plan on removing the following images:

  • docker-registry.wikimedia.org/golang, we've moved on to docker-registry.wikimedia.org/golang$version images and this image isn't used anymore
  • docker-registry.wikimedia.org/php7.2-cli, docker-registry.wikimedia.org/php7.2-fpm, docker-registry.wikimedia.org/php7.2-fpm-multiversion-base, the image definition was removed
  • docker-registry.wikimedia.org/kubeflow-kfserving-agent, docker-registry.wikimedia.org/kubeflow-kfserving-controller, docker-registry.wikimedia.org/kubeflow-kfserving-storage-initializer, the image definition was removed @elukey can you confirm?
  • docker-registry.wikimedia.org/loki, the image definition was removed @colewhite can you confirm?

No objection to removing the loki image from me.

Deleted docker-registry.wikimedia.org/golang, docker-registry.wikimedia.org/php7.2-cli, docker-registry.wikimedia.org/php7.2-fpm, docker-registry.wikimedia.org/php7.2-fpm-multiversion-base, docker-registry.wikimedia.org/loki

I plan on removing the following images:

+1!

Change #1038248 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[generated-data-platform/datasets/image-suggestions@main] README.md: Update go version

https://gerrit.wikimedia.org/r/1038248

Change #1038251 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/deployment-charts@master] miscweb: Use a random miscweb image for default value

https://gerrit.wikimedia.org/r/1038251

Change #1038256 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/software/thumbor-plugins@master] Empty commit to trigger rebuild

https://gerrit.wikimedia.org/r/1038256

docker-registry.wikimedia.org/kubeflow-kfserving-agent, docker-registry.wikimedia.org/kubeflow-kfserving-controller, docker-registry.wikimedia.org/kubeflow-kfserving-storage-initializer removed.

@akosiaris Do we need to trigger a rebuild of all the icu67 images?

@Eevans docker-registry.wikimedia.org/wikimedia/generated-data-platform-datasets-image-suggestions needs to be rebuilt through this CR, then the service deployment needs to be updated to use the new version.

@taavi As far as I can tell, https://docker-registry.wikimedia.org/wikimedia/labs-libraryupgrader/tags/ is deprecated in favor of https://docker-registry.wikimedia.org/repos/ci-tools/libup/web/tags/, can you confirm?

docker-registry.discovery.wmnet/wikimedia/mediawiki-services-apple-search removed (service was undeployed and the repository is archived)

docker-registry.wikimedia.org/wikimedia/mediawiki-services-example-node-api I'm not sure should actually be built as it seems to be a skeleton example project.

@dancy @Tgr docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api needs to be rebuilt through this CR

docker-registry.wikimedia.org/wikimedia/mediawiki-services-similar-users is being undeployed T345274: Remove similar-users service from k8s

@Dzahn docker-registry.wikimedia.org/wikimedia/operations-container-miscweb is apparently deprecated, this CR changes the default for the miscweb chart, once merged I'll remove it.

@hnowlan docker-registry.wikimedia.org/wikimedia/operations-software-thumbor-plugins needs to be rebuilt through this CR, then the service deployment needs to be updated to use the new version.

@dcausse docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater seems to be deprecated in favor of docker-registry.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater, can you confirm?

@dancy @Tgr docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api needs to be rebuilt through this CR

No idea who owns that service these days; maybe @Urbanecm_WMF would know.
I note the build fails with

/srv/service/node_modules/jsonc-parser/lib/umd/impl/format.js:52
            indentValue = string_intern_1.cachedSpaces[options.tabSize || 4] ?? repeat(string_intern_1.cachedSpaces[1], options.tabSize || 4);
                                                                              ^
SyntaxError: Unexpected token ?

(wrong node version?)

@dancy @Tgr docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api needs to be rebuilt through this CR

No idea who owns that service these days; maybe @Urbanecm_WMF would know.

Based on the README, this appears to be the legacy image suggestions service, which used to run at image-suggestion-api.wmcloud.org. Is that right? If so, I think we can archive the repo and drop the containers rather than rebuilding. If this is behind the production image-suggestion service, then this definitely needs to be fixed, and would be something for Structured-Data-Backlog.

FWIW, I already gave the thumbs up for archival from Growth's perspective at T344740: Migrate image-suggestion-api to GitLab, but it looks like no change was done. @BPirkle was involved in that discussion – pinging here as well.

@dancy @Tgr docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api needs to be rebuilt through this CR

No idea who owns that service these days; maybe @Urbanecm_WMF would know.

Based on the README, this appears to be the legacy image suggestions service, which used to run at image-suggestion-api.wmcloud.org. Is that right? If so, I think we can archive the repo and drop the containers rather than rebuilding. If this is behind the production image-suggestion service, then this definitely needs to be fixed, and would be something for Structured-Data-Backlog.

FWIW, I already gave the thumbs up for archival from Growth's perspective at T344740: Migrate image-suggestion-api to GitLab, but it looks like no change was done. @BPirkle was involved in that discussion – pinging here as well.

The production image-suggestion service is using the wikimedia/generated-data-platform-datasets-image-suggestions image (whose build fails as well), so yes, it looks like it's the image for image-suggestion-api.wmcloud.org

@dcausse docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater seems to be deprecated in favor of docker-registry.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater, can you confirm?

Yes (all the images under docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater should no longer be used and can be safely removed if needed)

@dcausse docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater seems to be deprecated in favor of docker-registry.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater, can you confirm?

Yes (all the images under docker-registry.wikimedia.org/wikimedia/wikidata-query-flink-rdf-streaming-updater should no longer be used and can be safely removed if needed)

Removed, thanks.

@dancy @Tgr docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api needs to be rebuilt through this CR

No idea who owns that service these days; maybe @Urbanecm_WMF would know.

Based on the README, this appears to be the legacy image suggestions service, which used to run at image-suggestion-api.wmcloud.org. Is that right? If so, I think we can archive the repo and drop the containers rather than rebuilding. If this is behind the production image-suggestion service, then this definitely needs to be fixed, and would be something for Structured-Data-Backlog.

FWIW, I already gave the thumbs up for archival from Growth's perspective at T344740: Migrate image-suggestion-api to GitLab, but it looks like no change was done. @BPirkle was involved in that discussion – pinging here as well.

The production image-suggestion service is using the wikimedia/generated-data-platform-datasets-image-suggestions image (whose build fails as well), so yes, it looks like it's the image for image-suggestion-api.wmcloud.org

If Growth is done with the old nodejs-based image-suggestion-api, then it can go away now. Nikki and I worked on that ages ago, and it was superseded by something else. I switched to other work, so I can't say what production may be using these days, but I'm confident it isn't that.

wikimedia/generated-data-platform-datasets-image-suggestions looks like a different codebase.

If Growth is done with the old nodejs-based image-suggestion-api, then it can go away now. Nikki and I worked on that ages ago, and it was superseded by something else. I switched to other work, so I can't say what production may be using these days, but I'm confident it isn't that.

FWIW, the wmcloud service is down for quite a while now. So yes, that container can definitely go away. Filled T366534: Archive mediawiki/services/image-suggestion-api to trigger archival as well.

docker-registry.wikimedia.org/wikimedia/mediawiki-services-image-suggestion-api deleted, thanks.

Change #1038256 abandoned by Clément Goubert:

[operations/software/thumbor-plugins@master] Empty commit to trigger rebuild

Reason:

Superseded by Ica8b989fad1669c10cbc9f7bfe614566f666f99f

https://gerrit.wikimedia.org/r/1038256

Change #1038251 merged by jenkins-bot:

[operations/deployment-charts@master] miscweb: Use a random miscweb image for default value

https://gerrit.wikimedia.org/r/1038251

docker-registry.discovery.wmnet/wikimedia/operations-container-miscweb deleted

docker-registry.wikimedia.org/php7.4-cli-icu67 and docker-registry.wikimedia.org/php7.4-fpm-icu67 deleted, all base images should be clean now.

Service images:

  • Deleted docker-registry.wikimedia.org/wikimedia/mediawiki-services-example-node-api:2021-11-09-141841-candidate, the built image is very old and it's an example repo.
  • docker-registry.wikimedia.org/wikimedia/analytics-datahub:2022-02-18-143157-production looks deprecated
  • docker-registry.wikimedia.org/wikimedia/generated-data-platform-datasets-image-suggestions:stable waiting for CR to be merged, I don't have merge rights there
  • docker-registry.wikimedia.org/wikimedia/labs-libraryupgrader:web waiting for confirmation of deprecation
  • docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox:icu67 CR up for review
  • docker-registry.wikimedia.org/wikimedia/mediawiki-services-similar-users:2022-08-17-104210-production will delete after service turndown in T345274

Change #1041056 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] blubber: Stop building icu67 variant

https://gerrit.wikimedia.org/r/1041056

Change #1041122 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/deployment-charts@master] shellbox: Bump shellbox image versions

https://gerrit.wikimedia.org/r/1041122

Change #1041122 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox: Bump shellbox image versions

https://gerrit.wikimedia.org/r/1041122

  • docker-registry.wikimedia.org/wikimedia/labs-libraryupgrader:web waiting for confirmation of deprecation

Yup, replaced by docker-registry.discovery.wmnet/repos/ci-tools/libup/web now – see https://gitlab.wikimedia.org/repos/ci-tools/libup/-/jobs/278875

Deleted docker-registry.wikimedia.org/wikimedia/labs-libraryupgrader:web, merged change to stop building docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox:icu67 and removed it.

Deleted docker-registry.wikimedia.org/wikimedia/analytics-datahub

Deleted docker-registry.wikimedia.org/wikimedia/mediawiki-services-similar-users following resolution of T345274

Change #1038248 merged by Eevans:

[generated-data-platform/datasets/image-suggestions@main] Update go version

https://gerrit.wikimedia.org/r/1038248

Change #1042188 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/deployment-charts@master] image-suggestion: Update image

https://gerrit.wikimedia.org/r/1042188

Change #1042188 merged by jenkins-bot:

[operations/deployment-charts@master] image-suggestion: Update image

https://gerrit.wikimedia.org/r/1042188